Recently search electronic yes time found an interesting website, a lot of fine school version of e-books, because of curiosity, want to do a crawler name summary. Search for the title yourself and find the download address. Ten minutes, the script is basically finished, a night time can almost run out.

Share code, just for reference (rough).

package com.fun

import com.fun.db.mysql.MySqlTest
import com.fun.frame.httpclient.FanLibrary
import com.fun.utils.Regex
import org.slf4j.Logger
import org.slf4j.LoggerFactory

class T extends FanLibrary {

    static Logger logger = LoggerFactory.getLogger(T.class)


    public static void main(String[] args) {
//        test(322) def list = 1.. 1000 as List list.each { x -> try {test(x)} Catch (Exception e) {∫ logger.error(x.tostring ()) output(e)} logger.warn(x.tostring ()) sleep(2000)}testOver()
    }

    static def test(int id) {
//        def get = getHttpGet("https://****/books/9798.html")
        def get = getHttpGet("https://****/books/" + id + ".html")
        def response = getHttpResponse(get)
        def string = response.getString("content")
        if (string.contains("The file you requested does not exist.")|| string.contains("Page not found")) return
        output(string)
        def all = Regex.regexAll(string, "class=\"bookpic\"> ).get(0)
        def all2 = Regex.regexAll(string, "Content =\" Content description.*? \ "").get(0)
        def all3 = Regex.regexAll(string, "Title =\" author:.*? \ "").get(0)
        def all40 = Regex.regexAll(string, "https://sobooks\\.cc/go\\.html\\? Url = HTTPS {0, 1} : / /. *? \\.ctfile\\.com/.*?\"")
        def all4 = all40.size() == 0 ? "" : all40.get(0)
        def all50 = Regex.regexAll(string, "https://sobooks\\.cc/go\\.html\\? Url = HTTPS {0, 1} : / / pan\\.baidu\\.com/. *? \ "")
        def all5 = all50.size() == 0 ? "" : all50.get(0)
        output(all)
        output(all2)
        output(all3)
        output(all4)
        output(all5)
        def name = all.substring(all.lastIndexOf("=") + 2, all.length() - 1)
        def author = all3.substring(all3.lastIndexOf("=") + 2, all3.length() - 1)
        def intro = all2.substring(all2.lastIndexOf("=") + 2, all2.length() - 1)
        def url1 = all4 == "" ? "" : all4.substring(all4.lastIndexOf("=") + 1, all4.length() - 1)
        def url2 = all5 == "" ? "" : all5.substring(all5.lastIndexOf("=") + 1, all5.length() - 1)
        output(name, author, intro, url1, url2)
        def sql = String.format("INSERT INTO books (name,author,intro,urlc,urlb,bookid) VALUES (\"%s\",\"%s\",\"%s\",\"%s\",\"%s\",%d)", name, author, intro, url1, url2, id)
        MySqlTest.sendWork(sql)
    }
}
Copy the code

Personally, I am quite satisfied with it.

Technical articles selected

  • Java one line of code to print a heart
  • Linux performance monitoring software Netdata Chinese version
  • Interface Test Code Coverage (JACOCO) solution sharing
  • Performance testing framework
  • How to perform performance tests on a Linux command line interface
  • Graphic HTTP brain map
  • How do I test a probabilistic business interface
  • Httpclient handles multiple online users simultaneously
  • Automatically turn the Swagger document into test code
  • Five lines of code to build a static blog
  • How does HttpClient handle 302 redirects
  • Probe into the Linear interface testing framework based on Java
  • Tcloud cloud measurement platform — the integrator
  • How do I test a probabilistic business interface
  • Python Plotly encapsulates methods that process interface performance test data
  • Single sign-on performance test solution

Selected non-technical articles

  • Why software testing as a career path?
  • 10 steps to becoming a Great Java developer
  • Programming thinking for everyone
  • Barriers to automated testing
  • The problem with automated testing
  • Test the Code Immortal brain map
  • 7 Steps to becoming a Good Automation Test Engineer
  • Attitude of a good software developer
  • How to perform functional API tests correctly
  • New trends in software testing in the next 10 years
  • New trends in software testing in the next 10 years
  • What problems do automated tests solve
  • 17 Common Productivity Skills for Software Testers – Up
  • 17 Productivity Skills used by Software testers – below
  • An important reason for the existence of manual testing
  • 17 Tips for writing test Cases

Higher-ups elegant demeanour

  • Tcloud cloud measurement platform — the integrator
  • Android App test tools and knowledge collection
  • 4399AT UI automation CI with CD
  • Android App routine test content
  • Objects and heaps for the JVM

Click on the map of the official account