Selenium Usage Summary (Java version)

Selenium has been used a lot in recent development. Because it has not been used before, I have experienced too many pitfalls

1. Configure the environment

Configuration essentials:

1. Webdriver must match the browser version. For Chrome, Use ChromeDriver and Chrome, and for Firefox, use GeckoDrive and Firefox

2. Support headless: Local development using the MAC environment, supported by default; Linux requires xvF8 installation (virtual GUI)

3. Maven project build, using Selenium-3.9.1 or the latest version

4. Linux Configuration reference: Chrome: blog.csdn.net/qq_39802740… ; Firefox:blog.csdn.net/u014283248/… www.xnathan.com/2017/12/04/…

2. Chromium project use

Chrome startup parameters reference: Pepeter. sh/ Experiments…

1. Set system environment variables to webdriver.chrome.driver=DRIVER_PATH

2. Common options configuration:

– headless	Browserless mode
–no-sandbox	Non-sandbox mode, mandatory for Linux deployment
–disable-gpu	Disable the GPU. Liinux deployment is required to prevent unknown bugs
blink-settings=imagesEnabled=false	Not loading images
– the user-agent value = ua	Set the ua

3. Webdriver instantiation:

// Set system environment variables
System.setProperty("webdriver.chrome.driver", env.getProperty("path.chrome.driver"));
WebDriver webDriver = null;
try{
    ChromeOptions options = new ChromeOptions();
	options.addArguments("--headless"); // No browser mode
	options.addExtensions(new File(env.getProperty("path.chrome.proxy")));// Add proxy extension
    webDriver = new ChromeDriver(options);/ / instantiate
}catch(Exception e){
    e.printStackTrace();
}finally{
    //使用完毕，关闭webDriver
    if(webDriver ! =null){ webDriver.quit(); }}Copy the code

3. Gecko project

1. Set system environment variables to webdriver.gecko.driver=DRIVER_PATH

2. Common options configuration:

– headless	Browserless mode
–no-sandbox	Non-sandbox mode, mandatory for Linux deployment
–disable-gpu	Disable the GPU. Liinux deployment is required to prevent unknown bugs
– the user-agent value = ua	Set the ua

The preference configuration:

permissions.default.image	2	Not loading images

3. Webdriver instantiation:

// Set system environment variables
System.setProperty("webdriver.gecko.driver", env.getProperty("path.gecko.driver"));
WebDriver webDriver = null;
try{
    FirefoxOptions options = new FirefoxOptions();
	options.addArguments("--headless"); // No browser mode
    FirefoxProfile profile = new FirefoxProfile();
	profile.addExtensions(new File(env.getProperty("path.chrome.proxy")));// Add proxy extension
    profile.setPreference("permissions.default.image".2);// Do not display images
    options.setProfile(profile);
	/ / instantiate
    webDriver = new FirefoxDriver(options);
}catch(Exception e){
    e.printStackTrace();
}finally{
    //使用完毕，关闭webDriver
    if(webDriver ! =null){ webDriver.quit(); }}Copy the code

4. Note: Some requests (JS requests, etc.) will be masked by default loading

4.Selenium projects use basic operations

Reference: www.cnblogs.com/linxinmeng/…

WebDriver instantiation: see 2, 3

Driver.get (URL);

2. Close the page: driver.close(); Stop the process: driver.quit();

Load waiting: used during initial page loading or element loading, in three ways:

1. The Thread is forced to sleep, thread.sleep (3000);

Driver.manage ().timeouts().implicitlyWait(3, timeunit.seconds); 3.

3. Explicit wait (controllable wait, recommended). You can set the waiting event and waiting time for a single operation

WebDriverWait wait = new WebDriverWait(webDriver, 60);// Wait 60 seconds for initialization
wait.until(ExpectedConditions.presenceOfElementLocated(By.id("xx")));// Wait for the xx element to appear
// Customize wait events
WebElement frame = wait.until((ExpectedCondition<WebElement>) driver -> {
                WebElement element = driver.findElement(By.id("iframepage"));
                return element;
            });
Copy the code

3. Get content

String page = driver.getPageSource();// Get the full information of the page
String url = driver.getCurrentUrl();// Get the current page URL
WebElement element = driver.findElement(By.id("xx"));// Get the element
element.sendKeys("test");// Set the element value
element.click();// Click the element
element.sendKeys(Keys.BACK_SPACE);// Space keyboard event emulation
Copy the code

Iv. Switching Windows, forms and pop-ups:

1. Window operations

String handle = driver.getWindowHandle();// Get the current window handle
Set<String> handles = driver.getWindowHandles();// Get all window handles
// Switch to another window
for(String h : handles){
    if(! h.equals(handle)){ driver.switchTo().window(h);// Switch Windows
    }else{ driver.close(); }}Copy the code

2. The form operation, the frame switch, needs to be obtained layer by layer from the outside in, and can be handled according to pageSource

driver.switchTo().frame("top");// Switch to top Frame
driver.switchTo().frame("inner");// Switch to inner Frame
Copy the code

3. Pop-up operation

Alert alert = webDriver.switchTo().alert();// Get the popover
if(alert ! =null){
    alert.sendKeys("123");// Set input box text. Only one input box text can be set
    alert.getText();// Get popup information
    alert.accept();// Click Ok
    alert.dismiss();// Click Cancel
}
Copy the code

5. Js script execution

String js = "window.open('http://www.baicu.com')";// javascript to open a new page
((JavascriptExecutor)driver).executeScript("");// Execute the js script
Copy the code

5. PageFactory mode

Introduction: Integrated packaging simplifies WebDriver business operation process, using factory mode with FindBy annotations

PageGenerator page = new PageGenerator(driver);// Abstract the page object
// Instantiate the LoginPage object and call the click method
page.GetInstance(LoginPage.class).click("admin"."123456");
Copy the code

LoginPage.java

public class LoginPage extends BasePage{
    private final static String URL = "https://www.testlogin.com";
    @FindBy(id = "login")
    private WebElement login;
    public LoginPage(WebDriver driver) {
        super(driver);
    }
    public Boolean clickLogin(String name, String pwd){
        get(URL);
        click(login);
        return wait(ExpectedConditions.urlContains("main.html")); }}Copy the code

BasePage.java

public class BasePage extends PageGenerator {
    private Long timeOut = 60L;// The default timeout period is 60 seconds
    public BasePage(WebDriver driver) {
        super(driver);
    }
    public <T> void get(String url) {
        driver.get(url);
    }
    public <T> T wait(ExpectedCondition<T> c){
        return wait(c, timeOut);
    }
    public <T> T wait(ExpectedCondition<T> c, Long t){
        WebDriverWait webDriverWait = new WebDriverWait(this.driver, t==null? timeOut:t);try{
            return webDriverWait.until(c);
        }catch (Exception e){
            return null; }}public <T> void click (T elementAttr) {
        if(elementAttr.getClass().getName().contains("By")) {
            driver.findElement((By) elementAttr).click();
        } else{ ((WebElement) elementAttr).click(); }}}Copy the code

PageGenerator.java

public class PageGenerator {
    public WebDriver driver;
    public PageGenerator(WebDriver driver){
        this.driver = driver;
    }
    public  <T extends BasePage> T GetInstance (Class<T> pageClass) {
        try {
            return PageFactory.initElements(driver,  pageClass);
        } catch (Exception e) {
            e.printStackTrace();
            throwe; }}}Copy the code

6. Proxy use

Github.com/lightbody/b…

I. No Auth authentication proxy

String proxyServer = "2:666";
Proxy proxy = new Proxy().setHttpProxy(proxyServer).setSslProxy(proxyServer);
options.setProxy(proxy);
Copy the code

2. Auth authentication agent is required

Use BrowserMobProxy as a proxy (or other proxy)

// Create a local proxy
BrowserMobProxyServer bmpServer = new BrowserMobProxyServer();
bmpServer.setChainedProxy(new InetSocketAddress("proxy.com".222));// Proxy address port
bmpServer.chainedProxyAuthorization("user"."pwd",AuthType.BASIC);// Proxy user name password
bmpServer.setTrustAllServers(true);// Trust all services
bmpServer.start(11112);// Start a local proxy service with port 11112 and access address localhost:11112
// Use the local proxy
String proxyServer = "localhost:11112";
Proxy proxy = new Proxy().setHttpProxy(proxyServer).setSslProxy(proxyServer);
options.setProxy(proxy);
Copy the code

Local agents can be independently distributed, multi-node, and managed using ZK

3. Use your browser to extend extensions

1. Chrome extension: unable to use loading extension in Headless mode, not resolved yet

chromeOptions.addExtensions(new File(env.getProperty("path.chrome.proxy")));// Proxy extensions need to be packaged after the proxy account password is configured in background.js
Copy the code

Reference:

2. Firefox extension: cannot be used. The new Version of Firefox cannot be loaded due to authentication problems

3. Firefox uses close-proxy-authentication plug-in, which requires an older version: www.site-digger.com/html/articl… Addons.thunderbird.net/zh-tw/thund…

Use phantom. Js

Selenium’s new API no longer supports phantom. Js and can use the old API

Other functions of the agency

1. Set a blacklist

2. Set the header

3. The other

7. Encountered pits

1. Slow page loading :(verification required)

Chromium is more process execution, but js engine is a single thread, multiple Windows open at the same time, will only load a page, until the end of the load or open a window to load the next page, reference (blog.csdn.net/ouyanggengc…

Firefox can load multiple Windows at the same time, and some requests are blocked by default

Two, set blacklist: block some webpage loading (set header the same)

1. Through the proxy setting, BrowserMobServer

BrowserMobProxy server = new BrowserMobProxyServer();
server.blacklistRequests("http://.*\\.blacklist.com/.*".200);Method 1: Set a single blacklist
server.setBlacklist();// Set a blacklist
Copy the code

2. Through the expansion of Settings, it is not completely completed temporarily

The BMP agent cannot be connected

1. Use the correct connection mode

/ / error
Proxy proxy = ClientUtil.createSeleniumProxy(server);// The agent host cannot be obtained
/ / right
String proxyServer = "localhost:11112";
Proxy proxy = new Proxy().setHttpProxy(proxyServer).setSslProxy(proxyServer);
Copy the code

2. Use the latest Maven package

4. Firefox is compatible with chrome extensions

Developer.mozilla.org/zh-CN/docs/…

5. When using auth agent, a login box is displayed

1. Options Disable pop-ups

2. Get Alert and close the popover

Six, grid start

1. Start with the Grid: The chromedriver has the same permissions as the Grid

2. Start grid node: – Dwebdriver. Chrome. Driver parameters on – the Java jar behind – jar – Dwebdriver. Chrome. The driver = / Users/apple/Downloads/chromedriver The selenium server – standalone – 3.141.5. Jar – role node – the hub – http://localhost:4444/grid/register browser browserName = chrome

Distributed 8.

Start multiple nodes with grid

Note: It is best to use Geckodriver when using multiple threads on a single node, as the Chromium JS engine is single-threaded

9. Wiki and doc reference arrangement

Gecko official documentation: github.com/mozilla/gec…

Firefox-source-docs.mozilla.org/testing/gec…

Chromium official documentation: chromedriver.chromium.org/getting-sta…

10. Actual combat strategy of the project: with HTTP request, multi-threading, grid use