Installation and Core Concepts

How Selenium Works

Selenium is a web application automation framework that enables writing programs to interact with web interfaces and extract information from them.

The automation workflow involves these steps:

1. The automation script envokes Selenium client library methods (such as clicking a button element)
2. The client libray sends Selenium commands to the browser driver
3. The browser driver receives commands and instructs the browser to execute them
4. The browser executes the commands
5. The browser driver retrieves the execution results and returns them to the automation script
6. The automation script processes the returned results

Installation

1. Install the client library

pip install selenium

2. Install the browser driver

Different browsers require different drivers, and the driver version must match the browser version.

Chrome driver download: Official Chrome Driver

Quick Start Example

Automating Chrome browser to open Baidu:

from selenium import webdriver

# Create WebDriver instance, specifying Chrome driver path
driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
# Use get method to open the specified URL
driver.get('https://www.baidu.com')
# Close the browser window
driver.quit()

If the driver directory is added to system PATH, you can omit the path:

driver = webdriver.Chrome()

Element Selection Strategies

Finding Elements by ID

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('https://www.baidu.com')
# Find element by ID, returns a WebElement object
element = driver.find_element_by_id('kw')
# Input text into the search box
element.send_keys('Selenium\n')

Finding Elements by Class Name

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('http://cdn1.python3.vip/files/selenium/sample1.html')
# Find elements by class name, returns a list
elements = driver.find_elements_by_class_name('animal')

# Iterate through the list and print text content of each element
for element in elements:
    print(element.text)

Finding Elements by Tag Name

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('http://cdn1.python3.vip/files/selenium/sample1.html')
# Find all elements with tag name 'div', returns a list
elements = driver.find_elements_by_tag_name('div')
for element in elements:
    print(element.text)

Scoped Element Search

You can search for elements within a specific WebElement:

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('http://cdn1.python3.vip/files/selenium/sample1.html')
container = driver.find_element_by_id('container')
# Limit search scope to within the container element
spans = container.find_elements_by_tag_name('span')
for span in spans:
    print(span.text)

Handling Dynamic Elements

When performing web operations, some elements may not appear immediately because the code execution is faster than server response.

If you try to find an element before the search results load, you will get a NoSuchElementException.

Solution: Implicit Waits

Selenium provides implicit waits: when an element is not found, it periodically (every 0.5 seconds) retries finding the element until it is found or the maximum wait time expires, then raises an exception. For find_elements methods, it returns an empty list.

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
# Set implicit wait to 10 seconds
driver.implicitly_wait(10)
driver.get('https://www.baidu.com')
search_box = driver.find_element_by_id('kw')
search_box.send_keys('Selenium\n')
result = driver.find_element_by_id('1')
print(result.text)

Element Interactions

Clicking Elements

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('https://www.baidu.com')
search_box = driver.find_element_by_id('kw')
search_box.send_keys('Selenium')
submit_button = driver.find_element_by_id('su')
# Click the element
submit_button.click()

Input Field Operations

from selenium import webdriver

driver = webdriver.Chrome(r'd:\webdrivers\chromedriver.exe')
driver.get('http://cdn1.python3.vip/files/selenium/test3.html')
input_field = driver.find_element_by_id('input1')
# Clear existing text
input_field.clear()
# Enter new text
input_field.send_keys('Hello')

Retrieving Element Information

1. Get text content

element = driver.find_element_by_id('animal')
print(element.text)

2. Get element attribute

element = driver.find_element_by_id('input_name')
print(element.get_attribute('class'))

3. Get element HTML

# Get outer HTML
element = driver.find_element_by_id('searchtext')
print(element.get_attribute('outerHTML'))

# Get inner HTML
container = driver.find_element_by_id('container')
print(container.get_attribute('innerHTML'))

4. Get input field value

element = driver.find_element_by_id('input')
print(element.get_attribute('value'))

CSS Selector Reference

Basic Usage

# Find single element
driver.find_element_by_css_selector('selector')

# Find all elements
driver.find_elements_by_css_selector('selector')

Child and Descendant Selection

# Direct children: element > child
driver.find_element_by_css_selector('div > span')

# Descendants: element descendant
driver.find_element_by_css_selector('div span')

Attribute-Based Selection

# Exact match
driver.find_element_by_css_selector('[href="http://example.com"]')

# Has attribute (any value)
driver.find_element_by_css_selector('[href]')

# Tag with specific class
driver.find_element_by_css_selector('div[class="SKnet"]')

# Contains substring
driver.find_element_by_css_selector('a[href*="miitbeian"]')

# Starts with
driver.find_element_by_css_selector('a[href^="http"]')

# Ends with
driver.find_element_by_css_selector('a[href$="gov.cn"]')

# Multiple attributes
driver.find_element_by_css_selector('div[class=misc][ctype=gun]')

Group Selection

driver.find_element_by_css_selector('h1, h2, h3')

nth-child Selectors

# nth child
driver.find_element_by_css_selector('li:nth-child(2)')

# nth from end
driver.find_element_by_css_selector('li:nth-last-child(2)')

# nth of type
driver.find_element_by_css_selector('li:nth-of-type(2)')

# even/odd
driver.find_element_by_css_selector('li:nth-child(even)')
driver.find_element_by_css_selector('li:nth-child(odd)')

Sibling Selection

# Adjacent sibling
driver.find_element_by_css_selector('h1 + p')

# All following siblings
driver.find_element_by_css_selector('h1 ~ p')

Frame and Window Management

In HTML, frame and iframe elements embed separate HTML documents. By default, Selenium operations only target the main document, not the embedded content. You must switch context to access elements inside frames.

Switching to Frame

driver.switch_to.frame('frame_name')  # by name or ID
driver.switch_to.frame(frame_element)  # by WebElement

Returning to Main Document

driver.switch_to.default_content()

Switching Between Windows

# Switch to window by handle
driver.switch_to.window(window_handle)

The window_handles attribute returns a list of all window handles in the browser.

Switching to specific window:

for handle in driver.window_handles:
    driver.switch_to.window(handle)
    if 'Target' in driver.title:
        break

Returning to original window:

main_window = driver.current_window_handle
driver.switch_to.window(main_window)

Dropdown Select Elements

Select Class Methods

from selenium.webdriver.support.ui import Select

select = Select(driver.find_element_by_id('select_id'))

# Select by value attribute
select.select_by_value('value')

# Select by index (0-based)
select.select_by_index(1)

# Select by visible text
select.select_by_visible_text('Option Text')

# Deselect by value
select.deselect_by_value('value')

# Deselect by index
select.deselect_by_index(1)

# Deselect by visible text
select.deselect_by_visible_text('Option Text')

# Deselect all
select.deselect_all()

Advanced Techniques

Mouse Actions

Perform right-click, double-click, hover, drag-and-drop using ActionChains:

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Chrome()
driver.implicitly_wait(5)
driver.get('https://www.baidu.com/')
actions = ActionChains(driver)
# Move mouse to element
actions.move_to_element(
    driver.find_element_by_css_selector('#s-usersetting-top')
).perform()

Freezing the Page

To freeze the interface for inspection, run in browser console:

setTimeout(function(){debugger}, 5000)

This pauses execution after 5 seconds, freezing the UI.

Handling Dialogs

Dialog types: Alert, Confirm, and Prompt.

# Get alert text
alert_text = driver.switch_to.alert.text

# Accept (click OK)
driver.switch_to.alert.accept()

# Dismiss (click Cancel)
driver.switch_to.alert.dismiss()

# Send input to prompt
driver.switch_to.alert.send_keys('input text')

Additional Techniques

# Get window size
driver.get_window_size()

# Resize window
driver.set_window_size(800, 600)

# Get window title
print(driver.title)

# Get current URL
print(driver.current_url)

# Take screenshot
driver.get_screenshot_as_file('screenshot.png')

Mobile Device Emulation

from selenium import webdriver

mobile_emulation = {"deviceName": "Nexus 5"}
options = webdriver.ChromeOptions()
options.add_experimental_option('mobileEmulation', mobile_emulation)
driver = webdriver.Chrome(desired_capabilities=options.to_capabilities())
driver.get('http://www.baidu.com')

File Upload

# Locate file input element
file_input = driver.find_element_by_css_selector('input[type=file]')
# Send file path
file_input.send_keys(r'd:\upload.png')

Edge Browser Automation

from selenium import webdriver

driver = webdriver.Edge(r'd:\webdrivers\msedgedriver.exe')
driver.get('https://www.baidu.com')

Electron Application Automation

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.binary_location = r'C:\electronApp.exe'
driver = webdriver.Chrome(r'e:\chromedriver.exe', options=options)

Handling Certificate Errors

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
driver = webdriver.Chrome(options=options)

XPath Selectors

Most modern browsers support XPath 1.0 syntax.

Path Selection

# Absolute path
driver.find_elements_by_xpath('/html/body/div')

# Relative path
driver.find_elements_by_xpath('//div//p')

# Wildcard
driver.find_elements_by_xpath('//div/*')

Attribute-Based Selection

# By ID
driver.find_elements_by_xpath('//*[@id="id_value"]')

# By class
driver.find_elements_by_xpath('//div[@class="class_value"]')

# Has attribute
driver.find_elements_by_xpath('//*[@href]')

# Contains substring
driver.find_elements_by_xpath('//*[contains(@style,"color")]')

# Starts with
driver.find_elements_by_xpath('//*[starts-with(@style,"color")]')

# Ends with - XPath 2.0 only, not supported by browsers

Positional Selection

# nth child of its parent
driver.find_elements_by_xpath('//li[2]')

# nth from last
driver.find_elements_by_xpath('//li[last()-1]')

# First n elements
driver.find_elements_by_xpath('//li[position()<=3]')

# Last n elements
driver.find_elements_by_xpath('//li[position()>=last()-2]')

Group Selection, Parent, and Siblings

# Union/group
driver.find_elements_by_xpath('//h1 | //h2')

# Parent
driver.find_element_by_xpath('//*[@id="china"]/..')

# Multiple levels up
driver.find_element_by_xpath('//*[@id="china"]/../../..')

# Following siblings
driver.find_elements_by_xpath('//*[@class="single_choice"]/following-sibling::*')

# Following siblings of specific type
driver.find_elements_by_xpath('//*[@class="single_choice"]/following-sibling::div')

# Preceding siblings
driver.find_elements_by_xpath('//*[@class="single_choice"]/preceding-sibling::*')

XPath Scope Note

When using XPath on a WebElement, add a dot prefix to limit scope:

# Without dot - searches entire document
container = driver.find_element_by_id('china')
results = container.find_elements_by_xpath('//p')  # Returns ALL p elements

# With dot - searches within container
results = container.find_elements_by_xpath('.//p')  # Returns p elements inside container