Selenium Setup and Install Webdriver with Python
Selenium is a tool for browser automation and testing. Rather than access a website directly with HTTP calls from an API, Selenium allows you to control an actual real browser. The advantage to this is that you will have the capabilities of a real browser and that your scripts will appear to the website to be an actual, real, user controlled browser. This can be helpful for sites that discourage automation or for cases when you want to run a test that will be as close as possible to what an actual end user would do.
In this guide we are going to be using Selenium with the webdriver and Python to automate the Firefox browser.
These instructions are tested and based around an Ubuntu Linux 20.04 environment. This should also work very similarly on Windows and MacOS with slight modifications. We’ve also tested this on MacOS and it works great.
- Firefox - You need to have this installed because this is the browser that we will be automating.
- Python - You will need Python 3 and pip3 installed. We will show you how to do this.
- Java - Even though we are use Python you will still need the JDK ( or at least the JRE ) installed and on your path.
Note that Ubuntu 20.04 comes with Python 3 installed by default. Running ‘python’ will give you Python 2 and running ‘python3’ will give you Python 3.
sudo apt install python3-pip
Install Selenium for Python using pip3:
pip3 install selenium
Download the geckodriver here: Selenium geckodriver download on GitHub
There are packages for Linux, MacOS, and Windows.
You can download directly or just grab the link for the latest version and pull it down with wget. If you use wget you might want to check the download page to make sure you still have the link for the latest version. The example here might not be the latest version at the time you read this.
Pull the geckodriver down with wget and unpack it with the tar command:
wget https://github.com/mozilla/geckodriver/releases/download/v0.29.1/geckodriver-v0.29.1-linux64.tar.gz tar xvfz geckodriver-v0.29.1-linux64.tar.gz
After unpacking you should have a single executable file. Make sure that the directory that you unpacked this file into is on your path. For example, I unpacked it to my home directory and just added that to my path like this:
You might also want to update your path variable in your .bashrc file so that it will be set any time you open a new terminal.
... ... export PATH=$PATH:/home/user1
Here is a short example showing how you can use selenium to query Google:
from selenium import webdriver from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0 from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0 import time driver = webdriver.Firefox() driver.get("http://www.google.com") # get page print( driver.title ) # print page title inputElement = driver.find_element_by_name("q") # find search field inputElement.send_keys("funny cat videos") # send search query inputElement.submit() # submit try: WebDriverWait(driver, 10).until(EC.title_contains("funny cat videos")) # wait for title print( driver.title ) # print title finally: time.sleep(10) # wait a little bit before closing driver.quit() # quit - this will close the browser
You can run this script like this:
We cover everything here in this video: