Selenium provides a wide range of ways to interact with sites, such as: Rarely anything is better in 'talking' to a website than a real, proper browser, right? for taking screenshots), which, of course, also includes the purpose of web crawling and web scraping. In the meantime, however, it has been adopted mostly as a general browser automation platform (e.g. Originally (and that has been about 20 years now!), Selenium was intended for cross-browser, end-to-end testing (acceptance tests). Selenium can control both, a locally installed browser instance, as well as one running on a remote machine over the network. The Selenium API uses the WebDriver protocol to control web browsers like Chrome, Firefox, or Safari. It supports bindings for all major programming languages, including our favorite language: Python. Selenium refers to a number of different open-source projects used for browser automation. Today we are going to take a look at Selenium (with Python ❤️ ) in a step-by-step tutorial.

In the last tutorial we learned how to leverage the Scrapy framework to solve common web scraping tasks.

Chrome driver download