Get data from any website with Selenium

Description of the web crawler

1. This is a simple Python class to get data from any website without API.
2. Scrapper makes use Selenium library for Python and chromedriver as well(or any other for your web browser).
3. It's easy to use in any browser with XPATH copy and paste. You can use your own XPATH or CSS selectors.

Setup

How to start python script in shell

1. Install python 3.7+ https://www.python.org/downloads/windows/ (or version for you OS)
2. Install pipenv pip install --user pipenv. Docs are here: https://github.com/pypa/pipenv
3. To install all required packages run this command: pipenv install
4. Download Chromedriver for yor own Google Chrome version (or your own, Gecko driver for Firefox etc.), from this page: https://chromedriver.chromium.org and put it next to the main.py file.
5. To start python script: pipenv run py main.py or pipenv run python main.py

License

MIT License

Author

JQ/Quoterbox