GitHub - Decodo/reddit-python-scraper: Scrape Reddit utilising Decodo's Web Scraping API
Scrape Reddit utilising Decodo's Web Scraping API
Dependencies
Authentication
Once you have an active Web Scraping API subscription, you can try sending a request right from the dashboard Web Scraping API > API playground method tab simply by clicking on Send Request. You will also see an example of a curl request generated on the right.
This Pyhton code example uses Base64 encoded user:pass authentication.
| Parser type | Example location | Download |
|---|---|---|
| HTML to JSON | reddit_python_scraper.py | curl https://raw.githubusercontent.com/Decodo/reddit-python-scraper/blob/main/reddit_python_scraper.py > reddit_python_scraper.py |
HTML to JSON
This Python script extracts Subreddit details, post data, and comments straight from the HTML of Reddit post pages and saves them to a JSON file.