|
Simple Scraper

Web Crawling With Simple Scraper

By the end of this tutorial, you will have created a web crawler in Simple Scraper, that will allow you to crawl for data from a website (Books To Scrape).

Updated:
June 9, 2021
|
Length:
7 Minutes
Join 1,000+ readers and get my bi-weekly newsletter. I share work that I've shipped and work that has piqued my curiosity. You'll also get access to all of my resources.
You're in! Check your inbox :)
There's an issue. Try Again :)

Project Resources

Simple Scraper

Books To Scrape

Table Of Contents

1. Setting Up Your Web Scraper

2. Running Your Web Scraper

3. Saving Your Web Scraper

4. Setting Up Your Web Crawler

5. Running Your Crawler

1. Setting Up Your Web Scraper

Go-to Books To Scrape.

Open Simple Scraper, and click the plus (+) sign.

First, you’ll want to scrape the titles: select a title. Everything which gets highlighted is what’ll get extracted. Name this data, ‘Title’. Then, click the tick to set it for when you run the scraper.

Second, you’ll want to scrape the price of each book. Again, click the plus (+) sign. Then, select the price of a book. Everything which gets highlighted is what’ll get extracted. Name this data, ‘Price’. And, click the tick to set it for when you run the scraper.

2. Running Your Web Scraper

To run your scraper, click ‘View Results’.

Once the web scraper has run, Simple Scraper will return the selected data. It will allow you to view that data in a table or as a JSON file. And, you will have the option of downloading the data as either a CSV file or JSON.

3. Saving Your Web Scraper

You must save the settings for your scraper, before configuring your crawler.

To save your scraper, click ‘Save Recipe’.

You’ll have to confirm the settings for your scraper when saving it. The settings that got entered for this project are:

Once you’ve entered the settings, click ‘Create Recipe’.

4. Setting Up Your Web Crawler

Click on the recipe you saved under ‘My Recipes’.

Then, click ‘Crawl’.

Insert the URLs you want to crawl. For this project, they are as follows:

5. Running Your Crawler

To run your scraper, click ‘Run Recipe’.

Once the web crawler has run, Simple Scraper will return the selected data. You can view the output of your crawler on the ‘Results’ page.

You’ll notice that Simple Scraper has crawled through fives pages, and returned the selected data. You’ll get given the option to view that data in a table or as a JSON file. And, you’ll have the option of downloading the data too.


Join 1,000+ readers and get my bi-weekly newsletter. I share work that I've shipped and work that has piqued my curiosity. You'll also get access to all of my resources.
You're in! Check your inbox :)
There's an issue. Try again :)