By the end of this tutorial, you'll have created a web crawler to scrape data from Market Watch. The crawler will export the data to Google Sheets.
• Simple Scraper
• Market Watch
• Google Sheets
Table Of Contents
1. Initial Scraper Setup
2. Running The Web Scraper
3. Saving The Web Crawler
4. Integrating Google Sheets
5. Running The Web Crawler
Go to Market Watch.
Search for 'AAPL' (or another stock).
Open Simple Scraper and click the plus (+) sign.
First, you'll want to scrape the name: select the stock name. Everything which gets highlighted is what'll get extracted. Name this data, 'Name'. Then, click the tick to set it for when you run the scraper.
Second, you'll want to scrape the yield. Again, click the plus (+) sign. Select the stock's yield. Everything which gets highlighted is what'll get extracted. Name this data, 'Yield'. And, click the tick to set it for when you run the scraper.
To run your scraper, click 'View Results'.
Once the web scraper has run, Simple Scraper will return the selected data. It will allow you to view that data in a table or as a JSON file. And, you will have the option of downloading the data as either a CSV file or JSON.
To save your web crawler, click 'Save Recipe'.
You'll have to confirm the settings for your web crawler when saving it. The settings that got entered for this project are:
Once you've entered the settings, click 'Create Recipe'.
Click on the recipe you saved under 'My Recipes'.
Then, click 'Integrate'.
Select the toggle button to activate Google Sheets export. And, you will want the crawler to 'Replace Previous Results' for the save method.
Go to Market Watch.
Search for 'AAPL'. Copy the URL and paste it into 'Crawl URLs'.
Repeat the process for each stock you want data on. For this project, you'll want data on; Apple, Microsoft, IBM, and Intel.
To run your crawler, click 'Run Recipe'.
Once the web crawler has run, Simple Scraper will return the selected data. You can view the output of your crawler on the 'Results' page.
To view the data in Google Sheets, go to 'Integrate' and click 'Open Google Sheet'. Now, you can remove unnecessary columns and analyse your data.