Stumbling upon useful information on the web (whether intentional or not) is a critical part of all our daily workflow. What you choose to do with those resources and how you later choose to leverage them directly correlates with the efficiency of your process.
Some of us add helpful sites to our Bookmarks, save webpage snippets to our Pocket or Evernote, &/or frequently exercise our copy/paste fingers. Others, well, others extract the data and create a searchable database of its intel…
When I first learned of web scraping/data mining, my internal “Hacker Alert!” alarm went off. I quickly learned, however, that (ethically) scraping a web page’s data is far from hacking – in fact, coding knowledge isn’t even required!
Meet Data Miner… a FREE Chrome extension that helps you scrape information from ANY website into a spreadsheet. Extraction criteria is defined by a set of instructions called a recipe, which specifies where key information is contained on a site, as well as how to transform it into a desirable, structured format.
In addition to the 50,000+ pre-made, public recipes that Data Miner’s 90,000+ users have made available for 1-click scraping on 15,000+ websites, Data Miner also offers CUSTOM recipe creation using their FREE Chrome extension, Recipe Creator.
Recipe Creator‘s user-friendly interface guides you through the recipe creation process via Information snippets and accessible video tutorials.
Clicking the Help icon reroutes you to Data Miner’s Live Support, whose offerings include talking with an expert, screen sharing, participating in group learning, walking through live demos, & even creating (paid) recipes.
Data Miner also provides a learning resource, Getting Started with Data Miner, that features step-by-step “How to” instructions for Scraping Web Pages and Basic Tables, Multi Page Scraping and Automation, & Writing Custom Recipes. Video tutorials accompany these lessons, as well (all of which are housed on the Data Miner YouTube Channel).
Tips for Creating Custom Recipes – Choosing a Unique Identifier for a Selected Element
- Begin by checking the fewest element attribute boxes possible.
- Don’t be afraid to play around with the “Select Parent” functionality.
- Access your browser’s Developer Tools to inspect the selected element and cross-reference Data Miner’s Recipe Creator Selector Tricks, Tips, & Scenarios.
Examples:
Other Tips – Utilize Data Miner’s Community!
- Bounce questions and ideas off the Data Miner User Community’s Facebook Group.
- Expand your knowledge by following @DataMinerIO on Twitter.
- Actually reach out to Data Miner’s Support Team – They have a warm, customer-centric attitude and are more than willing and able to help you!
- Visit Data Miner’s website to explore additional features and resources.
Happy Scraping, y’all!
Hi there – what is the navigation element selector for going from page to page on LinkedIn
Hi Travis! That depends on what you’re trying to paginate on LinkedIn. (LI Basic’s search result pages? LI Recruiter’s search result pages? The profile pages associated with a LI Recruiter search? Etc.) Feel free to reach out to me directly or post in the Data Miner user community (https://www.facebook.com/groups/DataMinerIO) for clarification. Thanks!