A Set of Crawlers

Each crawler is built as part of another project. Different crawler techs are used:

• Selenium
• BeautifulSoup
• Scrapy
• Scholarly

Other possible crawlers that may speed up code flow (not used yet):

• serpAPI
• Octoparse

Data Collections

Canadian Top University Researchers Data

Dataset consists of 32,240 records of Google Scholar profiles from researchers affiliated with top 20 universities in Canada. Columns are GUID, full name, list of research interests, university name, and number of total citations per researcher.