Skip to content

UIUC-Chatbot/crawlee

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

UIUC.chat's Web Crawler

Hosted on Railway.app. I found Railway preferable to severless functions because it easily handles the large bundle size introduced by using Chromium for full javascript execution during web crawling.

It's powered by the excellent Crawlee.dev