Its a good start but there is so much ideas how to grow
Search providers
Each search provider could have own alignment by government (google, yandex, baidu etc) - makes sense give a choice.
Specialized search engines: reddit, Wikipedia, web-archive, youtube transcript
Richer search query lang (file type, ignore kw, require kw)
Richer metadata per search result if provider supports
Scrapers / extractors
Specific extractor types: pdf, images, web-page-source?
Richer metadata: relevance score?
Split: render/grab (playwright) then filter/extract (trafilatura)
Research DB
Large deep research would eventually run out of context, ideas:
- Keep each scraped result with index like
{@source-query, [kwd/tags], "title", "summary", "raw-text"}
- RAG it
- Perform facts 'aggregation'
Results
Allow variable research results output: full, thesis, relevant links, info-graphics
As it turns out - its a command line LMNotebook O_O
Its a good start but there is so much ideas how to grow
Search providers
Each search provider could have own alignment by government (google, yandex, baidu etc) - makes sense give a choice.
Specialized search engines: reddit, Wikipedia, web-archive, youtube transcript
Richer search query lang (file type, ignore kw, require kw)
Richer metadata per search result if provider supports
Scrapers / extractors
Specific extractor types: pdf, images, web-page-source?
Richer metadata: relevance score?
Split: render/grab (playwright) then filter/extract (trafilatura)
Research DB
Large deep research would eventually run out of context, ideas:
{@source-query, [kwd/tags], "title", "summary", "raw-text"}Results
Allow variable research results output: full, thesis, relevant links, info-graphics
As it turns out - its a command line LMNotebook O_O