This file is an overview of available sources along with descriptions and some guidance.
Some sources require additional dependencies, which you can install if needed.
pip3 install --user promnesia[optional]dependencies that bring some bells & whistles: logzero, python-magic
pip3 install --user promnesia[HPI]dependencies for HPI: HPI
pip3 install --user promnesia[html]dependencies for sources.html: beautifulsoup4, lxml
pip3 install --user promnesia[markdown]dependencies for sources.markdown: mistletoe
pip3 install --user promnesia[org]dependencies for sources.org: orgparse
pip3 install --user promnesia[telegram]dependencies for sources.telegram: dataset
Alternatively, you can just install all of them in bulk: pip3 install --user promnesia[all].
These are included with the current Promnesia distribution:
- promnesia.sources.auto
- discovers files recursively
- guesses the format (orgmode/markdown/json/etc) by the extension/MIME type
- can index most of plaintext files, including source code!
- autodetects Obsidian vault and adds `obsidian://` app protocol support promnesia.sources.obsidian
- autodetects Logseq graph and adds `logseq://` app protocol support promnesia.sources.logseq
- promnesia.sources.fbmessenger
Uses HPI for the messages data.
- promnesia.sources.github
Uses HPI github module
- promnesia.sources.guess
- promnesia.sources.html
Extracts links from HTML files
- promnesia.sources.hypothesis
Uses HPI hypothesis module
- promnesia.sources.instapaper
Uses HPI instapaper module
- promnesia.sources.pocket
Uses HPI for Pocket highlights & bookmarks
- promnesia.sources.reddit
Uses HPI reddit module
- promnesia.sources.roamresearch
Uses HPI for Roam Research data
- promnesia.sources.rss
- promnesia.sources.shellcmd
- promnesia.sources.smscalls
Uses HPI smscalls module
- promnesia.sources.takeout
Uses HPI google.takeout module
- promnesia.sources.telegram
Uses telegram_backup database for messages data
- promnesia.sources.viber
Uses all local SQLite files found in your Viber Desktop configurations: usually in
~/.ViberPC/**/viber.db(one directory for each telephone number). - promnesia.sources.signal
When path(s) given, uses the SQLite inside Signal-Desktop’s configuration directory (see the sources for more parameters & location of the db-file for each platform)
- promnesia.sources.twitter
Uses HPI for Twitter data.
- promnesia.sources.vcs
Clones & indexes Git repositories (via sources.auto)
- promnesia.sources.website
Clones a website with wget and indexes via sources.auto
Adding your own data sources is very easy! One day I’ll add some proper documentation, but the easiest at the moment is to use existing simple modules as a reference, for example:
- pocket, 25 lines of code
- fbmessenger, 30 lines of code
After that, you can simply import your custom source in config.py, and use it, same way as any builtin source.