Fix stability issues

[kaiokendev]

Some other user mentioned they want to integrate this into SillyTavern, so I will fix these issues to get it to a usable state, and add the UI for adding sources

• if I try to run anything else before loading the URL, it I get an index out of range error

• after loading the URL (I can load basically any URL without issue) to clear the error, it does not appear to make new sources if I just copy-paste a bunch of text in

• never tries to chunk the context without the addition of a URL

• When dealing with URLs alone, I tried loading a fairly heavy Reddit thread (using old.reddit) and got the following error: chromadb.errors.NotEnoughElementsException: Number of requested results 3 cannot be greater than number of elements in index 2

• Further attempts to ask questions didn't lead to a reattempt at scanning the URL just use an old URL if prompted

• until the Settings and Sources UI tabs are completed. I'd consider those a high priority for baseline usability. Without it, it's hard to know what sources are being considered without reading the console output

[kaiokendev]

Following issues resolved:

• if I try to run anything else before loading the URL, it I get an index out of range error

This has been fixed

• after loading the URL (I can load basically any URL without issue) to clear the error, it does not appear to make new sources if I just copy-paste a bunch of text in

Added a cache_key for sources based on the md5 hash of the content -- it will attempt to retrieve the contents again, but will not re-chunk them unless the content has changed. However for the second part of this issue, see below

• never tries to chunk the context without the addition of a URL

This has been fixed? If this means it cannot chunk the text content, the text content should be added as a TextSource. It is difficult to infer the parts of context that should be chunked vs the parts that should be used for search. It either has to be manually set via adding a source, or through special infer patterns. Putting the content into the ### Data part was equivalent to adding a TextSource.

• When dealing with URLs alone, I tried loading a fairly heavy Reddit thread (using old.reddit) and got the following error: chromadb.errors.NotEnoughElementsException: Number of requested results 3 cannot be greater than number of elements in index 2

This.. should be fixed? I'm able to load very long webpages fine. The particular error here however is most likely from there being no content to store -- probably because the default User-Agent was blocked. I have fixed both the issue from no content, as well as added a User-Agent of SuperBIG.

• Further attempts to ask questions didn't lead to a reattempt at scanning the URL just use an old URL if prompted

Also resolved via the cache key, and the source selector has been fixed to choose the best source based on the input question

• until the Settings and Sources UI tabs are completed. I'd consider those a high priority for baseline usability. Without it, it's hard to know what sources are being considered without reading the console output

Since this is a Python package now, you would add explicit sources via:

from superbig.source import UrlSource
...
provider.add_source("mysource", UrlSource(url))

TODO:

• Make the search input user-definable, i.e. prompt.set_search_text(...)
• Add methods to provide a view into the settings (RAM usage, speeds, etc)
• Add a ChatChunker that chunks a list of strings (so you can integrate easier with chat backends that pass each message in the chat history as a string, we can't reliably use the NaiveChunker because it won't rejoin the cut partitions properly)

I won't add a UI though, since this is not a frontend, just a package you should use in another frontend of your choice. The display of settings and source UI should be the role of the frontend integration

[digiwombat]

Thanks for the hard work.

Any thoughts on best approach to integrating into JS frontends like SillyTavern, assuming they want to? There are python interop libraries, that'll probably be the only way without someone getting deep in there and porting to JS and duckdb-wasm/hnswlib-node. Those libs do exist though so it might be doable.

[kaiokendev]

2 approaches:

1. Serve it via a lite API wrapper. I will add a simple one today so you don't have to do the work of interop
2. Interop. Honestly, the fetching is fast enough as-is and I haven't really optimized or cleaned up, since that will come later.

Interop will give you much more direct control but you'd have to make your own bindings.

[KuAndroid]

Just want to check in whether this works with Silly Tavern now? And if so, are there instructions available?

[digiwombat]

You'd need to make an issue on their github to ask for direct SillyTavern support.

Currently, there is support via the simple-proxy script that is included in there. It's a bit undocumented, I think. Haven't checked the README lately. But there's a py file in the /src/ folder for simple proxy, run that and edit your config to enable superbig and you're good to go, hypothetically. I haven't tested it much other than verifying that it worked.