Keep an eye on node:vfs

[jacoscaz]

Hello! I'm the maintainer of quadstore, I just saw your messages on the Comunica repo. Just as a heads-up, keep an eye on nodejs/node#61478 . If/when this lands, it will support using the classic-level backend, which is far more performant than memory-level when record count becomes significant, while keeping things in-memory.

[faubulous]

Awesome, thanks for the info @jacoscaz! Let's hope it'll be merged soon. 👍

There's been a reported issue that someone had a very large dataset and it basically crashed Mentor. Now that Mentor builds for Node.js and the Browser at the same time, it would probably be possible to use a more scalable version of the store on the Desktop, right?

[jacoscaz]

@faubulous desktop / Node use would be a lot better served using classic-level on a temp file even without node:vfs. It doesn't take that much to overload memory-level, unfortunately.

[faubulous]

The dual build is actually brand new in Mentor so I did not have the time to look into more platform specific features. But it sounds like a good next step. I'll give it a try. I can imagine that I add an option so people can choose which store backend they want to use. Should be fairly simple.

The only thing I need to consider is that persistence is actually a bit problematic as files in workspaces might be changed in between startups of the IDE and by Git operations.. That's why in-memory is currently the only option. Will temp files in the classic-level be automatically be deleted?

[jacoscaz]

Apologies, I should have been clearer in my writing. By "temp files" I meant pointing classic-level to somewhere in the OS's temp directory. I've never worked on VS Code extensions but I presume there must be a way to hook into their lifecycle to clean up any such temporary folder before the extension is ultimately offloaded / deactivated or the user simply quits the editor.

[faubulous]

Thanks for the clarification! Yes, there is such a hook. I'll give it a try.