Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Client-side full-text search #568

Open
srid opened this issue Feb 24, 2021 · 6 comments
Open

Client-side full-text search #568

srid opened this issue Feb 24, 2021 · 6 comments
Labels
NeuronNG/Triage Web Interface UI related

Comments

@srid
Copy link
Owner

srid commented Feb 24, 2021 •
edited
Loading

Depends on Relates to #567

Checkout how mdBook does full-text search: https://rust-lang.github.io/mdBook/ ... it seems to be using http://elasticlunr.com/docs/index.html
@srid srid added Web Interface UI related help wanted Extra attention is needed labels Feb 24, 2021
@flyinggrizzly
Copy link
Contributor

flyinggrizzly commented Feb 27, 2021 •
edited
Loading

I can maybe look at this--I'm not a search expert, but the ElasticLunr docs look pretty friendly.

The issue says this depends on the seemingly more basic reinstatement of client-side search, and I'm understanding the idea here is to use the cache.json file to build up an ElasticLunr index, and then use it for search. Is that about right?

I'm not a Haskell developer, so I won't be much use in the first step, but once there is a cache file, I could pick this up to start it rolling.

And if so, my thought would be to:

  • spike out building the index using vanilla JS, in the browser
  • spike out the searcher using vanilla JS

Then, once the general approach is validated

  • look into TypeScript (one of my primary languages), or PureScript (unfamiliar, but I imagine I could pick it up) for the searcher
  • look at extending and tuning searcher functionality

And, finally, if generating the search index in the client is too slow, investigate what moving that to a server-side (pre)-render might look like.

Given that this depends on #567, please feel no pressure from me. I'll start looking at mdBook and ElasticLunr in the meantime--they look quite interesting.

@srid
Copy link
Owner Author

srid commented Feb 27, 2021

@flyinggrizzly Feel free to to prototype a working version. I haven't done any research on this, and I don't know how the two issues are exactly related, but I imagine we can do even the non-fulltext metadata search (#567) using elasticlunr itself?

/cc @TheMatten who expressed some interest in the PureScript part.

@srid
Copy link
Owner Author

srid commented Feb 27, 2021 •
edited
Loading

What does the elasticlunr index look like? If it is not too complex, I can probably quickly do the index generation part done in Haskell. Otherwise, I imagine you'd fiddle with cache.json ...

By the way, cache.json is already available in the static site, eg: https://neuron.zettel.page/cache.json

@srid srid mentioned this issue Feb 27, 2021
Open
@flyinggrizzly
Copy link
Contributor

flyinggrizzly commented Feb 27, 2021 •
edited
Loading

Oh weird. I must have just had a generation locally where cache.json didn't show up since I was getting 404s fetching it. But yea, if cache.json is available, I can use that to generate an index at page load (I think). I've just opened an issue since neuron gen -wS doesn't serve the cache file locally (#574).

The index document looks structurally pretty straightforward, but the index element itself isn't exactly simple. I haven't looked at the ElasticLunr index generation code, so I don't know how it would look to reimplement it in Haskell.

To start, I would say it'd be better to spike it straight out in JS so we can see what it'll look like in Neuron before committing to replicating search indexing in Haskell.

In terms of getting scripts in, is there an easy way to hook into the document <head> other than adding a head.html partial to the site, or should I just work with that for now?

@flyinggrizzly flyinggrizzly mentioned this issue Feb 27, 2021
Closed
@srid
Copy link
Owner Author

srid commented Feb 27, 2021

The cache.json bug is now fixed.

To start, I would say it'd be better to spike it straight out in JS so we can see what it'll look like in Neuron before committing to replicating search indexing in Haskell.

Sounds good.

In terms of getting scripts in, is there an easy way to hook into the document other than adding a head.html partial to the site, or should I just work with that for now?

I guess it depends on how much HTML you plan on writing. If not much, then perhaps write your JS in a separate file like search.js and then include that in head.html using the <script> tag. I imagine your search.js would add an input element to the DOM, and hook it to do search. If you are writing a bunch of HTML, then maybe you have to write search.html, and if you store that in the static/ folder you can access it at http://localhost:8080/static/search.html.

@flyinggrizzly
Copy link
Contributor

The cache.json bug is now fixed.

Awesome--thank you!

I guess it depends on how much HTML you plan on writing. If not much, then perhaps write your JS in a separate file like search.js and then include that in head.html using the <script> tag. I imagine your search.js would add an input element to the DOM, and hook it to do search. If you are writing a bunch of HTML, then maybe you have to write search.html, and if you store that in the static/ folder you can access it at http://localhost:8080/static/search.html.

That works for me--I'll figure out which is easiest as a starting point, and get started on this in the next couple days.

@flyinggrizzly flyinggrizzly mentioned this issue Feb 28, 2021
Draft
15 tasks
@srid srid added NeuronNG/Triage and removed help wanted Extra attention is needed labels Aug 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeuronNG/Triage Web Interface UI related
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@srid @flyinggrizzly