Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IndexingQueue to handle indexing tasks sequentially #1593

Closed
wants to merge 52 commits into from

Conversation

priyashpatil
Copy link
Contributor

Description

  • Introduce IndexingQueue class
  • Update core.ts to use IndexingQueue

Checklist

  • The base branch of this PR is dev, rather than main
  • The relevant docs, if any, have been updated or created

sestinj and others added 30 commits June 14, 2024 09:11
* fix(extensions): schema for db ctx provider

* fix: invalid enum
* 👷 CI for jetbrains

* default working dir

* changelog

* build binaries

* binary testing setup

* idesettings

* core binary testing

* run binary tests in ci

* remove unused targets

* needs build

* console.log bin contents

* fix ci

* fix win32 binary download

* test

* no linux arm64

* macos latest

* macos-12

* binary permissions

* upload logs

* fix

* upload full folder as binary artifact

* test

* test macos only

* set full execute permissions

* copy sqlite binary

* cd

* it worked!

* build again in test job

* debug

* remove timeout

* info

* log

* log2

* more logs

* catch

* fewer logs

* test all platforms

* test downloaded artifact

* needs build

* updates

* build

* false

* release

* add tag and upload binaryes

* change tag name

* proper artifact upload

* jest updates

* ✅ generate a few unit tests with Continue

* fix imports related to IdeSettings

* run tsc on PRs

* remove shareSession command (unused)

* update release process

* update plugin version

* don't show ghost text when jetbrains completion visible

* run jetbrains ci in main
* ✨ use and cache imports for autocomplete

* fix tsc
Patrick-Erichsen and others added 22 commits June 24, 2024 14:21
* feat: toast notification for config updates

* feat: only trigger toast on config.json save
* feat: open pane on activation

* comment out testing code
* doc: prompt file typo + clarifications

* fix: add back correct docs
* chore(gui): remove unused pages

* feat: add embeddings step

* feat: update styles

* feat: copy button updates

* fix: correct pull command for embed model

* fix: remove commented code

* fix: remove commented code

* feat: simplify copy btn props

* chore: rename onboarding selection event

* feat: add provider config

* fix: undo msg name

* remove dead code

* fix: invalid mode check
…uedev#1584)

- Introduce reranker concept
- List available reranker options
- Provide configuration instructions
- Update keywords to include "reranker"
- Added NPM script method
- Added VS Code task method
- Update contributing guidelines
- Change PR target to `dev` branch
- Update `CONTRIBUTING.md` instructions
…ontinuedev#1579)

- Moved examples to configuration.md
- Deleted the separate examples.md file
- Updated sidebar order and links
- Improved readability and structure in configuration.md
- Introduce IndexingQueue class
- Update core.ts to use IndexingQueue
- Enhance IndexingProgressBar UI
- Update IndexingProgressUpdate interface
@sestinj
Copy link
Contributor

sestinj commented Jun 28, 2024

I think we're going to want to do something slightly different here. The biggest concern for me is that the docs indexing won't happen until main indexing has completed, which might force the user to wait some time.

The alternative that I think would be ideal is to keep a "stack" of currently running jobs. They would all run in parallel, but all updates would be attached to a certain "job id" and the last created job id would always be the one displayed in the loading bar, until it was done, at which point the job beneath it, if still progressing, would take over. On hover, it could display the full stack of current jobs.

I think this would require a) updating the IndexingProgressUpdate type to include a "jobId", b) setting that "jobId" to a random UUID at the beginning of any job and making sure that the same ID is used throughout the job, c) writing the basic "stack" logic in the IndexingProgressBar component

I'm going to close this, but the thought process here was really valuable, so thank you for making the draft

@sestinj sestinj closed this Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants