Updating v3 docs post JupyterCon

docs/source/contributors/index.md

@@ -9,7 +9,7 @@ If you would like to build applications that enhance Jupyter AI, please see the
 Maintainers of Jupyter AI have adopted principles that contributors should also follow. These principles, which build on top of [the Zen of Python](https://peps.python.org/pep-0020/), are intended to earn users' trust by keeping their data under their control. The following list is non-exhaustive; maintainers have discretion to interpret and revise these principles.
 
 1. Jupyter AI is **vendor-agnostic**. Jupyter AI does not discriminate between available models, and gives users a choice of model providers. A feature in Jupyter AI may be specific to one model or model provider if it cannot be used with other models or providers.
-2. Jupyter AI **only responds to an explicit prompt**; it does not watch files and it does not send prompts automatically. Any change that watches user files must be opt-in only.
+2. Jupyter AI **only responds to an explicit prompt**; it does not watch files and it does not send prompts automatically. Any change that watches user files must be opt-in only. One form of opt-in is users building agentic personas that undertake non-deterministic actions, including sending prompts to other personas or agents, managing the file system, etc.
 3. Jupyter AI is **transparent** with its chat prompts. The chat interface and magic commands use system messages and prompt templates that are open source, so that users know what gets sent to language models.
 4. Jupyter AI is **traceable**; users know when it has been used to generate content. When Jupyter AI generates a notebook, the notebook says that it was generated by Jupyter AI. When a user runs a Jupyter AI magic command in a notebook, output cells say, in their metadata, that they were generated by Jupyter AI.
 5. Jupyter AI uses a **human-centered design**. The chat interface should look and feel like chat applications that are generally available. The magic commands should look and work like other IPython magic commands. Settings screens should be used minimally, and wherever they are used, they should be readable and understandable, even for users not fluent in the user interface language.
@@ -18,13 +18,13 @@ Issues and pull requests that violate the above principles may be declined. If y
 
 ## Prerequisites
 
-You can develop Jupyter AI on any system that can run a supported Python version up to and including 3.12, including recent Windows, macOS, and Linux versions.
+You can develop Jupyter AI on any system that can run a supported Python version up to and including 3.13, including recent Windows, macOS, and Linux versions.
 
 You should have the newest supported version of JupyterLab installed.
 
 You will need [a supported version of node.js](https://github.com/nodejs/release#release-schedule) to use Jupyter AI.
 
-## Automatic development setup (recommended)
+<!-- ## Automatic development setup (recommended)
 
 To get a development setup automatically, you must first install a Python
 environment manager. The development setup script currently supports
@@ -131,11 +131,120 @@ jlpm dev:reinstall
 jlpm dr
 ```
 
-This is generally necessary when updating `pyproject.toml` files.
+This is generally necessary when updating `pyproject.toml` files. -->
+
+## Development Setup
+
+In v3, Jupyter AI is no longer a monorepo. The various components of Jupyter AI are modularized as submodules in the [`jupyter-ai-contrib`](https://github.com/jupyter-ai-contrib) org under various repositories. These are Jupyter extensions that are desined to enable humans and AI to collaborate together. The `jupyter-ai` repository is installed with all the submodules.
+
+The following repositories are stable, i.e., this means the repos are, or are quickly becoming, stable and ready for production use.
+
+- [jupyter-ai-chat-commands](https://github.com/jupyter-ai-contrib/jupyter-ai-chat-commands) - Default set of chat commands in Jupyter AI
+- [jupyter-ai-jupyternaut](https://github.com/jupyter-ai-contrib/jupyter-ai-jupyternaut) - Package which provides the Jupyternaut, the default AI persona, in Jupyter AI
+- [jupyter-ai-litellm](https://github.com/jupyter-ai-contrib/jupyter-ai-litellm) - LiteLLM based model abstraction for Jupyter AI
+- [jupyter-ai-persona-manager](https://github.com/jupyter-ai-contrib/jupyter-ai-persona-manager) - The core manager & registry for AI personas in Jupyter AI
+- [jupyter-ai-router](https://github.com/jupyter-ai-contrib/jupyter-ai-router) - The core routing layer used in Jupyter AI to process chat messages
+- [jupyter-ai-tools](https://github.com/jupyter-ai-contrib/jupyter-ai-tools) - Jupyter Server extension that exposes a collection of powerful, agent-friendly tools for interacting with notebooks and Git repositories
+- [jupyter-server-documents](https://github.com/jupyter-ai-contrib/jupyter-server-documents) - Server side document handling with improved output handling and kernel management/status.
+- [jupyter-server-mcp](https://github.com/jupyter-ai-contrib/jupyter-server-mcp) - An MCP interface/extension for Jupyter Server
+- [jupyterlab-diff](https://github.com/jupyter-ai-contrib/jupyterlab-diff) - JupyterLab commands to show cell and file diffs
+- [jupyter-ai-magic-commands](https://github.com/jupyter-ai-contrib/jupyter-ai-chat-commands) - The code for handling the `%ai` and `%%ai` magic commands in Jupyter notebooks.
+
+These repositories are experimental and under active development:
+
+- [jupyter-ai-claude-code](https://github.com/jupyter-ai-contrib/jupyter-ai-claude-code) - A Jupyter AI persona for Claude Code
+- [jupyter-ai-personas](https://github.com/jupyter-ai-contrib/jupyter-ai-personas) - AI Personas for Jupyter AI
+- [jupyter-ai-demos](https://github.com/jupyter-ai-contrib/jupyter-ai-demos) - A set of demos for new features of Jupyter AI
+- [jupyter-floating-chat](https://github.com/jupyter-ai-contrib/jupyter-floating-chat) - A jupyterlab extension to add a floating chat input
+- [jupyter-server-ai-tools](https://github.com/jupyter-ai-contrib/jupyter-server-ai-tools) - A Jupyter Server extension for discovering tools across extensions.
+- [jupyterlab-cell-input-footer](https://github.com/jupyter-ai-contrib/jupyterlab-cell-input-footer) - JupyterLab Plugin that provides a cell input footer
+- [jupyterlab-commands-toolkit](https://github.com/jupyter-ai-contrib/jupyterlab-commands-toolkit) - JupyterLab commands as an AI Toolkit
+- [jupyterlab-document-collaborators](https://github.com/jupyter-ai-contrib/jupyterlab-document-collaborators) - A JupyterLab extension for showing collaborators at the top of a document
+
+- [jupyterlab-magic-wand](https://github.com/jupyter-ai-contrib/jupyterlab-magic-wand) - An in-cell AI assistant for JupyterLab notebooks
+- [jupyterlab-notebook-awareness](https://github.com/jupyter-ai-contrib/jupyterlab-notebook-awareness) - Track current notebook and active cell in JupyterLab's awareness
+- [jupyterlab-ai-commands](https://github.com/jupyter-ai-contrib/jupyterlab-ai-commands) - A set of JupyterLab commands for use with AI agents
+
+### Dev install using `just` and `uv`
+
+[jupyter-ai-devrepo](https://github.com/dlqqq/jupyter-ai-devrepo) is a "developer repo" intended for Jupyter AI contributors. This facilitates installing Jupyter AI along with all its submodules with a small number of steps. By cloning the repo and following the steps below, you can have an editable developer installation of all Jupyter AI subpackages.
+
+#### 0. Clone the repo
+
+```
+git clone --recurse-submodules <url>
+cd jupyter-ai-devrepo/
+```
+
+#### 1. Install root dependencies
+
+This monorepo requires `git`, `uv`, and `just`. Use `homebrew` to install these if you do not have them installed.
+
+No dedicated Python environment is required because `uv` automatically manages a local venv.
+
+If you use `conda`/`mamba`/`micromamba`, you can run the following commands to
+install these dependencies into the `base` environment:
+
+```sh
+{conda,mamba,micromamba} activate base
+{conda,mamba,micromamba} install uv just
+# make sure to activate the `base` environment before working in this repo
+```
+
+Otherwise, you can use your OS's package manager. For example, on macOS:
+
+```sh
+brew install uv just
+```
+
+#### 2. Pull in latest changes
+
+This command switches to the `main` branch on every submodule and pulls from it.
+
+```
+just sync
+```
+
+If you do not have your RSA key set up to access GitHub from the CLI, you will be prompted to authenticate yourself on GitHub for each of the subpackages as they get cloned into the `jupyter-ai-devrepo` folder.
+
+#### 3. Install all packages
+
+This command automatically installs each of the packages in editable mode.
+
+```
+just install
+```
+
+#### 4. Start JupyterLab
+
+Start JupyterLab by running:
+
+```
+just start
+```
+
+This command will always run `uv run jupyter lab` from the root of this devrepo,
+even if your current directory is inside of a submodule.
+
+#### Useful commands
+
+- `just start`: start JupyterLab
+
+  - `Ctrl + Z` + `kill -9 %1` stops JupyterLab in case `Ctrl + C` does not work
+
+- `just sync`: switch to `main` in all submodules and pull in all upstream changes
+
+- `just build-all`: build all frontend assets in every submodule
+
+- `just install`: perform an editable, developer installation of all packages
+
+- `just uninstall`: uninstall everything (useful for testing the `just` commands)
+
+- `just uninstall && just install`: re-install everything (useful for fixing a broken venv)
 
 ## Building documentation
 
-The `./scripts/dev-setup.sh` should automatically install the documentation
+The `just install` script should automatically install the documentation
 dependencies. You will need to install [pandoc](https://pandoc.org/) as well. You can install [pandoc from the conda-forge channel](https://anaconda.org/conda-forge/pandoc):
 
 ```
@@ -144,7 +253,6 @@ conda install -c conda-forge pandoc
 
 Otherwise have a look at pandoc's [installation instructions](https://pandoc.org/installing.html).
 
-
 To build the documentation locally, run
 
 ```
@@ -160,19 +268,18 @@ choice to easily view your local documentation build.
 After making any changes, make sure to rebuild the documentation locally via
 `make html`, and then refresh your browser to verify the changes visually.
 
-
-## Development uninstall
+<!-- ## Development uninstall
 
 To uninstall your Jupyter AI development environment, deactivate and remove the Conda environment:
 
 ```
 conda deactivate
 conda env remove -n jupyter-ai
-```
+``` -->
 
 ## Testing
 
-### Integration / E2E tests
+### Integration / E2E tests (**TO BE UPDATED**)
 
 This extension uses Playwright for the integration / E2E tests (user-level tests).
 More precisely, the JupyterLab helper
@@ -182,12 +289,12 @@ test the extension in JupyterLab.
 Install test dependencies (needed only once):
 
 ```sh
-cd ./packages/jupyter-ai/ui-tests/
+cd ./packages/<subpackage>/ui-tests/
 jlpm install
 jlpm playwright install
 ```
 
-Tests involve snapshot comparisons against a reference snapshots generated by the Github CI. If you are using an OS other than Linux, you will need to generate local snapshots before running the tests locally for the first time. To do this, execute the command:
+Tests involve snapshot comparisons against reference snapshots generated by the Github CI. If you are using an OS other than Linux, you will need to generate local snapshots before running the tests locally for the first time. To do this, execute the command:
 
 ```sh
 cd ./packages/jupyter-ai/ui-tests/

docs/source/index.md

@@ -1,30 +1,90 @@
-# Jupyter AI
+# Jupyter AI v3
 
 Welcome to Jupyter AI, which brings generative AI to Jupyter. Jupyter AI provides a user-friendly
 and powerful way to explore generative AI models in notebooks and improve your productivity
 in JupyterLab and the Jupyter Notebook. More specifically, Jupyter AI offers:
 
-* An `%%ai` magic that turns the Jupyter notebook into a reproducible generative AI playground.
+- A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant, and also enables interaction with the active notebook.
+- An `%%ai` magic that turns the Jupyter notebook into a reproducible generative AI playground.
   This works anywhere the IPython kernel runs (JupyterLab, Jupyter Notebook, Google Colab, VSCode, etc.).
-* A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant.
-* Support for a wide range of generative model providers and models
-  (AI21, Anthropic, Cohere, Gemini, Hugging Face, MistralAI, OpenAI, SageMaker, NVIDIA, etc.).
+- Support for a wide range of generative model providers and models
+  (AI21, Amazon, Anthropic, Cohere, Gemini, Hugging Face, MistralAI, OpenAI, NVIDIA, etc.).
+- Multiple editable chat threads are available, each thread saved to a separate Jupyter server document with extension `.chat`.
+- Real time collaboration (RTC) is enabled in both chat and Jupyter notebook, if cloud deployments support it.
+- Support for hundreds of LLMs from many additional providers.
+- Chat personas with agentic capabilities, with a default `Jupyternaut` persona.
+
+Below is the look and feel of Jupyter AI v3. You can see the chat panel on the left and the notebooks on the right. The chat panel shows the Jupyternaut persona responding to prompts in the chat, as well as interacting with the code as context from the notebook on the right. The right panel shows the use of the `%%ai` magics commands.
 
 <img src="_static/jupyter-ai-screenshot.png"
     alt='A screenshot of Jupyter AI showing the chat interface and the magic commands'
+    width="95%" 
     class="screenshot" />
 
 ## JupyterLab support
 
-**Each major version of Jupyter AI supports *only one* major version of JupyterLab.** Jupyter AI 1.x supports
+**Each major version of Jupyter AI supports _only one_ major version of JupyterLab.** Jupyter AI 1.x supports
 JupyterLab 3.x, and Jupyter AI 2.x supports JupyterLab 4.x. The feature sets of versions 1.0.0 and 2.0.0
-are the same. We will maintain support for JupyterLab 3 for as long as it remains maintained.
+are the same. We will maintain support for JupyterLab 3 for as long as it remains maintained. Jupyter AI v3 supports JupyterLab 4.x.
 
 The `main` branch of Jupyter AI targets the newest supported major version of JupyterLab. All new features and most bug fixes will be
 committed to this branch. Features and bug fixes will be backported
 to work on JupyterLab 3 only if developers determine that they will add sufficient value.
 **We recommend that JupyterLab users who want the most advanced Jupyter AI functionality upgrade to JupyterLab 4.**
 
+## Quickstart
+
+It is best to install `jupyter-ai` in an environment. Use [conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html) to create an environment that uses Python 3.13 and the latest version of JupyterLab:
+
+    $ conda create -n jupyter-ai python=3.13 jupyterlab
+    $ conda activate jupyter-ai
+
+To install both the `%%ai` magic and the JupyterLab extension, you can run:
+
+    $ pip install jupyter-ai==<version number>
+
+Choose the version number, the latest version is `3.0.0b9`.
+
+For an installation with all related packages, use:
+
+    $ pip install "jupyter-ai[all]"==<version number>
+
+To start Jupyter AI in Jupyter Lab, run
+
+```
+jupyter lab
+```
+
+You should see an interface similar to the one above. Use the `+ Chat` button in the chat panel to open a chat thread and enter prompts. In the chat box enter `@` to see a list of personas, and you can select one before entering your query.
+
+To connect a LLM for use in your chat threads you can select the `Settings` dropdown menu, select `Jupyternaut settings` in it to see the settings panel, in which you can select a chat model, specify model parameters if needed, and also add API keys for using LLMs that require it.
+
+To use `uv` instead of `pip`:
+
+Create a virtual environment with `uv` in any folder:
+
+```
+uv venv --python 3.13
+```
+
+Activate the environment:
+
+```
+source .venv/bin/activate
+```
+
+Install with
+
+```
+uv pip install "jupyter-ai[all]"==<version number>
+```
+
+Run with
+
+```
+jupyter lab
+```
+
 ## Contents
 
 ```{toctree}