Alex's Breadboard Chrome Extension Friction Log

[alexandraM98]

Building a Chrome extension using Vite and React:

• The purpose of the extension will be to summarise the text from the currently opened chrome tab (e.g. an article, a news page, a documentation page, etc) using Breadboard and Claude AI.
• I started by initialising a new React app using the npm create vite@latest breadboard-chrome-extension -- --template react-ts command and created a basic UI and initial board following a previous example I've written at: https://github.com/ExaDev/create-breadboard/tree/main/templates/vite-react-ts/src
• Then, I created the summarisation board. To do this, I used the new syntax following a similar approach to that on DevPulse (see https://github.com/ExaDev/breadboard/blob/breadboard-chrome-extension-for-summarising-page/packages/breadboard-chrome-extension/src/breadboard/index.ts)
• As there was an issue with kits imported from @exadev/breadboard-kits, StringKit and ClaudeKit, have been re-written inside code nodes (see https://github.com/ExaDev/breadboard/blob/breadboard-chrome-extension-for-summarising-page/packages/breadboard-chrome-extension/src/breadboard/kits-as-code-node.ts)
• Some helpful articles I looked at while learning to build a Chrome extension for the first time were: https://dev.to/cookiemonsterdev/chrome-extension-using-vite-react-3bhj ; https://arglee.medium.com/chrome-extensions-using-vite-typescript-react-stepwise-process-6d013f5332b9 ; https://betterprogramming.pub/creating-chrome-extensions-with-typescript-914873467b65
• To get the extension up and running initially, the text to summarise was passed to the board via user input (a <textarea> tag). Then, the next step was to get the text content of the active tab instead. For this, the following solutions were considered:
  • Readability.js - had some trouble setting this up inside chrome.scripting.executeScript
  • chrome.pageCapture - I tried using the saveAsHTML function, but wasn't sure what parameters were actually needed.

The current solution uses the chrome.scripting API:

		const callback = await chrome.scripting.executeScript({

			target: { tabId: tab.id ? tab.id : 0 },
			
			func: () => document.body.innerText,
			
			});

Using the above function I execute a script that returns the innerText of the document's body for the active tab (target is set to the active tab's ID for this reason).
The following threads were helpful in reaching the current solution: https://stackoverflow.com/questions/70736493/chrome-extension-obtaining-active-tabs-dom-information-through-background-scr; https://stackoverflow.com/questions/8499376/chrome-extension-get-entire-text-content-of-the-current-tab

[alexandraM98]

Here is a detailed list of updates and work in progress related to the previously described chrome extension that uses Breadboard:

• To improve the Claude output, as it was generating text much longer than the input, the prompt was changed from "Summarise the discussion regarding this post:" to "Shorten the discussion regarding this post in less than the original number of words". Executing the board with the updated prompt on the article at: https://developer.chrome.com/blog/io24-web-ai-wrapup?hl=en, results in the following output:

• In the initial version, the board was not serialized and it was simply invoked on the frontend as const boardRun = await claudeBoard({ message: "TEXT-FROM-ACTIVE-TAB", claudeKey: "myClaudeApiKey" });
  As a first step to make it easier to run any board from the chrome extension in the future, I decided to serialize the board and write the result to a JSON file. Then, the following class was created that allows me to run a board, given a JSON file (and runtime kits if required): https://github.com/ExaDev/breadboard/blob/breadboard-chrome-extension-for-summarising-page/packages/breadboard-chrome-extension/src/breadboard/classes/ExtensionBoardRunner.ts
  Using the above class, the extension popup component can run a board by creating a new instance of the new runner class and passing it a serialized board and kits like so:

const extensionRunner = new ExtensionBoardRunner(serializedClaudeBoard, [
      ClaudeKit,
    ]);
    const boardRun = await extensionRunner.runBoard({
      message: result,
      claudeKey: key,
    });

• Besides running the board on the entire page, there is also the possibility of selecting only a certain piece of text on the page and running it from the context menu. The recording below shows a short demo of this process:

Screen.Recording.2024-05-24.at.16.04.11.mov

To add some visual feedback for users, badges were added to indicate the status of the board (ON if running and DONE if completed).

• The extension had a default options page that I was able to customise to some extent, however, for more control over the styling and behaviour, I removed the default and instead used a custom popup window (similar to how raindrop.io presents their settings) when right-clicking the extension icon. Below is a short recording of how secrets can be set as key-value pairs from the settings page (the styling was meant to be similar to the one used in the Settings windows from the breadboard-web app):

Screen.Recording.2024-05-24.at.16.06.24.mov

[alexandraM98]

Updates to extension described above:

Breadboard summariser

• Tidying up the summariser once more and noted a few non-crucial updates for the future, such as:
  - Using SCSS and moving all the component styles into modules and importing the styles from there. As part of this, there could also be a central theming.module.scss holding the colours used for the UI and allow for importing colour codes from here instead of hardcoding them in the css or inline styling.
  - Adding more helper functions and hooks where relevant (e.g., for repeated functionality). This is already something I've done in the code so far, but there is still room for improvement in this area.

Two other boards had been integrated with the chrome extension, one for text labelling and one for sentiment analysis

1. Sentiment analysis board (see https://github.com/ExaDev/breadboard/tree/breadboard-sentiment-analysis-chrome-extension)

• This board uses the lxyuan/distilbert-base-multilingual-cased-sentiments-student model from Hugging Face (see: https://huggingface.co/lxyuan/distilbert-base-multilingual-cased-sentiments-student)
• This board works on a piece of selected text and displays the output in a table inside the chrome extension popup. For demo, I will be updating this post later on with a video recording.

3. Text labelling board

• For this, I'm using the board developed at: https://github.com/ExaDev/breadboard-examples/tree/Hugging-Face-Clean-History/src/examples/labelling by @jig925

[dglazkov]

@paullewis, since you've been poking at Chrome extensions