Smol Helper

Smol helper is a small client for a small LLM model to assist on very basic tasks, the main focus in on performance and availability.

It has 3 available modes:

• Conversational: Default mode no extra parameters required, shows a prompt for a conversation
• -oneshot: reads the whole stdin input before processing on the model, useful for completion tasks or scripts
• -headless: a conversational mode without a prompt, useful for scripts which require continuous inputs or bots.

For the conversational modes a blank line tells the model that it should act on the input up to that point. This is not suitable to send code which has a lot of blank lines in between the statements, for this -oneshot is recommended.

Installation

Binaries

On the releases page you can find the last compiled releases.

Docker

Images are tagged based on the git tag but also the latest tag is available

docker run -it rafadotel/smol-helper

will download and execute the latest model

# docker run -it rafadotel/smol-helper
✔ docker.io/rafadotel/smol-helper:latest
Trying to pull docker.io/rafadotel/smol-helper:latest...
Getting image source signatures
Copying blob 711049d4ff0d done
Copying blob f56be85fc22e skipped: already exists
Copying blob ac91355af1d2 done
Copying config 8143429fea done
Writing manifest to image destination
Storing signatures
User:

Nixos

This repository uses flakes so can be installed easily within a Nixos + Flakes setup.

Using Go tooling

This project uses standard Golang tooling, so it can be installed with:

go install -v github.com/rafa-dot-el/smol-helper@latest

Running in interactive mode

This is the default mode, it will start a conversation which will have a context.

smol-helper

The conversation is split between User and Helper:

User: What is Debian?

Helper: Debian is a free and open-source operating system. It is developed and
maintained by a community of volunteers who work together to provide a complete
and stable operating system. Debian is known for its simplicity and ease of use,
and is used by many individuals, organizations, and governments around the
world.

User: What is Nixos?

Helper: NixOS is a Linux distribution based on the Nix package manager. It is
designed to be simple, flexible, and secure, and is designed to be used in
production environments. NixOS is known for its focus on security and privacy,
and provides a secure and isolated environment for its users.

User: What is the Dark Brotherhood?

Helper: Dark Brotherhood is a fictional organization in the video game The Elder
Scrolls V: Skyrim. It is a group of assassins who serve the Daedric Prince Molag
Bal. The Dark Brotherhood is known for their secrecy, their use of dark magic,
and their willingness to kill anyone who threatens the stability of the game's
world.

Running in headless mode

Using the flag -headless no other output than the server output will be printed out. This is to be used on shell scripts and as a backend for applications.

smol-helper -headless

Now your application can read from the process stdout and write to the process stdin without worrying about formats or parsing data.

Running in one shot mode

One shot mode is usefull for integration which don’t require keeping the context, and at the same time have bigger inputs, as in pieces of code, error logs and general tasks which require line breaks and just one bigger context.

If using an interactive shell, type the input without worrying about line breaks and finish it with Ctrl + D. If not running interactively, is possible to pipe or input a file to the model as the following:

cat example-docstring | smol-helper -oneshot

Or

smol-helper -oneshot < example-docstring

It will give the following output:

```py
def sum(a: int, b: int) -> int:
    """
    Sum two numbers.

    Parameters
    ----------
    a : int
        First number.
    b : int
        Second number.

    Returns
    -------
    int
        The sum of the two numbers.
    """
    return a + b
```

The contents of the task file are the following:

Given the code below, write a python docstring with the Google standard listing all parameters.


def sum(a: int, b:int) -> int:
  return a+b

Extensibility

Since smol-helper uses the unix philosophy, it is easily composeable. Below some examples and use cases to show how embed it into your scripts.

Debug kernel problems

Error messages can be piped into smol-helper and a simple diagnostic can be taken. Below an example of the composability of smol-helper to debug a hardware problem.

function what-is-wrong() {
    local LAST_ERRORS=$(dmesg | grep -i error | cut -d ']' -f 2 | sort | uniq | tail -n 3)
    echo "I found this error on dmesg of my linux, can you help me fix it? ${LAST_ERRORS}"  | smol-helper -oneshot
}

After defining the above function, at anytime you can just run what-is-wrong, and it will take the last 3 lines containing error (case insensitive) from your kernel logs and ask the model what could be the cause of it.

As for example my last 3 lines on dmesg are

[    6.081375] usb 1-8: device descriptor read/64, error -71
[    7.095335] usb 1-8: device not accepting address 6, error -71
[    7.735333] usb 1-8: device not accepting address 7, error -71

Running what-is-wrong results in

These errors usually indicate that the USB device is not properly configured or
is not supported by the host computer. To resolve this issue, you should check
the following steps:

1. Check the device's specifications and make sure that it is compatible with
   your host computer.
2. Ensure that the USB cable is properly connected and that the device is not
   being powered by another source.
3. Try plugging the device into a different USB port on the host computer.

Using with python

It can be embedded within simple python scripts as for example:

import subprocess
p = subprocess.run(['smol-helper','-oneshot'], input='What is the Unix Philosophy?', capture_output=True, text=True)
print(p)

Build

It uses standard Golang tooling, to build just run

go build -o smol-helper cmd/smol-helper/main.go

Release

Releases are executed using gorelease:

goreleaser release --snapshot --clean

Developing

This project uses nix and direnv, just

direnv allow

and

nix develop

Are enough to give you a development environment.

Disclaimer

This or previous program is for Educational purpose ONLY. Do not use it without permission. The usual disclaimer applies, especially the fact that me (Rafa-dot-el) is not liable for any damages caused by direct or indirect use of the information or functionality provided by these programs. The author or any Internet provider bears NO responsibility for content or misuse of these programs or any derivatives thereof. By using these programs you accept the fact that any damage (dataloss, system crash, system compromise, etc.) caused by the use of these programs is not Rafa-dot-el’s responsibility.