How are you handling AI based commit message summaries?

[gobijan]

I catch myself often jumping to vscode for generating a commit message using Copilot (Claude 3.5 at the time). How do you guys handle commit message generation? Is this something that is on the Roadmap for GitSavvy?

[kaste]

Typically I just write the message but I like proofing it. For that I just use the available tooling.

For Sublime, https://packagecontrol.io/packages/OpenAI%20completion is probably the standard AI helper. I would not like to re-implement anything from it, but calling a function/command and using its outcome would be nice. Let's invite @yaroslavyaroslav

Is there a function in OpenAI -- or one that you can think of providing -- that given a prompt and a StringIO/buffer fills that buffer with an answer from an AI so that we can either spell check a given commit message or write a commit message from a given diff (etc).

[yaroslavyaroslav]

Currently I'm using plain bash script that just reaches gh llm instance with git diff command within curl and pastes the llm output to my clipboard. That's because I prefer to stick with SM when it comes to committing stuff and this is the maximum it is capable to provide rn.

However in the dev branch of my plugin there's a pending feature just for that: it takes output panel content and the user command as an input and provides the output to either output panel or phantom.

There's a caveat in that though. I was too tired of Python and so I'm reimplementing the core of the plugin in rust to just wrap it then and leave just the ST front-end layer in Python. So it would take time till I'll release that feature. Month or two I suppose.

[kaste]

Haha, I thought the Python plugin is a short wrapper around web-calls, t.i. basically view code. Can you show us the bash script, sounds like it is just a http-call with not that much parsing?

However, I actually thought you can provide a function e.g. def ask_ai(prompt: str, out_stream: StringIO, options: dict). Probably out_stream would only have two methods append/receive and done (plus cancel?). (StringIO has among others write and close). Anyway, that would mean you don't render the outcome but stream it to an in-memory IO/buffer/object, and we intercept/observe that and write it to the correct place in the commit message. (Or elsewhere.)

[yaroslavyaroslav]

Haha, I thought the Python plugin is a short wrapper around web-calls, t.i. basically view code.

Not short enough as I wish. It's actually not that bad in the code structure or its complication as with the lack of infrastructure. I wanted to leverage types to decode/encode stuff rather than plain dicts used so far and failed to do so in a declarative way within the dependencies available. This was the last drop for me to switch to rust.

However I'm providing the bash script down below, so here I'd like to focus on your feature request. As long as I'm implementing thing in rust I'll willing to publish it as the plain python package with 0 dependencies. So I think that this would be more convenient to use on your side. This is how API of the package looks like at a glance.

    def my_handler(data: str) -> None:
         print(f'Received data: {data}')

    worker = Worker(window_id=101, path='/tmp/', proxy=PROXY)

    contents = SublimeInputContent(
        InputKind.ViewSelection, 'This is the test request, provide me 30 words response'
    )

    settings = AssistantSettings(
        'TEST',
        OutputMode.Phantom,
        'gpt-4o-mini',
        token=os.getenv('OPENAI_API_TOKEN'),
        assistant_role="You're echo bot. You'r just responsing with what you've been asked for",
        tools=None,
        parallel_tool_calls=None,
        stream=True,
        advertisement=False,
    )

    worker.run(1, PythonPromptMode.View, [contents], settings, my_handler)

commit_diff=$(git diff --cached -w | sed 's/"/\\"/g; s/$/\\n/g' | tr -d '\n' 2>/dev/null)

if [[ $? -ne 0 || -z "$commit_diff" ]]; then
    echo "Error: Unable to get the commit diff." >&2
    exit 1
fi


response=$(curl -s -X POST -H "Content-Type: application/json" -H "Authorization: Bearer __TOKEN__" -d "{\"model\": \"meta-llama/Llama-3.3-70B-Instruct-Turbo\", \"messages\": [{\"role\": \"system\", \"content\": \"Generate a commit message for the following, just provide the content of the commit, no intro, no thinging, no nothing\" }, {\"role\": \"user\", \"content\": \"$commit_diff\" } ], \"max_tokens\": 300 }" https://llm.hosting.com/v1/chat/completions)

if [[ $? -ne 0 || -z "$response" ]]; then
    echo "Error: API request failed." >&2
    exit 1
fi

commit_msg=$(jq -r '.choices[0].message.content' <<< "$response")

if [[ $? -ne 0 || -z "$commit_msg" ]]; then
    echo "Error: Unable to parse the commit message from the response." >&2
    exit 1
fi

echo "$commit_msg" | pbcopy

[yaroslavyaroslav]

Here's the updated script, it handles \ No newline at end of file git diff thing properly.

commit_diff=$(git diff --cached -w --no-ext-diff | perl -0777 -pe '
  s/\\ No newline at end of file\s*//g;   # remove the unwanted line (and trailing whitespace)
  s/"/\\"/g;                            # escape all double quotes
  s/\n/\\n/g;                           # replace newline characters with literal \n
' 2>/dev/null)

if [[ $? -ne 0 || -z "$commit_diff" ]]; then
    echo "Error: Unable to get the commit diff." >&2
    exit 1
fi

response=$(curl -s -X POST -H "Content-Type: application/json" -H "Authorization: Bearer xxx" -d "{\"model\": \"meta-llama/Llama-3.3-70B-Instruct-Turbo\", \"messages\": [{\"role\": \"system\", \"content\": \"Generate a commit message for the following, just provide the content of the commit, no intro, no thinking, no nothing, use appropriate prefix off: feat, test, refactor, fix, perf, chore\" }, {\"role\": \"user\", \"content\": \"$commit_diff\" } ], \"max_tokens\": 300 }" https://api.together.xyz/v1/chat/completions)

if [[ $? -ne 0 || -z "$response" ]]; then
    echo "Error: API request failed." >&2
    exit 1
fi

commit_msg=$(jq -r '.choices[0].message.content' <<< "$response")

# Check if commit_msg is null or empty
if [[ "$commit_msg" == "null" || -z "$commit_msg" ]]; then
    echo "Error: Unable to parse the commit message from the response." >&2
    exit 1
fi

echo "$commit_msg" | pbcopy

[gobijan]

https://github.com/TerminalFi/LSP-copilot is probably the bridge we want there as it won't require an additional API key from OpenAI filled with credits.

Under the hood we need to copy the output of the git diff command and ask AI to "write a git commit message for these changes" and take the output as commit message.
Copilot Chat will probably deliver here. Just not sure how the python API for accessing looks like.

[kaste]

Is there any other "free" service even out there? (E.g. hacking on Sublime Text plugins I cannot use any paid services.) copilot is free -- since a month or so -- in a limited way.

I have this, just using web calls

>>> Write a hello world function in Python!

Certainly! Here is a simple "Hello, World!" function in Python:

```python
def hello_world():
    print("Hello, World!")

# Call the function
hello_world()
```

When you run this code, it will output:

```
Hello, World!
```

>>> Change it to output Hello, Sir!

Sure! Here is the modified function to output "Hello, Sir!":

```python
def hello_sir():
    print("Hello, Sir!")

# Call the function
hello_sir()
```

When you run this code, it will output:

```
Hello, Sir!
```

>>> Thanks

You're welcome! If you have any more questions or need further assistance, feel free to ask. Happy coding!

>>>

so far. (>>> denoting my prompt input.)

[yaroslavyaroslav]

I doubt about the completely free one, but I have a few calls to llama 3.3 and with the limited context window on my free gh account with a my personal token.

I have faced the limitation of this particular hosting already, diff of an commit could be big, so if the hosting is not advanced enough you just wouldn't have enough context window to put whole diff within and thus you'd got error or not that good response.

[kaste]

8k context window says the chat ("my context window size is 8192 tokens" -- can we trust it?). But you don't want to spend 100k tokens on a commit message. It is just so expensive.

Do you pay that by yourself (also @gobijan) or does your company also allow usage in your free time?

[gobijan]

@kaste yes I have several subscriptions but I guess that's not the point here. The commit message summarization could be done using local LLMs or Services. GitHub Copilot unfortunately has no public API and the workaround most integrations use https://github.com/github/copilot.vim and extract the js based language server: https://github.com/github/copilot.vim/blob/release/dist/language-server.js

[kaste]

Doch, doch. That was the point. The above "chat" comes from a short python script talking to Github Copilot using standard web calls (without the LSP part). It may have limitations, esp. if you use the free copilot. A generic solution, one that can speak to various services, likely is way more complicated than speaking to a single endpoint. (They would need multiple subscriptions on my side for testing alone.)

[yaroslavyaroslav]

FYI: Package Control team have added my ST compatible rust/python package to the dependencies list. So you're free to leverage on it to reach llm from within your plugin. The api to perform a call is quite straightforward: https://github.com/yaroslavyaroslav/ai_helper/blob/c21ea6369bb0f5f205a457bbe6da75ba49f6e13d/tests/test_worker_python.py#L79-L114

PS: During the a few forthcoming weeks breaking changes could appear within the API, but then I assume it to be quite stable in a long term.