Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PullModelAndEnsureSuccessAsync gives a '500 Internal Server Error' from the ollama api #349

Closed
lrolvink opened this issue Jun 18, 2024 · 0 comments
Labels
bug Something isn't working providers

Comments

@lrolvink
Copy link

lrolvink commented Jun 18, 2024

Describe the bug

Hi,

Started using LangChain but I ran into a problem with my own models in combination with ollama.
As long as you use known models, the code works, but as soon as you use a self-made model, a 500 Internal Server Error is returned with 'llm.GenerateAsync("Hi!")'.

Steps to reproduce the bug

  1. Having a running ollama server.
  2. Executing the following snippet:
var embeddingModel = new OllamaEmbeddingModel(provider, id: "all-minilm");
var llm = new OllamaChatModel(provider, id: "mycustommodel");

Console.WriteLine($"LLM answer: {await llm.GenerateAsync("Hi!").ConfigureAwait(false)}");

Expected behavior

It should be a option to allow pulling a model. By commenting out the LangChain.Providers.Ollama,GenerateAsync()

//await Provider.Api.Models.PullModelAndEnsureSuccessAsync(Id, cancellationToken: cancellationToken).ConfigureAwait(false);

solves my isssue.

Screenshots

No response

NuGet package version

No response

Additional context

http:

POST /api/pull HTTP/1.1
Host: 172.28.219.196:11434
Content-Type: application/json; charset=utf-8

{"model":"mycustommodel","insecure":false,"stream":false}
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Date: Tue, 18 Jun 2024 17:45:35 GMT
Content-Length: 52

{"error":"pull model manifest: file does not exist"}
@lrolvink lrolvink added the bug Something isn't working label Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working providers
Projects
None yet
Development

No branches or pull requests

2 participants