Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An error occurred while trying to add a local document. #476

Closed
pro777s opened this issue Nov 18, 2024 · 4 comments
Closed

An error occurred while trying to add a local document. #476

pro777s opened this issue Nov 18, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@pro777s
Copy link

pro777s commented Nov 18, 2024

Describe the bug

When trying to add a local document, an error occurs:

Error: System.AggregateException: One or more errors occurred. (SQLite Error 1: 'near "-": syntax error'.)
 ---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'near "-": syntax error'.
   at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
   at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements()+MoveNext()
   at Microsoft.Data.Sqlite.SqliteCommand.GetStatements()+MoveNext()
   at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
   at System.Data.Common.DbCommand.ExecuteNonQueryAsync(CancellationToken cancellationToken)
--- End of stack trace from previous location ---
   at LangChain.Databases.Sqlite.SqLiteVectorDatabase.CreateCollectionAsync(String collectionName, Int32 dimensions, CancellationToken cancellationToken) in /_/src/Sqlite/src/SqLiteVectorDatabase.cs:line 99
   at LangChain.Databases.Sqlite.SqLiteVectorDatabase.GetOrCreateCollectionAsync(String collectionName, Int32 dimensions, CancellationToken cancellationToken) in /_/src/Sqlite/src/SqLiteVectorDatabase.cs:line 78
   at LangChain.Extensions.VectorDatabaseExtensions.AddDocumentsFromAsync[TLoader](IVectorDatabase vectorDatabase, IEmbeddingModel embeddingModel, Int32 dimensions, DataSource dataSource, String collectionName, ITextSplitter textSplitter, DocumentLoaderSettings loaderSettings, EmbeddingSettings embeddingSettings, AddDocumentsToDatabaseBehavior behavior, CancellationToken cancellationToken) in /_/src/Core/src/Extensions/VectorDatabaseExtensions.cs:line 57
   at Program.g__MainLangchain|0_0() in C:\Ollama\LangchainNet8ConApp\Program.cs:line 42
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Program.<Main>$(String[] args) in С:\Ollama\LangchainNet8ConApp\Program.cs:line 13.

Here is the related code:

using Aspose.Pdf.Operators;
using LangChain.Databases.Sqlite;
using LangChain.DocumentLoaders;
using LangChain.Extensions;
using LangChain.Providers;
using LangChain.Providers.Ollama;

    var provider = new OllamaProvider();
    var embeddingModel = new OllamaEmbeddingModel(provider, id: "all-minilm");
    var llm = new OllamaChatModel(provider, id: "llama3.2");

    var vectorDatabase = new SqLiteVectorDatabase(dataSource: "sunnylight.db");

    //var vectorCollection = await vectorDatabase.AddDocumentsFromAsync<PdfPigPdfLoader>(
    var vectorCollection = await vectorDatabase.AddDocumentsFromAsync<HtmlLoader>(
        embeddingModel, // Used to convert text to embeddings
        dimensions: 1536, // Should be 1536 for TextEmbeddingV3SmallModel
        //dataSource: DataSource.FromPath(@"C:\4PDA\Sunny Light - 4PDA.pdf"),
        dataSource: DataSource.FromPath(@"C:\4PDA\Sunny_Light"),
        collectionName: "sunny-light", // Can be omitted, use if you want to have multiple collections
        textSplitter: null,
        behavior: AddDocumentsToDatabaseBehavior.JustReturnCollectionIfCollectionIsAlreadyExists);

The version of the LongChain package used is 0.15.3-dev.14.
The version .NET8.0 is used.

I attach a file with data in PDF format:

Sunny Light - 4PDA.pdf

Is it possible to quickly fix the error?

Steps to reproduce the bug

Compile and run the code given in the error description.

Expected behavior

No response

Screenshots

No response

NuGet package version

No response

Additional context

No response

@pro777s pro777s added the bug Something isn't working label Nov 18, 2024
@robalexclark
Copy link

I will try and take a look at this, if necessary, but, before I do can you take out the '-' from your collection name and try again?

@pro777s
Copy link
Author

pro777s commented Nov 18, 2024

I removed the "-" and also the space from the document name. However, the error remains.

@robalexclark
Copy link

did you remove it from the collection name? Not the document. Its because the error is occurring when the collection is created and specifically mentions "-".

So change

collectionName: "sunny-light"

to

collectionName: "sunnylight"

@pro777s
Copy link
Author

pro777s commented Nov 18, 2024

Thank you very much for the hint to solve this problem.
After removing the hyphen from the collection name, the document was uploaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants