Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.Net: Add PostgresVectorStore Memory connector. #9324

Open
wants to merge 73 commits into
base: main
Choose a base branch
from
Open

.Net: Add PostgresVectorStore Memory connector. #9324

wants to merge 73 commits into from

Commits on Oct 18, 2024

  1. Add PostgresVectorStore Memory connector. 

    Work in progress, some methods are not implemented yet.
    Rob Emanuele committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    8778d5f View commit details
    Browse the repository at this point in the history

  2. Add UpsertBatch, GetBatch, and DeleteBatch

    Rob Emanuele committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    ddad99a View commit details
    Browse the repository at this point in the history

  3. Remove unused CreateMapping

    Rob Emanuele committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    5447815 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    7533f8c View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    9a4f836 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Add vector search to PostgresVectorStore

    Rob Emanuele committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    68a000e View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. create index on collection creation

    Rob Emanuele committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    317f6af View commit details
    Browse the repository at this point in the history

  2. Support Guid, test distance functions

    Rob Emanuele committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    f4f5ba2 View commit details
    Browse the repository at this point in the history

  3. Format tests

    Rob Emanuele committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    2acf118 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    5db2c59 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Add service and kernel extensions

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    f4b4dc5 View commit details
    Browse the repository at this point in the history

  2. Default to Euclidean distance if no distance function is specified

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    5c58400 View commit details
    Browse the repository at this point in the history

  3. Add Postgres sample to concepts

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    8ea21cd View commit details
    Browse the repository at this point in the history

  4. Add docs for setting configuration in samples\Concepts

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    4dcd222 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    74b3764 View commit details
    Browse the repository at this point in the history

  6. Enforce dimension size in index creation

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    5c3e63f View commit details
    Browse the repository at this point in the history

  7. Create index for CreateTableIfNotExistsAsyc

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    6d9f1fd View commit details
    Browse the repository at this point in the history

  8. Log warning when index not created due to dimensions

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    b4266cc View commit details
    Browse the repository at this point in the history

  9. Refactor and tests; make SqlBuilder internal

    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    f86613a View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    716b794 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. Remove old migration note

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    8d8283b View commit details
    Browse the repository at this point in the history

  2. Fix docstring

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    89027fc View commit details
    Browse the repository at this point in the history

  3. Use parameter for tableName

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    8f45d9c View commit details
    Browse the repository at this point in the history

  4. Fix support for DateTime, DateTimeOffset

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    48811bd View commit details
    Browse the repository at this point in the history

  5. Fix warnings in test

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    1d6082d View commit details
    Browse the repository at this point in the history

  6. Remove kernel extensions, improve service extensions

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    eb0a683 View commit details
    Browse the repository at this point in the history

  7. Make PostgresSqlCommandInfo internal

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    a66d835 View commit details
    Browse the repository at this point in the history

  8. Default to a Hnsw index

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    53f1009 View commit details
    Browse the repository at this point in the history

  9. Default to cosine distance

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    08ea55f View commit details
    Browse the repository at this point in the history

  10. Consistently use includeVectors

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    319648b View commit details
    Browse the repository at this point in the history

  11. Simplify AsyncEnumerable return

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    5b52bdc View commit details
    Browse the repository at this point in the history

  12. Pass properties instead of full definition

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    cd845ee View commit details
    Browse the repository at this point in the history

  13. Throw instead of log for too high dimensionality

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    1d09a21 View commit details
    Browse the repository at this point in the history

  14. Remove DefaultVectorSize

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    74e9757 View commit details
    Browse the repository at this point in the history

  15. Remove unused using statements

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    ad5628c View commit details
    Browse the repository at this point in the history

  16. Remove VectorStore constructor that creates datsaource

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    dbf1aef View commit details
    Browse the repository at this point in the history

  17. Fix duplicate mapper call

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    a355bf7 View commit details
    Browse the repository at this point in the history

  18. Fix docstring typo

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    e499a80 View commit details
    Browse the repository at this point in the history

  19. Comment clarifying that multiple keys should be previously validated

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    c95e2b3 View commit details
    Browse the repository at this point in the history

  20. Refactor ExecuteNonQueryAsync calls to reduce code dupe

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    9d972b3 View commit details
    Browse the repository at this point in the history

  21. Forward Schema option. 

    Also _options shouldn't be nullable, and
use PostgresConstants.SupportedKeyTypes for key type check.
    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    6eb3793 View commit details
    Browse the repository at this point in the history

  22. Make PostgresVectorStoreDbClient internal

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    ed59fed View commit details
    Browse the repository at this point in the history

  23. Support more enumerable types

    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    1749adb View commit details
    Browse the repository at this point in the history

  24. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    ea7b01c View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. Refactor to support default + transactions 

    Defaults to Hnsw for index creation, but user can specify
IndexKind.None to avoid index creation.
Also wrap the table and index creation in a transaction
so the entire operation is atomic.
    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    86486d7 View commit details
    Browse the repository at this point in the history

  2. Fix issue with converting readonly array on upsert

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    b9b4a44 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    c53a8ee View commit details
    Browse the repository at this point in the history

  4. Fix SLN merge error

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    97ef60a View commit details
    Browse the repository at this point in the history

  5. Improve error handling

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    81e1805 View commit details
    Browse the repository at this point in the history

  6. Avoid CA1859 in test class 

    e.g. Change type of property 'CollectionInts' from 'System.Collections.Generic.ICollection<int>?' to 'System.Collections.Generic.HashSet<int>?' for improved performance
    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    a587260 View commit details
    Browse the repository at this point in the history

  7. Account for ngpsql missing func in .net std 2.0

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    e8fe800 View commit details
    Browse the repository at this point in the history

  8. Fix servicecollection tests

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    96c088e View commit details
    Browse the repository at this point in the history

  9. Logic for dimension max moved and tested elsewhere

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    0fc76f6 View commit details
    Browse the repository at this point in the history

  10. Remove unused using statement

    Rob Emanuele committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    266310b View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Remove logger from PostgresVectorStoreRecordCollection

    Rob Emanuele committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    08f110c View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2024

  1. Merge branch 'main' into feature/postgres-vector-store-dotnet

    @lossyrob
    lossyrob authored Oct 30, 2024
    Configuration menu
    Copy the full SHA
    26516c5 View commit details
    Browse the repository at this point in the history

  2. Use Flat instead of None index kind

    Rob Emanuele committed Oct 30, 2024
    Configuration menu
    Copy the full SHA
    5b44a80 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    b9b2487 View commit details
    Browse the repository at this point in the history

  2. Remove unnecessary overloads

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    24577a0 View commit details
    Browse the repository at this point in the history

  3. Change tests to be true to name

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    60d6512 View commit details
    Browse the repository at this point in the history

  4. Remove reduntant key type based test

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    5a66a13 View commit details
    Browse the repository at this point in the history

  5. Remove unnecessary overloads

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    581b6ab View commit details
    Browse the repository at this point in the history

  6. Better error handling for IAsyncEnumerable

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    494a0d4 View commit details
    Browse the repository at this point in the history

  7. Default to Flat (no index) instead of Hnsw

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    5f19889 View commit details
    Browse the repository at this point in the history

  8. Add enumerable to record mapper test

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    62ac8eb View commit details
    Browse the repository at this point in the history

  9. Remove unused fixture properties

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    364b592 View commit details
    Browse the repository at this point in the history

  10. Test StoragePropertyName in sql builder tests

    Rob Emanuele committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    bf58cab View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. Remove dynamic from integration test

    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    aa592de View commit details
    Browse the repository at this point in the history

  2. Add test to read from manually inserted record

    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    9a3b216 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    1ee09c1 View commit details
    Browse the repository at this point in the history

  4. Formatting, spelling

    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    b037075 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'upstream/main' into feature/postgres-ve… 

    …ctor-store-dotnet
    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    29d91ba View commit details
    Browse the repository at this point in the history

  6. Fix test.

    Rob Emanuele committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    c2937e0 View commit details
    Browse the repository at this point in the history