Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print an error when index creation fails #4064

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

russell
Copy link

@russell russell commented Sep 16, 2024

Thank you for LibreChat,

While investigating some performance issues, we found that some indexes were missing from MongoDB. We still don't know exactly what is the reason that we are missing indexes and are investigating. But one thing I came across was that Mongoose will only report some index failures if you explicitly do it https://mongoosejs.com/docs/guide.html#indexes

For example, if you apply this diff which adds an invalid index, to the default branch, the system will work as expected and no information about the invalid index will be visible in the logs.

diff --git a/api/models/schema/messageSchema.js b/api/models/schema/messageSchema.js
index 24acf29a..c618a28d 100644
--- a/api/models/schema/messageSchema.js
+++ b/api/models/schema/messageSchema.js
@@ -132,6 +132,8 @@ if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
 messageSchema.index({ createdAt: 1 });
 messageSchema.index({ messageId: 1, user: 1 }, { unique: true });

+messageSchema.index({ foobarbazbar: 1 }, { unique: true });
+
 /** @type {mongoose.Model<TMessage>} */
 const Message = mongoose.models.Message || mongoose.model('Message', messageSchema);

With the logging added in this PR, we will see errors like this

2024-09-16 09:45:01 error: Failed to create Message index MongoServerError: Index build failed: d5bfca52-6ace-48d7-9d3f-906b32f9ff7a: Collection LibreChat.messages ( 29a855ee-abf3-4d7a-abb2-7310628dc011 ) :: caused by :: E11000 duplicate key error collection: LibreChat.messages index: foobarbazbar_1 dup key: { foobarbazbar: null }

I think in the long run having the system error and exit when an index is not able to be created is the best behaviour, but initially adding logging show what is missing is useful.

I'm happy to extend this PR to include functionality to exit, but I wanted to bring this case to your attention before making assumptions about the right way to handle the inability to create an index.

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

It is described above

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • My changes do not introduce new warnings
  • Local unit tests pass with my changes (i'm having trouble getting them to run at all)

By adding an invalid index, like this

```
diff --git a/api/models/schema/messageSchema.js b/api/models/schema/messageSchema.js
index 24acf29a..c618a28d 100644
--- a/api/models/schema/messageSchema.js
+++ b/api/models/schema/messageSchema.js
@@ -132,6 +132,8 @@ if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
 messageSchema.index({ createdAt: 1 });
 messageSchema.index({ messageId: 1, user: 1 }, { unique: true });

+messageSchema.index({ foobarbazbar: 1 }, { unique: true });
+
 /** @type {mongoose.Model<TMessage>} */
 const Message = mongoose.models.Message || mongoose.model('Message', messageSchema);

```

we will see errors like this

```
2024-09-16 09:45:01 error: Failed to create Message index MongoServerError: Index build failed: d5bfca52-6ace-48d7-9d3f-906b32f9ff7a: Collection LibreChat.messages ( 29a855ee-abf3-4d7a-abb2-7310628dc011 ) :: caused by :: E11000 duplicate key error collection: LibreChat.messages index: foobarbazbar_1 dup key: { foobarbazbar: null }
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant