Qdrant collections can't be listed in a hosted environement

[nicolas-geysse]

Bug Description

Hello,
I may create, list, and anything I want using http request node to connect a Cloud version but also a hosted version of Qdrant.
In the n8n Qdrant node, I may list the collections directly without any problems, but it fails for the hosted version.

To Reproduce

1.Host an instance of Qdrant in your infrastructure.
2.Create your credentials, the connection with the credentials is working fine
3.Use the Qdrant node to list collections.
4. :(
5. Do that with the Cloud version of Qdrant
6. :)

Expected behavior

As my connection between n8n and my hosted version of Qdrant, it should handle the same way the Cloud version.

Operating System

?

n8n Version

1.72.1

Node.js Version

20.18.0

Database

PostgreSQL

Execution mode

queue

[Joffcom]

Hey @nicolas-geysse,

We have created an internal ticket to look into this which we will be tracking as "N8N-8014"

[Joffcom]

Hey @nicolas-geysse

Just to check are you using localhost as the host address for your local qdrant instance?

[nicolas-geysse]

Hello @Joffcom
No, it's hosted on Railway (Qdrant as n8n), proxied with Cloudflare. (but fails also without it)

[HT-Moh]

I have the same issue with Qdrant, my instance works, but when using n8n node for Qdrant credentials are green but I can not list the collections.

curl 'https://myqdrant_host_name/collections' \
  -H 'accept: application/json, text/plain, */*' \
  -H 'accept-language: en-US,en;q=0.7' \
  -H 'api-key: mytoken=' \
  -H 'content-type: application/json' \
  -H 'priority: u=1, i' \
  -H 'x-inference-proxy: true'
{"result":{"collections":[{"name":"contacts"}]},"status":"ok","time":0.00001614}

[jeanpaul]

@nicolas-geysse For me it's working correctly when I run everything locally, even when adding an API key. Is there anything special about your setup?

[jeanpaul]

@nicolas-geysse Ah, maybe it's something else... I tried a bit more and came to the conclusion that there might be an issue with saving the credential if you change the URL after you've entered the API key. Can you try re-entering your API key in the credential and saving it? You may need to use the three-dot menu near "Qdrant Collection" to refresh the list:

[nicolas-geysse]

@jeanpaul Hello, I don't see anything special in my setup...I wonder if that might be related to IP v4 and v6 (not supported with n8n, but with Qdrant)
I've seen this topic that seems related
qdrant/qdrant#5545
I have this in my logs :

fetch failed

TypeError: fetch failed
    at node:internal/deps/undici/undici:13185:13
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at fetchJson (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/openapi-typescript-fetch/dist/cjs/fetcher.js:135:22)
    at /usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/dist/cjs/api-client.js:46:26
    at handler (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/openapi-typescript-fetch/dist/cjs/fetcher.js:156:16)
    at /usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/dist/cjs/api-client.js:32:24
    at handler (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/openapi-typescript-fetch/dist/cjs/fetcher.js:156:16)
    at fetchUrl (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/openapi-typescript-fetch/dist/cjs/fetcher.js:162:22)
    at Object.fun [as getCollections] (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/openapi-typescript-fetch/dist/cjs/fetcher.js:168:20)
    at QdrantClient.getCollections (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/dist/cjs/qdrant-client.js:836:26)
    at LoadOptionsContext.qdrantCollectionsSearch (/usr/local/lib/node_modules/n8n/node_modules/@n8n/n8n-nodes-langchain/dist/nodes/vector_store/shared/methods/listSearch.js:51:22)
    at DynamicNodeParametersController.getResourceLocatorResults (/usr/local/lib/node_modules/n8n/dist/controllers/dynamic-node-parameters.controller.js:37:16)
    at handler (/usr/local/lib/node_modules/n8n/dist/decorators/controller.registry.js:93:24)
    at /usr/local/lib/node_modules/n8n/dist/response-helper.js:110:26
 
Connect Timeout Error
ConnectTimeoutError: Connect Timeout Error
    at onConnectTimeout (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/node_modules/undici/lib/core/connect.js:186:24)
    at /usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/node_modules/undici/lib/core/connect.js:133:46
    at Immediate._onImmediate (/usr/local/lib/node_modules/n8n/node_modules/@qdrant/js-client-rest/node_modules/undici/lib/core/connect.js:174:9)
    at processImmediate (node:internal/timers:483:21)

[jeanpaul]

@nicolas-geysse did you see my comment on re-adding your credential? If that's the issue, there should be a fix released soon. If that doesn't, we should look into it some more.

[nicolas-geysse]

@jeanpaul yes, I did that, same result. But it's getting weirder as if I set an empty API key in the credentials, the connection is allowed ! If I enter anything, it also works !
So I double checked the API key, it does exists, and I've tested this node into n8n :

async function execute() {
    const baseUrl = "https://myhostedqdrant";
    const apiKey = "myAPIkey";

    const options = {
        uri: `${baseUrl}/collections`,
        method: 'GET',
        headers: {
            'Content-Type': 'application/json',
            'Accept': 'application/json',
            'api-key': apiKey
        },
        json: true,
        timeout: 15000
    };

    try {
        const data = await this.helpers.request(options);
        
        return [{
            json: {
                status: "success",
                collections: data,
                endpoint: baseUrl
            }
        }];

    } catch (error) {
        return [{
            json: {
                status: "error",
                message: error.message,
                endpoint: baseUrl,
                errorDetails: error.response ? error.response.data : null
            }
        }];
    }
}

return execute();

With an empty API key, or anything, it doesn't work (as expected). With the correct API key, no problem, I do have the list of my collections.

[jeanpaul]

We're aware that the credential-test (on the credential-screen) is not working as expected (it's requesting the wrong path that always succeeds, whether or not you provided correct credentials). However, if you added your API key again to the credential, saved it, and refreshed the collection-list while still coming up empty, I'm unsure of how to reproduce your issue. It might be that your host or cloudflare are doing something different (blocking the request maybe?)

[nicolas-geysse]

@jeanpaul I'd agree with your point if the custom code node given previously was not running smoothly on the same n8n instance that does not list the collections with the n8n Qdrant vector store...

In the .env I've added
NODE_FUNCTION_ALLOW_EXTERNAL=@qdrant/js-client-rest
to do so, and, here it works perfectly fine, so it does not seems to be a network issue here :(

I suspect that the node code might be more consistent in the IPv6 handling than the qdrant vector store node ? (I've recreated the credentials, refreshed, multiple times...)

Maybe it is on the Qdrant side that might handle connections in a more robust way on the cloud version ?
Or I'm doing something wrong, but I'm missing what...

If you use a n8n in queue mode with Qdrant locally, there's no issue for you I suppose ?

[jeanpaul]

I didn't try queue mode, but that shouldn't have any effect on retrieving the collections. The collection search method in n8n uses the @qdrant/js-client-rest method as well, so I don't know what could be the issue. Could you try and connect with an HTTP-node to your qdrant server and retrieve the collections? Please use "include response headers and status" from the Response option.

[nicolas-geysse]

@jeanpaul it works ! Something I see is that the port 6333 must not be set in that case.

[nicolas-geysse]

I've created a tunnel with cloudflare :
Zero Trust > Networks > Tunnels

Deleted the CNAME entry

Created into Railway :
Connect to your service over TCP using a proxied domain and port
railway.proxy:12345 -> :6333

And it finally works but...In the log of n8n:
Api key is used with unsecure connection.

I think I may dig to internally share the service directly into Railway to connect n8n / Qdrant to avoid all this.

I also think this might be related:
qdrant/qdrant#3890

[jeanpaul]

Hi @nicolas-geysse, glad to hear this resolved the issue. If you don't use TLS for the connection, it is indeed not a secured connection.

Thanks for reporting back.

[nicolas-geysse]

Hello @jeanpaul, thanks, unfortunaltely , to me it's not prod ready with such security issue :(

[jeanpaul]

I don't know how well the qdrant SDK plays with TLS. The n8n code uses the SDK version that's supplied via LangChain. It could be an upstream issue, but I'm not able to help you out with that, sorry.