Skip to content

Conversation

fenos
Copy link
Contributor

@fenos fenos commented Oct 3, 2025

What kind of change does this PR introduce?

Feature

What is the new behaviour?

Implement Vector Bucket data source

Supported Operations:

  • CreateIndex
  • DeleteIndex
  • GetIndex
  • ListIndexes
  • PutVectors
  • ListVectors
  • ListVectorBuckets
  • QueryVectors
  • DeleteVectors
  • GetVectorBucket
  • GetVectors

Authentication mechanisms:

  • SignV4
  • JWT service_role

Copy link

snyk-io bot commented Oct 3, 2025

Snyk checks have failed. 8 issues have been found so far.

Status Scanner Critical High Medium Low Total (8)
Code Security 0 8 0 0 8 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@fenos fenos force-pushed the feat/vector-buckets branch 6 times, most recently from 01797cc to a2715e7 Compare October 10, 2025 11:12
@coveralls
Copy link

coveralls commented Oct 10, 2025

Pull Request Test Coverage Report for Build 18461456855

Details

  • 2036 of 2315 (87.95%) changed or added relevant lines in 40 files are covered.
  • 23 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.8%) to 77.138%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/http/plugins/jwt.ts 3 5 60.0%
src/http/routes/vector/create-bucket.ts 43 45 95.56%
src/http/routes/vector/create-index.ts 64 66 96.97%
src/http/routes/vector/delete-bucket.ts 43 45 95.56%
src/http/routes/vector/delete-index.ts 51 53 96.23%
src/http/routes/vector/delete-vectors.ts 48 50 96.0%
src/http/routes/vector/get-bucket.ts 43 45 95.56%
src/http/routes/vector/get-index.ts 62 64 96.88%
src/http/routes/vector/get-vectors.ts 47 49 95.92%
src/http/routes/vector/list-buckets.ts 44 46 95.65%
Files with Coverage Reduction New Missed Lines %
src/http/plugins/signature-v4.ts 1 46.09%
src/storage/protocols/s3/signature-v4.ts 22 73.11%
Totals Coverage Status
Change from base Build 18352064985: 0.8%
Covered Lines: 23147
Relevant Lines: 29718

💛 - Coveralls

src/app.ts Outdated
app.register(routes.cdn, { prefix: 'cdn' })
app.register(routes.healthcheck, { prefix: 'health' })
app.register(routes.iceberg, { prefix: 'iceberg/v1' })
app.register(routes.vectors, { prefix: 'vectors' })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be singular ("vector")? to match: bucket, object, ...

: 400

if (statusCode === 500) {
console.log('error')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug log, remove

}
}

throw ERRORS.TransactionError('Transaction failed after maximum retries', lastError)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to make "after maximum retries" conditional on if there were retires, and/or maybe include the retry count in the error message to avoid confusion if we need to debug this in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure i understood this, but in isolation mode: 'serialize' retrying the transaction is a common pattern
if the max retries are reached means that there are too many concurrent operations, which makes this transaction fail

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just meant adding the count to the error message or only saying "after maximum retries" if there were retries as currently it says that even for requests that had no retries.

@fenos fenos force-pushed the feat/vector-buckets branch from a2715e7 to 6bf5f59 Compare October 13, 2025 09:35
@fenos fenos force-pushed the feat/vector-buckets branch from 6bf5f59 to f2ae953 Compare October 17, 2025 09:33
search: query.search,
})

return response.status(200).send(bucket)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  Cross-site Scripting (XSS)

Unsanitized input from an HTTP parameter flows into send, where it is used to render an HTML page returned to the user. This may result in a Cross-Site Scripting attack (XSS).

Line 119 | CWE-79 | Priority score 848 | Learn more about this vulnerability
Data flow: 14 steps

Step 1 - 3

const query = request.query

Step 4 - 7 src/http/routes/iceberg/bucket.ts#L112

Step 8 - 12 src/http/routes/iceberg/bucket.ts#L111

Step 13 - 14

return response.status(200).send(bucket)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants