Skip to content

Conversation

@djeebus
Copy link
Contributor

@djeebus djeebus commented Oct 16, 2025

This allows us to cache, compress, etc.

  • Add a new OpenSeekableObject method to the storage provider that is used to read seekable files
  • Remove the ReadAt method from the ObjectProvider interface (only used by SeekableObjectProvider).
  • Remove the Delete method from the ObjectProvider interface (never used).
  • Add a SeekableObjectProvider interface.
  • Add new ObjectType and SeekableObjectType types that defines which file type is being passed in, in order to help the compiler prevent mistakes

Note

Introduces separate seekable/non-seekable storage interfaces with typed OpenObject/OpenSeekableObject, updates providers, caching, orchestrator code, mocks, and tests accordingly.

  • Storage API:
    • Split interfaces into ObjectProvider (non-seekable) and SeekableObjectProvider (seekable); removed ReadAt/Delete from ObjectProvider, added Exists.
    • Added ObjectType and SeekableObjectType enums; updated StorageProvider with OpenObject(ctx, path, ObjectType) and OpenSeekableObject(ctx, path, SeekableObjectType).
    • Refactored caching: new CachedObjectProvider and CachedSeekableObjectProvider; moved chunked read/size logic to seekable cache; reorganized cache helpers.
  • Providers (GCP/AWS/FS):
    • Implement new interfaces and methods; added Exists; adjusted OpenObject/OpenSeekableObject signatures.
  • Orchestrator:
    • Updated commands and template code to pass object kinds and use OpenSeekableObject for memfile/rootfs and OpenObject for headers/metadata/snapfile.
    • storage_diff: constructor now returns (*StorageDiff, error); added diff→object type mapping; use OpenSeekableObject.
    • Template build/upload paths now typed; server upload checks Exists instead of Size.
  • Tests & Mocks:
    • Replaced old mocks with mockobjectprovider and mockseekableobjectprovider; updated unit tests for new interfaces and behaviors.
  • Config:
    • .mockery.yaml updated to generate mocks for new interfaces.

Written by Cursor Bugbot for commit 77c3db5. This will update automatically on new commits. Configure here.

this allows us to cache, compress, etc.
cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@djeebus djeebus requested a review from dobrac October 30, 2025 16:06
cursor[bot]

This comment was marked as outdated.

Copy link
Contributor

@dobrac dobrac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good. Small nits there, so approving!

Just wondering, wouldn't it be better to do instead of:
obj, err := s.OpenObject(ctx, files.StorageMetadataPath(), storage.MetadataObjectType)

something like this?
obj, err := s.OpenObject(ctx, storage.MetadataObjectType{ Path: files.StorageMetadataPath()})
(we could then return the storage.MetadataObjectType directly instead of the string path)

@dobrac dobrac assigned dobrac and unassigned ValentaTomas Oct 31, 2025
@djeebus djeebus merged commit 6e5cad0 into main Nov 7, 2025
27 checks passed
@djeebus djeebus deleted the split-large-small-read-write branch November 7, 2025 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants