Skip to content

CB‐Spider Object Storage API Guide (Basic Auth, JSON Format)

Jump to bottom Edit New page
ByoungSeob Kim edited this page Apr 3, 2026 · 11 revisions

Related Links

CB-Spider Object Storage API Guide (Basic Auth / JSON Format)

This guide uses JSON format (Accept: application/json) for CB-Spider-specific integrations and is not compatible with standard AWS S3 tools or SDKs, which expect XML. For AWS S3-compatible usage, refer to the CB-Spider Object Storage API Guide (SigV4 / XML-Format).

Key Features

  • Multi-Cloud Support: Unified management of Object Storage across AWS, IBM Cloud, GCP, and other CSPs
  • JSON Format (CB-Spider Specific): Returns JSON responses via Accept: application/json
  • PreSigned URL REST API: Provides PreSigned URL generation REST API
  • Simple Authentication: Easy authentication with ConnectionName parameter

JSON Format Usage

To receive JSON responses, include the following header in your requests:

Accept: application/json

Without this header, the API returns XML format responses (default S3 behavior).

API List

1. Bucket Management

Function HTTP Method Endpoint Example
List Buckets GET /spider/s3 Example
Create Bucket PUT /spider/s3/{bucket-name} Example
Get Bucket Info GET /spider/s3/{bucket-name} Example
Check Bucket Exists HEAD /spider/s3/{bucket-name} Example
Get Bucket Location GET /spider/s3/{bucket-name}?location Example
Delete Bucket DELETE /spider/s3/{bucket-name} Example

2. Object Management

Function HTTP Method Endpoint Example
Upload Object (File) PUT /spider/s3/{bucket-name}/{object-key} Example
Upload Object (Form) POST /spider/s3/{bucket-name} Example
Download Object GET /spider/s3/{bucket-name}/{object-key} Example
Get Object Info HEAD /spider/s3/{bucket-name}/{object-key} Example
Delete Object DELETE /spider/s3/{bucket-name}/{object-key} Example
Delete Multiple Objects POST /spider/s3/{bucket-name}?delete Example

3. Multipart Upload

Function HTTP Method Endpoint Example
Initiate Multipart Upload POST /spider/s3/{bucket-name}/{object-key}?uploads Example
Upload Part PUT /spider/s3/{bucket-name}/{object-key}?uploadId={id}&partNumber={num} Example
Complete Multipart Upload POST /spider/s3/{bucket-name}/{object-key}?uploadId={id} Example
Abort Multipart Upload DELETE /spider/s3/{bucket-name}/{object-key}?uploadId={id} Example
List Parts GET /spider/s3/{bucket-name}/{object-key}?uploadId={id}&list-type=parts Example
List Multipart Uploads GET /spider/s3/{bucket-name}?uploads Example

4. Versioning Management

Function HTTP Method Endpoint Example
Get Bucket Versioning GET /spider/s3/{bucket-name}?versioning Example
Set Bucket Versioning PUT /spider/s3/{bucket-name}?versioning Example
List Object Versions GET /spider/s3/{bucket-name}?versions Example
Delete Versioned Object DELETE /spider/s3/{bucket-name}/{object-key}[?versionId=version-id] Example

5. CORS Management

Function HTTP Method Endpoint Example
Get Bucket CORS GET /spider/s3/{bucket-name}?cors Example
Set Bucket CORS PUT /spider/s3/{bucket-name}?cors Example
Delete Bucket CORS DELETE /spider/s3/{bucket-name}?cors Example

6. CB-Spider Special Features

Function HTTP Method Endpoint Example
Generate PreSigned URL (Download) GET /spider/s3/presigned/download/{bucket-name}/{object-key} Example
Generate PreSigned URL (Upload) GET /spider/s3/presigned/upload/{bucket-name}/{object-key} Example
Force Empty Bucket DELETE /spider/s3/{bucket-name}?empty Example
Force Delete Bucket DELETE /spider/s3/{bucket-name}?force Example

Note: CB-Spider special features are unique to CB-Spider and not part of AWS S3 standard.

API Usage Examples

1. Bucket Management

List Buckets

Prerequisites: Create a bucket and upload a sample object so the response contains meaningful data.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Upload a sample object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/sample.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/sample.txt -H "Content-Type: text/plain" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3?ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "Owner": {
    "ID": "aws-config01",
    "DisplayName": "aws-config01"
  },
  "Buckets": {
    "Bucket": [
      {
        "IId": {
          "NameId": "spider-test-bucket",
          "SystemId": "spider-test-bucket"
        },
        "CreationDate": "2026-03-30T06:23:25Z"
      }
    ]
  }
}

Cleanup (optional): Remove the created resources if no longer needed.

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Create Bucket

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -H "Accept: application/json"

Response:

HTTP/1.1 200 OK
Location: /spider-test-bucket
Vary: Origin
X-Amz-Id-2: 1774853768
X-Amz-Request-Id: 1774853768
Date: Mon, 30 Mar 2026 06:56:08 GMT
Content-Length: 0

Get Bucket Info

Prerequisites: Create a bucket and upload a sample object so the Contents field is populated.

# Upload a sample object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/sample.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/sample.txt -H "Content-Type: text/plain" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Prefix": "",
  "Marker": "",
  "MaxKeys": 1000,
  "IsTruncated": false,
  "Contents": [
    {
      "Key": "sample.txt",
      "LastModified": "2026-03-30T07:02:47Z",
      "ETag": "9dbd9a7eb648df939897919cb49d6fc2",
      "Size": 33,
      "StorageClass": "STANDARD"
    }
  ]
}

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/sample.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Check Bucket Exists (HEAD)

Prerequisites: Create a bucket to test the existing-bucket response.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -I "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Response for existing bucket:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1774854256
X-Amz-Request-Id: 1774854256
Date: Mon, 30 Mar 2026 07:04:16 GMT

Response for non-existing bucket:

HTTP/1.1 403 Forbidden
Vary: Origin
Date: Mon, 30 Mar 2026 07:06:44 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Get Bucket Location

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?location&ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "LocationConstraint": "ap-southeast-2"
}

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Delete Bucket

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774854411
X-Amz-Request-Id: 1774854411
Date: Mon, 30 Mar 2026 07:06:51 GMT

2. Object Management

Upload Object (From File)

Prerequisites: Create a bucket and prepare a local file.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Prepare local file
echo "Hello CB-Spider JSON Guide Test!" > /tmp/local-file.txt

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt \
  -H "Content-Type: text/plain" \
  -H "Accept: application/json"

Response:

HTTP/1.1 200 OK
Etag: 9dbd9a7eb648df939897919cb49d6fc2
Vary: Origin
X-Amz-Id-2: 1774854526
X-Amz-Request-Id: 1774854526
Date: Mon, 30 Mar 2026 07:08:46 GMT
Content-Length: 0

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Upload Object (Form Data)

Prerequisites: Create a bucket and prepare a local file.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Prepare local file
echo "Hello CB-Spider JSON Guide Test!" > /tmp/local-file.txt

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -F "key=test-file.txt" \
  -F "file=@/tmp/local-file.txt" \
  -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Key": "test-file.txt"
}

Notes:

  • The key form field specifies the object name/key
  • The file form field contains the file to upload
  • Optional Content-Type form field can be specified
  • Response includes ETag header for uploaded object
  • Supports browser-based HTML form uploads

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Download Object

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Upload object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  -o /tmp/downloaded-file.txt -H "Accept: application/json"

Response:

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Get Object Info (HEAD)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Upload object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -I "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"

Response:

HTTP/1.1 200 OK
Content-Length: 33
Content-Type: text/plain; charset=utf-8
Etag: 9dbd9a7eb648df939897919cb49d6fc2
Last-Modified: Mon, 30 Mar 2026 07:23:26 GMT
Vary: Origin
X-Amz-Id-2: 1774855409
X-Amz-Request-Id: 1774855409
Date: Mon, 30 Mar 2026 07:23:29 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Delete Object

Prerequisites: Create a bucket and upload an object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"
# Upload object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/local-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/local-file.txt -H "Content-Type: text/plain" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" -H "Accept: application/json"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774855503
X-Amz-Request-Id: 1774855503
Date: Mon, 30 Mar 2026 07:25:03 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" -H "Accept: application/json"

Delete Multiple Objects

Prerequisites: Create a bucket and upload multiple objects.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
for f in file1.txt file2.txt file3.txt; do
  echo "Content of $f" > /tmp/$f
  curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/$f?ConnectionName=aws-config01" \
    --data-binary @/tmp/$f -H "Content-Type: text/plain"
done

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket?delete&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "Delete": {
      "Objects": [
        {"Key": "file1.txt"},
        {"Key": "file2.txt"},
        {"Key": "file3.txt"}
      ],
      "Quiet": false
    }
  }'

JSON Response Example:

{
  "Deleted": [
    {
      "Key": "file1.txt"
    },
    {
      "Key": "file2.txt"
    },
    {
      "Key": "file3.txt"
    }
  ]

Notes:

  • The delete query parameter is required
  • Uses JSON format with Content-Type: application/json and Delete.Objects array
  • Supports deleting multiple objects in a single request
  • Returns detailed results for each deleted object
  • Quiet parameter: Set to true to suppress successful deletion responses

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

3. Multipart Upload

Important Notes:

  • Part Size Limit: Each part (except the last one) must be at least 5MB in size for AWS S3 compatibility
  • ETag Format: Multipart files have ETag format as {hash}-{part_count} (e.g., 50f9c71a2e1d2cd7706f6dfd0b12c9fd-3)
  • Upload ID Lifecycle: CB-Spider automatically invalidates Upload IDs after a period of inactivity (unlike AWS S3 standard where Upload IDs never expire). It's recommended to upload parts immediately after initiation

Initiate Multipart Upload

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream" \
  -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Key": "large-multipart-file.bin",
  "UploadId": "DyWKrqlXNp52pMra29i0pYsNOfSwBKTky64I57qTWnJai6ntfbq34nunPJQfQYBtJkoasU1B1D5i0PRYRxkVS9N_ODo.Yp_TsewMVo3Mj1QL2HXTE7BQ5ySQguUIN3ZujX0IxMFD1vJfRxHdwXFweg--"
}

Upload Part

Prerequisites: Initiate a multipart upload and note the UploadId from the response. Prepare large part files (at least 5MB each, except the last part).

# Create a 6MB test file
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6

Request:

# Upload Part 1 (6MB)
curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&partNumber=1&ConnectionName=aws-config01" \
  --data-binary @/tmp/large-part1.bin \
  -H "Content-Type: application/octet-stream"

Response:

HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Etag: 7387d9e076aeb5b8871e36e773868635
Vary: Origin
X-Amz-Id-2: 1774856993
X-Amz-Request-Id: 1774856993
Date: Mon, 30 Mar 2026 07:49:53 GMT
Content-Length: 0

Complete Multipart Upload

Prerequisites: Upload all required parts and have their ETags ready.

Note: Replace {UploadId} with the UploadId from the Initiate Multipart Upload response, and replace {ETag of Part 1} with the ETag value from the Upload Part response (e.g., "7387d9e076aeb5b8871e36e773868635").

Request:

# Replace {UploadId} with UploadId from Initiate Multipart Upload response
# Replace {ETag of Part 1} with ETag from Upload Part response (e.g. "7387d9e076aeb5b8871e36e773868635")
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId from Initiate response}&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "Parts": [
      {"PartNumber": 1, "ETag": "{ETag of Part 1}"}
    ]
  }'

JSON Response Example:

{
  "Location": "/spider-test-bucket/large-multipart-file.bin",
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Key": "large-multipart-file.bin",
  "ETag": "7e9b31e2e1fbea79b26cc366781259ac-1"
}

Note: The ETag ending with -1 indicates this file was assembled from 1 part.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Abort Multipart Upload

Prerequisites: Initiate a multipart upload and note the UploadId.

# Create bucket if needed
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream" -H "Accept: application/json"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response above.

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/abort-test-file.bin?uploadId={UploadId from Initiate response}&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774857693
X-Amz-Request-Id: 1774857693
Date: Mon, 30 Mar 2026 08:01:33 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

List Parts

Prerequisites: Initiate a multipart upload, upload at least one part, and note the UploadId.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload and capture UploadId
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream" -H "Accept: application/json"
# Upload Part 1 (6MB) using the UploadId from above
dd if=/dev/urandom of=/tmp/large-part1.bin bs=1M count=6
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&partNumber=1&ConnectionName=aws-config01" \
  --data-binary @/tmp/large-part1.bin -H "Content-Type: application/octet-stream"

Request:

Note: Replace {UploadId} with the UploadId returned from the Initiate Multipart Upload response in the Prerequisites above.

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&list-type=parts&ConnectionName=aws-config01" -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Key": "large-multipart-file.bin",
  "UploadId": "tzmFwRcpKwbvbXhdac_FpqPznAR9bd3DTp6o9Z7zOwkJHOBvbPvbXhN3_hl9BzZ55qxHfW_zABVUXY67nEjIBFWt870hCjJwof7o7dsxsHAcAUuVKOlv5N8rdcKbGBvSx0fJJWr4wkmf7rxbbb6B_Q--",
  "PartNumberMarker": 0,
  "NextPartNumberMarker": 1,
  "MaxParts": 1000,
  "IsTruncated": false,
  "Parts": [
    {
      "PartNumber": 1,
      "LastModified": "2026-03-30T08:36:09Z",
      "ETag": "\"89c8e427615fc3ad7b37d1c0454e10e0\"",
      "Size": 6291456
    }
  ],
  "Initiator": {
    "ID": "arn:aws:iam::123456789123:user/spider",
    "DisplayName": "spider"
  },
  "Owner": {
    "ID": "7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d",
    "DisplayName": ""
  },
  "StorageClass": "STANDARD"
}

Optional Parameters:

  • part-number-marker: Start listing after this part number
  • max-parts: Maximum number of parts to return (default: 1000)

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

List Multipart Uploads

Prerequisites: Initiate at least one multipart upload (do not complete or abort it).

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Initiate upload (leave it pending)
curl -u admin:your-secure-password -X POST "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploads&ConnectionName=aws-config01" \
  -H "Content-Type: application/octet-stream" -H "Accept: application/json"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?uploads&ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "KeyMarker": "",
  "UploadIDMarker": "",
  "NextKeyMarker": "large-multipart-file.bin",
  "NextUploadIDMarker": "BD7lUV6tsO8y40ogABkRaf5kNF3azVirP3Uq54nEHLh0_nS0GPN9zsMI903hxTghq3lAfFbnnSeyUAFY8ujApmPvWM6dbPptHVcftDw1WsSp1Lh83aBSSX6SvQSCHsK69S9A.PJWYAz5yIUIh2Y3ng--",
  "MaxUploads": 1000,
  "IsTruncated": false,
  "Uploads": [
    {
      "Key": "large-multipart-file.bin",
      "UploadID": "BD7lUV6tsO8y40ogABkRaf5kNF3azVirP3Uq54nEHLh0_nS0GPN9zsMI903hxTghq3lAfFbnnSeyUAFY8ujApmPvWM6dbPptHVcftDw1WsSp1Lh83aBSSX6SvQSCHsK69S9A.PJWYAz5yIUIh2Y3ng--",
      "Initiated": "2026-03-30T08:28:59Z",
      "StorageClass": "STANDARD",
      "Initiator": {
        "ID": "arn:aws:iam::123456789123:user/spider",
        "DisplayName": "spider"
      },
      "Owner": {
        "ID": "7d9f77b078e963b2aca023ffa7dc39a3cac3b138397f2214d2d66e33bee3384d",
        "DisplayName": ""
      }
    }
  ],
  "Prefix": "",
  "Delimiter": ""
}

Optional Parameters:

  • prefix: Only list uploads with keys beginning with this prefix
  • key-marker: Start listing after this key name
  • upload-id-marker: Start listing after this upload ID
  • delimiter: Character used to group keys
  • max-uploads: Maximum number of uploads to return (default: 1000)

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/large-multipart-file.bin?uploadId={UploadId}&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Multipart Upload Technical Notes

Upload ID Lifecycle:

  • Upload IDs must be used immediately after initiation
  • CB-Spider automatically invalidates unused Upload IDs (differs from AWS S3 standard)
  • Always initiate a new multipart upload for each file upload session

Part Size Requirements:

  • Each part (except the last) must be at least 5MB for AWS S3 compatibility

ETag Format:

  • Single uploads: Standard MD5 hash (e.g., "f1e52a8409901065abbeb82ed79ca42d")
  • Multipart uploads: Hash with part count suffix (e.g., "f4c473f8709573ad54fe52859f100ff8-1")

4. Versioning Management

Get Bucket Versioning

Prerequisites: Create a bucket and enable versioning.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" -d '{"Status": "Enabled"}'

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "Status": "Enabled"
}

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Set Bucket Versioning

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -i -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "Status": "Enabled"
  }'

Response:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1774861379
X-Amz-Request-Id: 1774861379
Date: Mon, 30 Mar 2026 09:02:59 GMT
Content-Length: 0

To suspend versioning:

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "Status": "Suspended"
  }'

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

List Object Versions

Prerequisites: Create a bucket, enable versioning, and upload the same object multiple times to create multiple versions.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" -d '{"Status": "Enabled"}'
# Upload version 1
echo "Version 1 content" > /tmp/file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v1.txt -H "Content-Type: text/plain"
# Upload version 2 (same key)
echo "Version 2 content" > /tmp/file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v2.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versions&ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Prefix": "",
  "KeyMarker": "",
  "VersionIdMarker": "",
  "NextKeyMarker": "",
  "NextVersionIdMarker": "",
  "MaxKeys": 1000,
  "IsTruncated": false,
  "Version": [
    {
      "Key": "same-file.txt",
      "VersionId": "9tUe9YparyuVtZqzLTvlVLLLyHJVLuJ2",
      "IsLatest": true,
      "LastModified": "2026-03-30T10:03:28Z",
      "ETag": "e0ef76395fbb2af0b3213158344bd881",
      "Size": 18,
      "StorageClass": "STANDARD",
      "Owner": {
        "ID": "aws-config01",
        "DisplayName": "aws-config01"
      }
    },
    {
      "Key": "same-file.txt",
      "VersionId": "xPF5C7xP7pQyCIHVXvhOZzGFoTpqtWqz",
      "IsLatest": false,
      "LastModified": "2026-03-30T10:03:27Z",
      "ETag": "0e8e102783f3fd35b701b8de312415e5",
      "Size": 18,
      "StorageClass": "STANDARD",
      "Owner": {
        "ID": "aws-config01",
        "DisplayName": "aws-config01"
      }
    }
  ],
  "DeleteMarker": []
}

Cleanup (optional):

# Force delete bucket with all versions
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Delete Versioned Object

Prerequisites: Create a bucket with versioning enabled, upload an object twice (same key), then list versions to get the versionId.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Enable versioning
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?versioning&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" -d '{"Status": "Enabled"}'
# Upload 2 versions
echo "Version 1 content" > /tmp/file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v1.txt -H "Content-Type: text/plain"
echo "Version 2 content" > /tmp/file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/file-v2.txt -H "Content-Type: text/plain"

List Versions Response Example (note the VersionId values — use one in the Request below):

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?versions&ConnectionName=aws-config01" -H "Accept: application/json"

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Prefix": "",
  "KeyMarker": "",
  "VersionIdMarker": "",
  "NextKeyMarker": "",
  "NextVersionIdMarker": "",
  "MaxKeys": 1000,
  "IsTruncated": false,
  "Version": [
    {
      "Key": "test-file.txt",
      "VersionId": "W1vJfH0cExg.E8g4L4IMLDUFfLZWrWrN",
      "IsLatest": true,
      "LastModified": "2026-03-30T10:05:04Z",
      "ETag": "e0ef76395fbb2af0b3213158344bd881",
      "Size": 18,
      "StorageClass": "STANDARD",
      "Owner": {
        "ID": "aws-config01",
        "DisplayName": "aws-config01"
      }
    },
    {
      "Key": "test-file.txt",
      "VersionId": "WBAQhmpA1v5_Bajfa_18X5HBesHqlBvg",
      "IsLatest": false,
      "LastModified": "2026-03-30T10:05:03Z",
      "ETag": "0e8e102783f3fd35b701b8de312415e5",
      "Size": 18,
      "StorageClass": "STANDARD",
      "Owner": {
        "ID": "aws-config01",
        "DisplayName": "aws-config01"
      }
    }
  ],
  "DeleteMarker": []
}

Request:

Note: Replace {versionId} with a VersionId from the List Versions response above.

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?versionId={VersionId}&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774865643
X-Amz-Request-Id: 1774865643
Date: Mon, 30 Mar 2026 10:14:03 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Versioning Important Notes

Creating Multiple Versions: When versioning is enabled, uploading an object with the same key creates a new version:

# Upload version 1
echo "Version 1 content" > file-v1.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @file-v1.txt -H "Content-Type: text/plain"

# Upload version 2 (same key)
echo "Version 2 content" > file-v2.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/same-file.txt?ConnectionName=aws-config01" \
  --data-binary @file-v2.txt -H "Content-Type: text/plain"

Versioning States:

  • Enabled: New versions are created for each upload
  • Suspended: New uploads don't create versions (but existing versions remain)
  • Unversioned: Default state (no versioning)

5. CORS Management

Get CORS Configuration

Prerequisites: Create a bucket and set a CORS configuration.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Set CORS configuration
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -d '{"CORSRule": [{"AllowedOrigin": ["*"], "AllowedMethod": ["GET", "PUT", "POST", "DELETE"], "AllowedHeader": ["*"], "ExposeHeader": ["ETag", "x-amz-server-side-encryption", "x-amz-request-id", "x-amz-id-2"], "MaxAgeSeconds": 3000}]}'

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Response Example (when CORS is configured):

{
  "CORSRule": [
    {
      "AllowedOrigin": [
        "*"
      ],
      "AllowedMethod": [
        "GET",
        "PUT",
        "POST",
        "DELETE"
      ],
      "AllowedHeader": [
        "*"
      ],
      "ExposeHeader": [
        "ETag",
        "x-amz-server-side-encryption",
        "x-amz-request-id",
        "x-amz-id-2"
      ],
      "MaxAgeSeconds": 3000
    }
  ]
}

JSON Error Response (when CORS is not configured):

{
   "Code" : "NoSuchCORSConfiguration",
   "Message" : "The CORS configuration for bucket 'spider-test-bucket' does not exist",
   "RequestId" : "1774866105",
   "Resource" : "/spider-test-bucket"
}

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Set CORS Configuration

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Basic CORS Configuration:

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "CORSRule": [
      {
        "AllowedOrigin": ["*"],
        "AllowedMethod": ["GET", "PUT", "POST", "DELETE"],
        "AllowedHeader": ["*"],
        "MaxAgeSeconds": 3000
      }
    ]
  }'

Advanced CORS Configuration (Multiple Rules):

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "CORSRule": [
      {
        "AllowedOrigin": ["https://example.com", "https://app.example.com"],
        "AllowedMethod": ["GET", "PUT"],
        "AllowedHeader": ["Content-Type", "Authorization"],
        "ExposeHeader": ["ETag"],
        "MaxAgeSeconds": 1800
      },
      {
        "AllowedOrigin": ["*"],
        "AllowedMethod": ["GET"],
        "MaxAgeSeconds": 300
      }
    ]
  }'

Response:

HTTP/1.1 200 OK
Vary: Origin
X-Amz-Id-2: 1774866299
X-Amz-Request-Id: 1774866299
Date: Mon, 30 Mar 2026 10:24:59 GMT
Content-Length: 0

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Test CORS with OPTIONS Request

Note: CORS preflight requests are sent by browsers without credentials. Use plain curl (no auth).

Prerequisites: Create a bucket and set a CORS configuration.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload a sample object
echo "Hello" > /tmp/test-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/test-file.txt -H "Content-Type: text/plain"
# Set CORS configuration
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -d '{"CORSRule": [{"AllowedOrigin": ["*"], "AllowedMethod": ["GET", "PUT", "POST", "DELETE"], "AllowedHeader": ["*"], "MaxAgeSeconds": 3000}]}'

Request:

curl -u admin:your-secure-password -X OPTIONS "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  -H "Origin: https://example.com" \
  -H "Access-Control-Request-Method: PUT" \
  -H "Access-Control-Request-Headers: Content-Type" \
  -v

Response:

*   Trying 127.0.0.1:1024...
* Connected to localhost (127.0.0.1) port 1024 (#0)
* Server auth using Basic with user 'admin'
> OPTIONS /spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01 HTTP/1.1
> Host: localhost:1024
> Authorization: Basic YWRtaW46eW91ci1zZWN1cmUtcGFzc3dvcmQ=
> User-Agent: curl/7.81.0
> Accept: */*
> Origin: https://example.com
> Access-Control-Request-Method: PUT
> Access-Control-Request-Headers: Content-Type
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< Access-Control-Allow-Headers: Content-Type
< Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE
< Access-Control-Allow-Origin: *
< Allow: OPTIONS, DELETE, GET, HEAD, POST, PUT
< Vary: Origin
< Vary: Access-Control-Request-Method
< Vary: Access-Control-Request-Headers
< Date: Mon, 30 Mar 2026 10:27:47 GMT
<
* Connection #0 to host localhost left intact

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Delete CORS Configuration

Prerequisites: Create a bucket and set a CORS configuration.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Set CORS configuration
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01" \
  -H "Content-Type: application/json" \
  -d '{"CORSRule": [{"AllowedOrigin": ["*"], "AllowedMethod": ["GET"], "MaxAgeSeconds": 300}]}'

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?cors&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774866635
X-Amz-Request-Id: 1774866635
Date: Mon, 30 Mar 2026 10:30:35 GMT

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

CORS Configuration Elements

Element Description Required Example
AllowedOrigin Specifies the origins allowed to access the bucket Yes * or https://example.com
AllowedMethod Specifies the HTTP methods allowed Yes GET, PUT, POST, DELETE
AllowedHeader Specifies the headers allowed in the request No *, Content-Type, Authorization
ExposeHeader Specifies which headers are exposed to the browser No ETag, x-amz-request-id
MaxAgeSeconds Specifies how long the browser can cache the preflight response No 3600 (1 hour)

CORS Configuration Examples

Allow All Origins (Development):

{
  "CORSRule": [
    {
      "AllowedOrigin": ["*"],
      "AllowedMethod": ["GET", "PUT", "POST", "DELETE"],
      "AllowedHeader": ["*"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Production Configuration (Specific Domains):

{
  "CORSRule": [
    {
      "AllowedOrigin": ["https://example.com", "https://app.example.com"],
      "AllowedMethod": ["GET", "PUT"],
      "AllowedHeader": ["Content-Type", "Authorization"],
      "ExposeHeader": ["ETag"],
      "MaxAgeSeconds": 1800
    }
  ]
}

Read-Only Access:

{
  "CORSRule": [
    {
      "AllowedOrigin": ["*"],
      "AllowedMethod": ["GET"],
      "AllowedHeader": ["Range"],
      "ExposeHeader": ["Content-Length", "Content-Range"],
      "MaxAgeSeconds": 300
    }
  ]
}

CORS Best Practices

  1. Security: Use specific origins instead of * in production
  2. Performance: Set appropriate MaxAgeSeconds to reduce preflight requests
  3. Headers: Only allow necessary headers to minimize security risks
  4. Methods: Only specify required HTTP methods
  5. Testing: Use browser developer tools to verify CORS behavior

6. CB-Spider Special Features

Generate PreSigned URL (Download)

Prerequisites: Create a bucket and upload a sample object.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload object
echo "Hello CB-Spider JSON Guide Test!" > /tmp/test-file.txt
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01" \
  --data-binary @/tmp/test-file.txt -H "Content-Type: text/plain"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/presigned/download/spider-test-bucket/test-file.txt?ConnectionName=aws-config01&expires=3600" \
  -H "Accept: application/json"

JSON Response Example:

{
  "PresignedURL": "https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/test-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAZH5OR44MIE5O2C6Y%2F20260325%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20260325T042239Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=1d2d8a284f5cf681b4d126dd9d7c9439e54f0c2c3568fbc7d0704eef2e30faac",
  "Expires": 3600,
  "Method": "GET"
}

Optional Parameters:

  • expires: URL expiration time in seconds (default: 3600)
  • method: HTTP method for the URL (default: GET, also supports PUT)
  • response-content-disposition: Content-Disposition header for downloads

Usage Example:

# Download a file using the generated PreSigned URL
curl -X GET "{PresignedURL}" -o /tmp/downloaded-file.txt

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Generate PreSigned URL (Upload)

Prerequisites: Create a bucket first.

curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Request:

curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/presigned/upload/spider-test-bucket/test-file.txt?ConnectionName=aws-config01&expires=3600" \
  -H "Accept: application/json"

JSON Response Example:

{
  "PresignedURL": "https://spider-test-bucket.s3.dualstack.ap-southeast-2.amazonaws.com/test-file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAZH5OR44MIE5O2C6Y%2F20260325%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20260325T042239Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=b793567cee15cbd60553c8cb18be304f783e7ccf523ccae42d7fa2d9763ca7eb",
  "Expires": 3600,
  "Method": "PUT"
}

Optional Parameters:

  • expires: URL expiration time in seconds (default: 3600)

Usage Example:

# Create a file to upload
echo "Hello CB-Spider S3 SigV4 Test!" > /tmp/local-file.txt
# Upload a file using the generated PreSigned URL
curl -X PUT "{PresignedURL}" --data-binary "@/tmp/local-file.txt" -H "Content-Type: text/plain"

Note: This feature is unique to CB-Spider and not part of AWS S3 standard.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket/test-file.txt?ConnectionName=aws-config01"
curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Force Empty Bucket

Prerequisites: Create a bucket and upload some objects.

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
for f in file1.txt file2.txt file3.txt; do
  echo "Content of $f" > /tmp/$f
  curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/$f?ConnectionName=aws-config01" \
    --data-binary @/tmp/$f -H "Content-Type: text/plain"
done

Request:

curl -u admin:your-secure-password -i -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?empty&ConnectionName=aws-config01" -H "Accept: application/json"

Response:

HTTP/1.1 204 No Content
Vary: Origin
X-Amz-Id-2: 1774867989
X-Amz-Request-Id: 1774867989
Date: Mon, 30 Mar 2026 10:53:09 GMT

JSON Error Response (when the ?empty parameter is not used):

  "Code": "BucketNotEmpty",
  "Message": "The bucket you tried to delete is not empty. It contains 3 objects. Use force=true parameter to force delete.",
  "Resource": "/spider-test-bucket",
  "RequestId": "1774868480"
}

Verification:

# Check that bucket is now empty
curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Empty Bucket Response:

{
  "IId": {
    "NameId": "spider-test-bucket",
    "SystemId": "spider-test-bucket"
  },
  "Prefix": "",
  "Marker": "",
  "MaxKeys": 1000,
  "IsTruncated": false,
  "Contents": []
}

Note: This feature is unique to CB-Spider and not part of AWS S3 standard. The ?empty query parameter is required as a safety mechanism.

Cleanup (optional):

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"

Force Delete Bucket

Prerequisites: Create a bucket and upload some objects (to demonstrate the non-empty case).

# Create bucket
curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01"
# Upload objects
for f in file1.txt file2.txt file3.txt; do
  echo "Content of $f" > /tmp/$f
  curl -u admin:your-secure-password -X PUT "http://localhost:1024/spider/s3/spider-test-bucket/$f?ConnectionName=aws-config01" \
    --data-binary @/tmp/$f -H "Content-Type: text/plain"
done

Request:

curl -u admin:your-secure-password -X DELETE "http://localhost:1024/spider/s3/spider-test-bucket?force&ConnectionName=aws-config01"

Response:

HTTP/1.1 204 No Content

JSON Error Response (bucket not empty, without force):

{
  "Code": "BucketNotEmpty",
  "Message": "The bucket you tried to delete is not empty. It contains 3 objects. Use force=true parameter to force delete.",
  "Resource": "/spider-test-bucket",
  "RequestId": "1774871856"
}

Verification:

# Check that bucket no longer exists
curl -u admin:your-secure-password -X GET "http://localhost:1024/spider/s3/spider-test-bucket?ConnectionName=aws-config01" \
  -H "Accept: application/json"

JSON Deleted Bucket Response:

{
  "Code": "NoSuchBucket",
  "Message": "Resource 'spider-test-bucket' does not exist in connection 'aws-config01'",
  "Resource": "/spider-test-bucket",
  "RequestId": "1774412579"
}

Note: This feature forcefully deletes the bucket and all its contents. This feature is unique to CB-Spider and not part of AWS S3 standard. The ?force query parameter is required as a safety mechanism.

Related Links

Add a custom footer

Table of contents

Clone this wiki locally