S3 IO

[[TOC]]

Setup client node

/opt/seagate/eos-prvsnr/cli/setup-provisioner --singlenode

Install s3cmd client

salt-call state.apply components.s3clients

Add ClusterIP address to /etc/hosts

E.g.

172.16.8.1  s3.seagate.com sts.seagate.com iam.seagate.com   sts.cloud.seagate.com

Create an S3 account

E.g.

[root@smc21-m10 eos-prvsnr]# s3iamcli createaccount -n SanityAccountToDeleteAfterUse \
    -e [email protected] \
    --ldapuser sgiamadmin --ldappasswd ldapadmin
AccountId = 828634098320, CanonicalId = c0d3ee09f15e4c63ba60a119e3aea27acb86e2d59d5b4fc09cbb21d3ca1dd2f5, RootUserName = root, AccessKeyId = TfyaGuDsT6-yICzzmlXoGw, SecretKey = 0+yyCrFRDt6NBQ1lQg9rvhuuLy1vOwyqdg35ifRW

Create an S3 bucket

E.g.

[root@smc21-m10 eos-prvsnr]# s3cmd --access_key=TfyaGuDsT6-yICzzmlXoGw \
    --secret_key=0+yyCrFRDt6NBQ1lQg9rvhuuLy1vOwyqdg35ifRW mb s3://sanitybucket
Bucket 's3://sanitybucket/' created
[root@smc21-m10 eos-prvsnr]# s3cmd --access_key=TfyaGuDsT6-yICzzmlXoGw --secret_key=0+yyCrFRDt6NBQ1lQg9rvhuuLy1vOwyqdg35ifRW ls
2020-03-31 03:28  s3://sanitybucket

Perform I/O

[root@smc21-m10 hare]# s3cmd --access_key=TfyaGuDsT6-yICzzmlXoGw \
    --secret_key=0+yyCrFRDt6NBQ1lQg9rvhuuLy1vOwyqdg35ifRW \
    put /tmp/128M s3://sanitybucket/SanityObjectToDeleteAfterUse
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 1 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    26.58 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 2 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    63.74 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 3 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    58.22 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 4 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    63.42 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 5 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    56.33 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 6 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    1s    12.12 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 7 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    60.20 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 8 of 9, 15MB] [1 of 1]
 15728640 of 15728640   100% in    0s    61.91 MB/s  done
upload: '/tmp/128M' -> 's3://sanitybucket/SanityObjectToDeleteAfterUse'  [part 9 of 9, 2MB] [1 of 1]
 2170880 of 2170880   100% in    0s    27.58 MB/s  done

See also

test-boot1 CI job performs S3 I/O. The implementation: ci/m0vg/test-boot1-s3