MapReduce Benchmark : object key does not exist

[jingren1021]

As my access to "storage-module-test" S3 bucket in MapReduce benchmark was denied, I modified the code to use custom S3 bucket.
After running locally, I came across error as below.

jingren@hp-worker:~/vSwarm$ kubectl logs -n default -c user-container -l serving.knative.dev/service=mapper --tail=-1
INFO:root:Using inline or s3 transfers
INFO:root:Mapper 6 is invoked
[key: "part-00006"
] 6
ERROR:grpc._server:Exception calling application: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Is there any initial object that should be inserted into custom S3 bucket before running MapReduce benchmark or is it some other error?

[ustiugov]

@jingren1021 It uses one of the datasets from AMPLab Big Data Bench: https://amplab.cs.berkeley.edu/benchmark . If I recall correctly, we used 1node dataset.

[ustiugov]

@jingren1021 please DM Shyam on our slack if you still need it.

[jingren1021]

I don't need the dataset anymore. Thanks for the help!

[shyamjesal]

Hi @jingren1021, we sadly lost the access the the dataset. Did you manage to download it?

[jingren1021]

I didn't manage to download the dataset.

[ustiugov]

@jingren1021 soon after we did manage to find a backup. let us know if you need it :)