Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[put-object] when set readlimit, error raised: java.io.IOException: No position has been marked #5335

Open
kisscelia opened this issue Jun 26, 2024 · 4 comments
Open

[put-object] when set readlimit, error raised: java.io.IOException: No position has been marked #5335

kisscelia opened this issue Jun 26, 2024 · 4 comments
Assignees
@debora-ito
Labels
bug This issue is a bug. p2 This is a standard priority issue

Comments

@kisscelia
Copy link

Describe the bug

put-object code:

PutObjectRequest putObjectRequest = PutObjectRequest.builder()
					.storageClass(storageClass)
					.bucket(container)
					.key(object)
					.build();
			
			// 2024.6.25, sine.
//			RequestBody rBody = RequestBody.fromInputStream(data, length); // not support resetting.
			
			// set read limit to 128KB, don't know whether useful or not.
			// warning: no QA!
			RequestBody rBody = RequestBody.fromContentProvider(
					ContentStreamProvider.fromInputStream(data), 
					length, Mimetype.MIMETYPE_OCTET_STREAM);
			
			client.putObject(putObjectRequest, rBody);

Expected Behavior

put object successfully.

Current Behavior

2024-06-26 15:51:05,148 [ERROR] [AbstractOperator] - worker 5 fail to perform operation thisistestbucket-01/preobject-9
java.io.UncheckedIOException: java.io.IOException: No position has been marked
	at software.amazon.awssdk.utils.FunctionalUtils.asRuntimeException(FunctionalUtils.java:180)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:126)
	at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:140)
	at software.amazon.awssdk.http.ContentStreamProvider$1.newStream(ContentStreamProvider.java:101)
	at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$getBody$3(BaseClientHandler.java:139)
	at software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor$ChecksumCalculatingStreamProvider.lambda$newStream$0(SyncChecksumValidationInterceptor.java:112)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4(FunctionalUtils.java:108)
	at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:136)
	at software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor$ChecksumCalculatingStreamProvider.newStream(SyncChecksumValidationInterceptor.java:112)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage$ClosingStreamProvider.newStream(StreamManagingStage.java:78)
	at software.amazon.awssdk.http.auth.aws.internal.signer.AwsChunkedV4PayloadSigner.sign(AwsChunkedV4PayloadSigner.java:85)
	at software.amazon.awssdk.http.auth.aws.internal.signer.DefaultAwsV4HttpSigner.doSign(DefaultAwsV4HttpSigner.java:273)
	at software.amazon.awssdk.http.auth.aws.internal.signer.DefaultAwsV4HttpSigner.sign(DefaultAwsV4HttpSigner.java:66)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.doSraSign(SigningStage.java:113)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.lambda$sraSignRequest$1(SigningStage.java:93)
	at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.sraSignRequest(SigningStage.java:92)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:79)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.SigningStage.execute(SigningStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage2.executeRequest(RetryableStage2.java:93)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage2.execute(RetryableStage2.java:56)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage2.execute(RetryableStage2.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
	at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:10227)
	at com.intel.cosbench.api.sio.SIOStorageV2.createObject(SIOStorageV2.java:298)
	at com.intel.cosbench.driver.operator.Writer.doWrite(Writer.java:97)
	at com.intel.cosbench.driver.operator.Preparer.operate(Preparer.java:98)
	at com.intel.cosbench.driver.operator.AbstractOperator.operate(AbstractOperator.java:75)
	at com.intel.cosbench.driver.agent.WorkAgent.performOperation(WorkAgent.java:197)
	at com.intel.cosbench.driver.agent.WorkAgent.doWork(WorkAgent.java:177)
	at com.intel.cosbench.driver.agent.WorkAgent.execute(WorkAgent.java:134)
	at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:44)
	at com.intel.cosbench.driver.agent.AbstractAgent.call(AbstractAgent.java:1)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: No position has been marked
	at org.apache.commons.io.input.NullInputStream.reset(NullInputStream.java:249)
	at org.apache.commons.io.input.ProxyInputStream.reset(ProxyInputStream.java:118)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:124)
	... 65 more

Reproduction Steps

image

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.26.8

JDK version used

1.8.0_301

Operating System and version

CentOS 7.9
@kisscelia kisscelia added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jun 26, 2024
@kisscelia
Copy link
Author

same logic,
multipart upload is OK, can upload successfully.

	@Override
	public void createMultipartObject(String container, String object, InputStream data, long length, Config config) {
		super.createMultipartObject(container, object, data, length, config);

		CreateMultipartUploadRequest mRequest = CreateMultipartUploadRequest.builder()
				.storageClass(storageClass)
				.bucket(container)
				.key(object)
				.build();

		// Create a list of ETag objects. You retrieve ETags for each object part uploaded,
		// then, after each individual part has been uploaded, pass the list of ETags to
		// the request to complete the upload.
		List<CompletedPart> partETags = new ArrayList<CompletedPart>();

		try {
			CreateMultipartUploadResponse mResponse = client.createMultipartUpload(mRequest);
			String uploadId = mResponse.uploadId();

			long position = 0;
			long tempPartSize; // avoid to change the partSize, bug fix:#25

			for (int i = 1; position < length; i++) {
				// Because the last part could be less than 5 MiB, adjust the part size as
				// needed.
				tempPartSize = Math.min(partSize, (length - position));

				// Create the request to upload a part.
				UploadPartRequest uRequest = UploadPartRequest.builder()
						.bucket(container)
						.key(object)
						.uploadId(uploadId)
						.partNumber(i)
						.build();
				
				// 2024.6.25, sine.
//				RequestBody rBody = RequestBody.fromInputStream(data, tempPartSize); // not support resetting.
				
				// set read limit to 128KB, don't know whether useful or not.
				// warning: no QA!
				RequestBody rBody = RequestBody.fromContentProvider(
						ContentStreamProvider.fromInputStream(data), 
						tempPartSize, Mimetype.MIMETYPE_OCTET_STREAM);
				
				// Upload the part and add the response's ETag to our list.
				UploadPartResponse uResponse = client.uploadPart(uRequest, rBody);

				CompletedPart tempPart = CompletedPart.builder()
						.partNumber(i)
						.eTag(uResponse.eTag())
						.build();
				partETags.add(tempPart);

				position += tempPartSize;
			}

			// Complete the multipart upload.
			CompleteMultipartUploadRequest compRequest = CompleteMultipartUploadRequest.builder()
					.bucket(container)
					.key(object)
					.uploadId(uploadId)
					.multipartUpload(CompletedMultipartUpload.builder().parts(partETags).build())
					.build();
			
			client.completeMultipartUpload(compRequest);

		} catch (S3Exception se) {
			logger.debug("Service exception thrown.");
			throw new StorageException(se);
		} catch (SdkClientException sce) {
			logger.debug("Client exception thrown.");
			throw new StorageTimeoutException(sce);
		}
	}

@kisscelia
Copy link
Author

image
setReadLimit to 128KB

can we set it as we want?

e.g: use this method.
image

@debora-ito
Copy link
Member

@kisscelia I'm not quite clear on the issue you're reporting.

The error java.io.IOException: No position has been marked only happens with PutObjectRequest and when RequestBody.fromContentProvider is used, but not with uploadPart?

And have you tried changing the maxReadLimit?

@debora-ito debora-ito added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Aug 8, 2024
@debora-ito debora-ito self-assigned this Aug 8, 2024
@kisscelia
Copy link
Author

kisscelia commented Aug 8, 2024 •
edited
Loading

@kisscelia I'm not quite clear on the issue you're reporting.

The error java.io.IOException: No position has been marked only happens with PutObjectRequest and when RequestBody.fromContentProvider is used, but not with uploadPart?

And have you tried changing the maxReadLimit?

@debora-ito thanks for your reply.

I don't know how to change the maxReadLimit, can you tell me how to do that?

thank you.

BTW, because of the Best Practices of v1, so i set the readlimit in v2, needed or not?

😸

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@debora-ito debora-ito

Labels
bug This issue is a bug. p2 This is a standard priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@debora-ito @kisscelia