Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINIFICPP-2127 Add multipart upload support for PutS3Object processor #1586

Closed
wants to merge 32 commits into from

Conversation

lordgamez
Copy link
Contributor

@lordgamez lordgamez commented Jun 7, 2023

https://issues.apache.org/jira/browse/MINIFICPP-2127
https://issues.apache.org/jira/browse/MINIFICPP-2140


Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

@lordgamez lordgamez marked this pull request as ready for review June 8, 2023 13:56
@lordgamez lordgamez force-pushed the MINIFICPP-2127 branch 3 times, most recently from b199c4d to 32c1a98 Compare June 14, 2023 08:16
extensions/aws/tests/MultipartUploadStateStorageTest.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.h Outdated Show resolved Hide resolved
docker/test/integration/cluster/checkers/AwsChecker.py Outdated Show resolved Hide resolved
docker/test/integration/features/steps/steps.py Outdated Show resolved Hide resolved
extensions/aws/s3/MultipartUploadStateStorage.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/MultipartUploadStateStorage.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/MultipartUploadStateStorage.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
docker/test/integration/cluster/checkers/AwsChecker.py Outdated Show resolved Hide resolved
extensions/aws/s3/MultipartUploadStateStorage.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/tests/PutS3ObjectTests.cpp Show resolved Hide resolved
@lordgamez
Copy link
Contributor Author

lordgamez commented Jul 7, 2023

@szaszm Changed the state manager to be able to use it for multipart upload states in b91b756 It seems to be enough to return true for commit and rollback calls if no transaction is in progress in libminifi/src/controllers/keyvalue/KeyValueStateManager.cpp, so it wouldn't return any error in the ProcessSession in case all transactions were commited previously.

extensions/aws/processors/PutS3Object.h Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
libminifi/include/utils/TimeUtil.h Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.h Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Show resolved Hide resolved
@lordgamez lordgamez force-pushed the MINIFICPP-2127 branch 2 times, most recently from 9e7a2d8 to 4a74f69 Compare July 20, 2023 07:39
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/processors/PutS3Object.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.cpp Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.h Outdated Show resolved Hide resolved
extensions/aws/s3/S3Wrapper.h Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants