Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add & improve concurrent tests #1820

Open
wants to merge 122 commits into
base: main
Choose a base branch
from

Conversation

jbelkins
Copy link
Contributor

@jbelkins jbelkins commented Nov 13, 2024

Description of changes

S3Concurrency tests:

  • Add a new high concurrency (200x), small payload (256 byte) S3 concurrency test
  • All S3Concurrency tests are now enabled on Apple platforms & URLSessionHTTPClient
  • Update the logic for dummy data to randomize data & create data of exact count
  • Added some clarifying comments

TranscribeStreaming tests:

  • Add a new concurrent (25x) TranscribeStreaming test (currently disabled on CRT HTTP client due to spurious connection-related failures)
  • Add custom retry logic to TranscribeStreaming tests to ensure throttling & service-imposed resource limits don't fail test.

New/existing dependencies impact assessment, if applicable

No new dependencies were added to this change.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jbelkins and others added 30 commits August 30, 2024 22:30
* Add concurrent tests

* Add missing 100x to a test's name

* Remove 100MiB file and generate file instead.

* See if using UUID for filename fixes this possibly-concurrency-issue.

* Try temporaryDirectory property.

* Fix concurrency issue

* Comment fixes & remove STS getCallerIdentity concurrent calls.

* Remove unused import

* Remove concurrent test for event stream output

* Refactor tests a bit

* Use FileStream to write to file

* Swap out filemanager with writing emtpy data approach

* Tweak a bit

* Simplify; no need to safe generated dummy data to a file, just use 50MB memory.

* Change from 10x50MB to 100x5MB

* Reduce payload from 5MB to 3MB

* Reduce payload from 3MB to 1MB.

* Add test for 1.5 MB to test aws chunked & flexible checksum flow also.

---------

Co-authored-by: Sichan Yoo <[email protected]>
Co-authored-by: Josh Elkins <[email protected]>
@jbelkins jbelkins changed the title Jbe/20241113 reenable concurrent tests chore: Add & improve concurrent tests Nov 13, 2024
@jbelkins jbelkins marked this pull request as ready for review November 13, 2024 23:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants