You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
AFAICT, there is no direct way to access such information.
There is a class AbstractReadManager which tracks the number of read items, but it is used internally only by emr-dynamodb-connector to adjust the read throughput, and this information is not publicly exposed. We would need to investigate further whether this class effectively tracks the total number of read items (and not just the amount of read items per partition). If that’s the case, a basic way to expose that information would be through logs. Otherwise, we would have to use a Spark accumulator to keep track of all the read items across all the partitions.
It would be useful to have the following progress information during migrations:
The text was updated successfully, but these errors were encountered: