You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We need the key naming fn to randomize the key prefixes to ensure proper s3 sharding. This could be done by generating a timestamp and reversing the end of the timestamp to ensure randomness.
The input plugin would then need to be modified to make this more pleasing to use, and ensure order is maintained.
This would break prefixing, as it depends on keys sharing a prefix. This could be worked around by listing all keys and filtering from onyx, rather than having amazon's servers pre filter. If we go this route we should have a metric for the time it takes to filter the keys.
The text was updated successfully, but these errors were encountered:
We need the key naming fn to randomize the key prefixes to ensure proper s3 sharding. This could be done by generating a timestamp and reversing the end of the timestamp to ensure randomness.
The input plugin would then need to be modified to make this more pleasing to use, and ensure order is maintained.
This would break prefixing, as it depends on keys sharing a prefix. This could be worked around by listing all keys and filtering from onyx, rather than having amazon's servers pre filter. If we go this route we should have a metric for the time it takes to filter the keys.
The text was updated successfully, but these errors were encountered: