Add Feature to Migrate Partial Table Data Based on Filter Criteria

[malah-akh]

I propose adding a feature to the Scylla Migrator that allows users to migrate only a portion of a table, based on specific filter criteria applied to the source table. This feature would enable more flexible and efficient data migrations, particularly useful in scenarios where only a subset of data needs to be transferred.

The feature would introduce the ability to specify filter criteria in the source settings configuration. This filter would then be applied when querying the source table, allowing only the data that matches the criteria to be migrated.

[julienrf]

Hello @ricardoborenstein. We do support a where property in the configuration when migrating from a CQL-compatible source:

scylla-migrator/config.yaml.example

Lines 44 to 45 in 95826a2

	# Optional condition to filter source table data that will be migrated
	# where: race_start_date = '2015-05-27' AND race_end_date = '2015-05-27'

Does that address your need? Do you need something similar for DynamoDB-compatible sources?