[Data] Change log level for ignored exceptions (ray-project#44408)

bveeramani · web-flow · commit e8b0f8002ba6 · 2024-04-02T11:31:03.000-07:00
Ray Data gives you the option to ignore some number of errored blocks (for example, if you have some faulty data in your dataset). Currently, we log these exceptions as warnings. This PR updates the code to log the exceptions as errors.

Signed-off-by: Balaji Veeramani &lt;balaji@anyscale.com&gt;
diff --git a/python/ray/data/_internal/execution/streaming_executor_state.py b/python/ray/data/_internal/execution/streaming_executor_state.py
@@ -436,7 +436,7 @@ def process_completed_tasks(
                                 " Ignoring this exception with remaining"
                                 f" max_errored_blocks={remaining}."
                             )
-                            logger.get_logger().warning(error_message, exc_info=e)
+                            logger.get_logger().error(error_message, exc_info=e)
                         else:
                             error_message += (
                                 " Dataset execution will now abort."

Original file line number	Diff line number	Diff line change
`@@ -436,7 +436,7 @@ def process_completed_tasks(`
`436`	`436`	`" Ignoring this exception with remaining"`
`437`	`437`	`f" max_errored_blocks={remaining}."`
`438`	`438`	`)`
`439`		`- logger.get_logger().warning(error_message, exc_info=e)`
	`439`	`+ logger.get_logger().error(error_message, exc_info=e)`
`440`	`440`	`else:`
`441`	`441`	`error_message += (`
`442`	`442`	`" Dataset execution will now abort."`