You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 31, 2018. It is now read-only.
Support for checkpointing was removed with the monotasks change: if a RDD is checkpointed, the resulting job will fail with an exception that looks like: java.lang.Error: org.apache.spark.SparkException: Missing parent partition information for partition 0 of dependency org.apache.spark.OneToOneDependency@643812a8 (should have been set in DAGScheduler)
This can be fixed by fixing the way that the parent partitions get serialized in Macrotask, similar to what's done in other kinds of dependencies where the RDD can get checkpointed.
Once is done, the CheckpointSuite of tests should be re-enabled.
The text was updated successfully, but these errors were encountered:
Support for checkpointing was removed with the monotasks change: if a RDD is checkpointed, the resulting job will fail with an exception that looks like: java.lang.Error: org.apache.spark.SparkException: Missing parent partition information for partition 0 of dependency org.apache.spark.OneToOneDependency@643812a8 (should have been set in DAGScheduler)
This can be fixed by fixing the way that the parent partitions get serialized in Macrotask, similar to what's done in other kinds of dependencies where the RDD can get checkpointed.
Once is done, the CheckpointSuite of tests should be re-enabled.
The text was updated successfully, but these errors were encountered: