Skip to content

Offline Backfill requires backfillStartDate, which can be defaulted to end_date #939

@krisnaru

Description

@krisnaru

25/03/11 18:59:44 INFO o.ap.spark.SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.AssertionError: assertion failed: GroupBy:xxx has null backfillStartDate. This needs to be set for offline backfilling.
at scala.Predef$.assert(Predef.scala:223)
at ai.chronon.spark.GroupBy$.computeBackfill(GroupBy.scala:681)
at ai.chronon.spark.Driver$GroupByBackfill$.run(Driver.scala:423)
at ai.chronon.spark.Driver$.main(Driver.scala:1024)
at ai.chronon.spark.Driver.main(Driver.scala)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1029)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:194)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:217)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:91)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1120)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1129)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions