Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] [seatunnel] imap hdfs history-job-expire-minutes Invalid parameter #8386

Open
3 tasks done
fanxishu opened this issue Dec 26, 2024 · 1 comment
Open
3 tasks done

[Bug] [seatunnel] imap hdfs history-job-expire-minutes Invalid parameter #8386

fanxishu opened this issue Dec 26, 2024 · 1 comment
Labels
bug

Comments

@fanxishu
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

Imap The history-job-expire-minutes parameter in the IMAP configuration for HDFS persistence is not functioning properly. As the number of running jobs increases, the memory usage rises, ultimately leading to memory overflow.

SeaTunnel Version

2.3.8

SeaTunnel Config

seatunnel.yaml
seatunnel:
  engine:
    classloader-cache-mode: true
    history-job-expire-minutes: 1440
    backup-count: 1
    queue-type: blockingqueue
    print-execution-info-interval: 60
    print-job-metrics-info-interval: 60
    slot-service:
      dynamic-slot: false
      slot-num: 8
    checkpoint:
      interval: 300000
      timeout: 10000
      storage:
        type: hdfs
        max-retained: 3
        plugin-config:
          namespace: /tmp/asinainf/dataasync/checkpoint
          storage.type: hdfs
          fs.defaultFS: hdfs://10.11.11.19:9000
    telemetry:
      metric:
        enabled: false
map:
  engine*:
    map-store:
      enabled: true
      initial-mode: EAGER
      factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
      properties:
        type: hdfs
        namespace: /tmp/asiainf/dataasync/imap
        clusterName: seatunnel
        storage.type: hdfs
        fs.defaultFS: hdfs://10.11.11.19:9000


hazelcast.yaml
hazelcast:
  cluster-name: seatunnel
  network:
    rest-api:
      enabled: true
      endpoint-groups:
        CLUSTER_WRITE:
          enabled: true
        DATA:
          enabled: true
    join:
      tcp-ip:
        enabled: true
        member-list:
          - 10.11.11.19:5801
          - 10.12.83.88:5801
    port:
      auto-increment: true
      port: 5801
  map:
    engine*:
      map-store:
        enabled: true
        initial-mode: EAGER
        factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
        properties:
          type: hdfs
          namespace: /tmp/asiainf/dataasync/imap
          clusterName: seatunnel
          storage.type: hdfs
          fs.defaultFS: hdfs://10.11.11.19:9000
  properties:
    hazelcast.invocation.max.retry.count: 20
    hazelcast.tcp.join.port.try.count: 30
    hazelcast.logging.type: log4j2
    hazelcast.operation.generic.thread.count: 50
    hazelcast.heartbeat.failuredetector.type: phi-accrual
    hazelcast.heartbeat.interval.seconds: 2
    hazelcast.max.no.heartbeat.seconds: 180
    hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
    hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
    hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100

Running Command

0569429165604872  15976352660192_job  FAILED           2024-12-15 14:54:15.603  2024-12-15 14:56:30.729
919878581574696962  15955269670880_job  FAILED           2024-12-13 17:09:04.73   2024-12-13 17:09:48.09
919877656927797250  15955241454304_job  FAILED           2024-12-13 17:05:24.276  2024-12-13 17:06:38.813
919876389748867073  15955202781920_job  FAILED           2024-12-13 17:00:22.169  2024-12-13 17:00:36.894
919876383730040833  15955202546272_job  FAILED           2024-12-13 17:00:21.346  2024-12-13 17:00:36.478
919861655502848002  15954753130592_job  FAILED           2024-12-13 16:01:49.214  2024-12-13 16:02:38.001
919861588020690945  15954750837088_job  FAILED           2024-12-13 16:01:33.131  2024-12-13 16:01:46.554
919860338105516033  15954712834272_job  FAILED           2024-12-13 15:56:35.917  2024-12-13 15:56:49.226
919859761241915394  15954695319776_job  CANCELED         2024-12-13 15:54:21.778  2024-12-15 14:52:34.527
919859366633406465  15954683261280_job  FAILED           2024-12-13 15:52:43.507  2024-12-13 15:52:57.412
919859116359286785  15954675637472_job  FAILED           2024-12-13 15:51:43.93   2024-12-13 15:52:25.853
919788760994938882  15952528450656_job  FAILED           2024-12-13 11:12:09.805  2024-12-26 16:06:28.615
919783836689104897  15952378274272_job  FAILED           2024-12-13 11:07:48.926  2024-12-13 11:08:01.325
919786312528035842  15952453825376_job  FAILED           2024-12-13 11:02:26.103  2024-12-13 11:02:39.723
919782737332666370  15952344712544_job  FAILED           2024-12-13 10:48:13.646  2024-12-13 10:48:27.167
919782182703071233  15952327506144_job  FAILED           2024-12-13 10:46:01.47   2024-12-13 10:46:14.142
919781852535848961  15952317728992_job  FAILED           2024-12-13 10:44:42.682  2024-12-13 10:44:55.638
919780420550131714  15952274028768_job  DOING_SAVEPOINT  2024-12-13 10:39:01.264
919779366185992193  15952241852896_job  FAILED           2024-12-13 10:34:50.04   2024-12-13 10:35:02.175
919776731403911169  15952161448160_job  SAVEPOINT_DONE   2024-12-13 10:24:21.764  2024-12-13 10:35:17.772
919774749586882562  15952100943328_job  SAVEPOINT_DONE   2024-12-13 10:16:30.313  2024-12-13 10:20:29.282
919553839047114753  15945359302016_job  FAILED           2024-12-12 19:38:39.922  2024-12-12 19:38:51.905
919553615964667906  15945352500096_job  FAILED           2024-12-12 19:37:46.739  2024-12-12 19:37:59.258
919552356339679233  15945313951488_job  FAILED           2024-12-12 19:32:46.427  2024-12-12 19:33:29.841

Error Exception

0569429165604872  15976352660192_job  FAILED           2024-12-15 14:54:15.603  2024-12-15 14:56:30.729
919878581574696962  15955269670880_job  FAILED           2024-12-13 17:09:04.73   2024-12-13 17:09:48.09
919877656927797250  15955241454304_job  FAILED           2024-12-13 17:05:24.276  2024-12-13 17:06:38.813
919876389748867073  15955202781920_job  FAILED           2024-12-13 17:00:22.169  2024-12-13 17:00:36.894
919876383730040833  15955202546272_job  FAILED           2024-12-13 17:00:21.346  2024-12-13 17:00:36.478
919861655502848002  15954753130592_job  FAILED           2024-12-13 16:01:49.214  2024-12-13 16:02:38.001
919861588020690945  15954750837088_job  FAILED           2024-12-13 16:01:33.131  2024-12-13 16:01:46.554
919860338105516033  15954712834272_job  FAILED           2024-12-13 15:56:35.917  2024-12-13 15:56:49.226
919859761241915394  15954695319776_job  CANCELED         2024-12-13 15:54:21.778  2024-12-15 14:52:34.527
919859366633406465  15954683261280_job  FAILED           2024-12-13 15:52:43.507  2024-12-13 15:52:57.412
919859116359286785  15954675637472_job  FAILED           2024-12-13 15:51:43.93   2024-12-13 15:52:25.853
919788760994938882  15952528450656_job  FAILED           2024-12-13 11:12:09.805  2024-12-26 16:06:28.615
919783836689104897  15952378274272_job  FAILED           2024-12-13 11:07:48.926  2024-12-13 11:08:01.325
919786312528035842  15952453825376_job  FAILED           2024-12-13 11:02:26.103  2024-12-13 11:02:39.723
919782737332666370  15952344712544_job  FAILED           2024-12-13 10:48:13.646  2024-12-13 10:48:27.167
919782182703071233  15952327506144_job  FAILED           2024-12-13 10:46:01.47   2024-12-13 10:46:14.142
919781852535848961  15952317728992_job  FAILED           2024-12-13 10:44:42.682  2024-12-13 10:44:55.638
919780420550131714  15952274028768_job  DOING_SAVEPOINT  2024-12-13 10:39:01.264
919779366185992193  15952241852896_job  FAILED           2024-12-13 10:34:50.04   2024-12-13 10:35:02.175
919776731403911169  15952161448160_job  SAVEPOINT_DONE   2024-12-13 10:24:21.764  2024-12-13 10:35:17.772
919774749586882562  15952100943328_job  SAVEPOINT_DONE   2024-12-13 10:16:30.313  2024-12-13 10:20:29.282
919553839047114753  15945359302016_job  FAILED           2024-12-12 19:38:39.922  2024-12-12 19:38:51.905
919553615964667906  15945352500096_job  FAILED           2024-12-12 19:37:46.739  2024-12-12 19:37:59.258
919552356339679233  15945313951488_job  FAILED           2024-12-12 19:32:46.427  2024-12-12 19:33:29.841

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@fanxishu fanxishu added the bug label Dec 26, 2024
@wu-a-ge
Copy link
Contributor

wu-a-ge commented Jan 3, 2025

adjust history-job-expire-minutes ,reduce memory usage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wu-a-ge @fanxishu