[Bug] [excute jar] After modifying and re-uploading the JAR file, the task submission failed

[jiangwwwei]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

After re-uploading the modified JAR file (with the same name), task submission fails with the following error message:

2024-03-08 02:20:59 UTC ERROR org.dinky.aop.exception.UnKnownExceptionHandler 39 unknownException - java.lang.RuntimeException: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: ZipFile invalid LOC header (bad signature) org.dinky.data.exception.DinkyException: java.lang.RuntimeException: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: ZipFile invalid LOC header (bad signature)
	at org.dinky.aop.UdfClassLoaderAspect.round(UdfClassLoaderAspect.java:58) ~[dinky-admin-1.0.0-rc4.jar:?]
	at jdk.internal.reflect.GeneratedMethodAccessor532.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) ~[spring-aop-5.3.27.jar:5.3.27]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.27.jar:5.3.27]
....

More detailed error log:

Caused by: java.lang.RuntimeException: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: ZipFile invalid LOC header (bad signature)
	at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:89) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:65) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.trans.dml.ExecuteJarOperation.explain(ExecuteJarOperation.java:99) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.job.builder.JobJarStreamGraphBuilder.getJarStreamGraph(JobJarStreamGraphBuilder.java:82) ~[dinky-core-1.0.0-rc4.jar:?]
	at org.dinky.job.JobManager.executeJarSql(JobManager.java:259) ~[dinky-core-1.0.0-rc4.jar:?]
	at org.dinky.service.task.FlinkJarSqlTask.execute(FlinkJarSqlTask.java:46) ~[dinky-admin-1.0.0-rc4.jar:?]
	at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:189) ~[dinky-admin-1.0.0-rc4.jar:?]
...


Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: ZipFile invalid LOC header (bad signature)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist-1.16.0.jar:1.16.0]
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist-1.16.0.jar:1.16.0]
	at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:158) ~[flink-dist-1.16.0.jar:1.16.0]
	at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:85) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.trans.dml.ExecuteJarOperation.getStreamGraph(ExecuteJarOperation.java:65) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.trans.dml.ExecuteJarOperation.explain(ExecuteJarOperation.java:99) ~[dinky-client-base-1.0.0-rc4.jar:?]
	at org.dinky.job.builder.JobJarStreamGraphBuilder.getJarStreamGraph(JobJarStreamGraphBuilder.java:82) ~[dinky-core-1.0.0-rc4.jar:?]
...

Caused by: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature)
	at java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:1018) ~[?:?]
	at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:1028) ~[?:?]
	at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:478) ~[?:?]
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159) ~[?:?]
	at java.io.FilterInputStream.read(FilterInputStream.java:133) ~[?:?]
	at java.io.FilterInputStream.read(FilterInputStream.java:107) ~[?:?]
	at java.util.Properties$LineReader.readLine(Properties.java:502) ~[?:?]
	at java.util.Properties.load0(Properties.java:418) ~[?:?]
	at java.util.Properties.load(Properties.java:407) ~[?:?]
	at org.apache.flink.api.java.utils.ParameterTool.fromPropertiesFile(ParameterTool.java:137) ~[flink-dist-1.16.0.jar:1.16.0]
	at com.szfiu.dw.realtime.common.app.BaseApp.createParameterTool(BaseApp.java:303) ~[?:?]

I can see that the original error was caused by my code accessing properties resources using ParameterTool.

However, after restarting Dinky or modifying the JAR file name, I was able to successfully submit and run the task.

What you expected to happen

The JAR task executed successfully.

How to reproduce

dinky version : 1.0 rc4
resource: oss (aliyun)
task type : jar

step:

• The code includes ParameterTool for reading .properties files from the resources.
• The JAR task was executed first and ran successfully.
• Then, I stopped the task, modified the code, recompiled, packaged, and uploaded the JAR file.(The name of the JAR remains the same.)
• Running the JAR task again, an error occurred.

A temporary solution to the problem:

• Restarting Dinky. ( ./auto.sh restart )

Another temporary solution to the problem:

• Changing the name of the JAR file.

Anything else

It seems that some variables were cached during the execution of the JAR task and were not released.

Version

1.0.0

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Hello @jiangwwwei, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @jiangwwwei, 这个 issue 是关于 CDC/CDCSOURCE 的，所以我把它分配给了 @aiwenmo。如有任何问题，可以评论回复。

[jiangwwwei]

I upgraded to version 1.0 stable, and this bug hasn't reappeared.