Support Java 17 as runtime for Scala/Java based micro services

[bobbai00]

Motivation

Currently, Scala/Java based micro services run in Java 11.

Compared to Java 11, Java 17 is better at both language level and runtime & GC performance.

Besides that, modern Java libraries usually require Java 17+, including some GenAI component like mcp-java-sdk. If later we want to use these libraries, our existing Java/Scala code has to support Java 17.

How hard it is to support Java 17

It turns out that it is very easy to support Java 17. During some testing, the only incompatibility I encountered is org.ehcache.sizeOf we used in common/workflow-core/src/main/scala/org/apache/amber/core/tuple/Tuple.scala:

case class Tuple @JsonCreator() (
    @JsonProperty(value = "schema", required = true) schema: Schema,
    @JsonProperty(value = "fields", required = true) fieldVals: Array[Any]
) extends SeqTupleLike
    with Serializable {

  checkNotNull(schema)
  checkNotNull(fieldVals)
  checkSchemaMatchesFields(schema.getAttributes, fieldVals)

// This line causes the incompatibility
  override val inMemSize: Long = SizeOf.newInstance().deepSizeOf(this)

The incompatibility is because: Ehcache manages byte based cache limits by utilizing reflection, which is forbidden with Java 17. The error message is the following:

[2025-10-23 23:07:52,727] [WARN] [org.ehcache.sizeof.ObjectGraphWalker] [DP-thread] - The JVM is preventing Ehcache from accessing the subgraph beneath 'private final byte[] java.lang.String.value' - cache sizes may be underestimated as a result 
java.lang.reflect.InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not "opens java.lang" to unnamed module @5af5def9
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at org.ehcache.sizeof.ObjectGraphWalker.getAllFields(ObjectGraphWalker.java:245)
	at org.ehcache.sizeof.ObjectGraphWalker.getFilteredFields(ObjectGraphWalker.java:204)
	at org.ehcache.sizeof.ObjectGraphWalker.walk(ObjectGraphWalker.java:159)
	at org.ehcache.sizeof.SizeOf.deepSizeOf(SizeOf.java:74)
	at org.apache.amber.core.tuple.Tuple.<init>(Tuple.scala:55)
	at org.apache.amber.core.tuple.Tuple$Builder.build(Tuple.scala:236)
	at org.apache.amber.core.tuple.SeqTupleLike.enforceSchema(TupleLike.scala:64)
	at org.apache.amber.core.tuple.SeqTupleLike.enforceSchema$(TupleLike.scala:57)
	at org.apache.amber.core.tuple.TupleLike$$anon$6.enforceSchema(TupleLike.scala:141)
	at org.apache.amber.engine.architecture.worker.DataProcessor.outputOneTuple(DataProcessor.scala:184)
	at org.apache.amber.engine.architecture.worker.DataProcessor.continueDataProcessing(DataProcessor.scala:196)
	at org.apache.amber.engine.architecture.worker.DPThread.$anonfun$runDPThreadMainLogic$2(DPThread.scala:194)
	at org.apache.amber.engine.architecture.logreplay.ReplayLogManager.withFaultTolerant(ReplayLogManager.scala:77)
	at org.apache.amber.engine.architecture.logreplay.ReplayLogManager.withFaultTolerant$(ReplayLogManager.scala:71)
	at org.apache.amber.engine.architecture.logreplay.EmptyReplayLogManagerImpl.withFaultTolerant(ReplayLogManager.scala:87)
	at org.apache.amber.engine.architecture.worker.DPThread.org$apache$amber$engine$architecture$worker$DPThread$$runDPThreadMainLogic(DPThread.scala:192)
	at org.apache.amber.engine.architecture.worker.DPThread$$anon$1.run(DPThread.scala:95)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

To solve it, one simple solution is to set the JAVA_OPTS env variable as:

--add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED

Summary

• We should support Java 17 as Java/Scala based micro services' runtime
• It is very easy to make these services compatible with Java 17. Only minor change is needed

[chenlica]

Thanks for presenting this interesting idea. One question: can we replace org.ehcache with another library that works better with JAVA 17 and doesn't require this JAVA_OPTS env variable?

[aglinxinyuan]

I agree with Chen. I like Java 17, but I don't want to have the extra JAVA_OPTS.

[Yicong-Huang]

I recall that we can use put those configs in a file?

[bobbai00]

I am exploring the alternatives of Ehcache. And I also saw that Ehcache's maintainers were aware of this issue already: ehcache/ehcache3#3008.

[bobbai00]

Another compatibility issue: after switching to Java 17, the following issue will come out because of Arrow:

java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with --add-opens=java.base/java.nio=ALL-UNNAMED? (See https://arrow.apache.org/docs/java/install.html) at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:143) at org.apache.arrow.memory.ArrowBuf.setZero(ArrowBuf.java:1175) at org.apache.arrow.vector.BaseVariableWidthVector.initOffsetBuffer(BaseVariableWidthVector.java:243) at org.apache.arrow.vector.BaseVariableWidthVector.allocateBytes(BaseVariableWidthVector.java:469) at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:420) at org.apache.arrow.vector.BaseVariableWidthVector.allocateNew(BaseVariableWidthVector.java:380) at org.apache.arrow.vector.VectorSchemaRoot.allocateNew(VectorSchemaRoot.java:147) at org.apache.amber.engine.architecture.pythonworker.PythonProxyClient.sendECM(PythonProxyClient.scala:144) at org.apache.amber.engine.architecture.pythonworker.PythonProxyClient.mainLoop(PythonProxyClient.scala:116) at org.apache.amber.engine.architecture.pythonworker.PythonProxyClient.run(PythonProxyClient.scala:79) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)

The solution is: set the JDK_JAVA_OPTIONS to:

--add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED

This solution is explicitly required in Arrow's official document: https://arrow.apache.org/docs/java/install.html. @chenlica and @aglinxinyuan are you guys fine with this one?

[bobbai00]

Setting JDK_JAVA_OPTIONS also solves the Ehcache library compatibility issue.

[chenlica]

A general question is whether it's a good practice to use JDK_JAVA_OPTIONS to "solve" these problems. I want to know other members' opinions.

[Yicong-Huang]

I think it is definitely a good direction to support jdk 17. I don't see problems of using JDK_JAVA_OPTIONS to solve the incompatible issues during transition. But I also wonder if we can get rid of those incompatible code for the long run.

[chenlica]

If we all think it's a good idea to use Java 17, we can adopt this solution, and create an issue to remove the incompatible in the future. If those packages improve themselves to be compatible with Java 17, then this issue will be solved automatically.

[pjfanning]

Lots of your dependencies dropped Java 8 and 11 support years ago, eg jooq.
This force you to use old maintained dependencies.

[chenlica]

Thanks, @pjfanning .

@bobbai00 Can you suggest a plan?