Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GrokProcessor to esmero-cantaloupe #30

Open
aksm opened this issue Oct 15, 2021 · 9 comments
Open

Add GrokProcessor to esmero-cantaloupe #30

aksm opened this issue Oct 15, 2021 · 9 comments
Labels
enhancement New feature or request esmero-cantaloupe IIIF goodness in docker package

Comments

@aksm
Copy link

aksm commented Oct 15, 2021

What?

GrokProcessor is a nice middle ground between OpenJPEG and Kakadu (see benchmarks). Unfortunately, despite the reported bugs being resolved, it's still being held in Debian sid (unstable), and compiling from source isn't possible because of compiler requirements. Precompiled binaries for 9.2.0 are only partially working with some images but showing the following errors for certain JP2 sources:

15:51:55.761 [qtp572868060-152] DEBUG e.i.l.c.r.ImageRepresentation - write(): Unsupported source format: JPEG2000
edu.illinois.library.cantaloupe.processor.SourceFormatException: Unsupported source format: JPEG2000
	at edu.illinois.library.cantaloupe.processor.GrokProcessor.process(GrokProcessor.java:406)
	at edu.illinois.library.cantaloupe.resource.ImageRepresentation.process(ImageRepresentation.java:205)
	at edu.illinois.library.cantaloupe.resource.ImageRepresentation.copyOrProcess(ImageRepresentation.java:167)
	at edu.illinois.library.cantaloupe.resource.ImageRepresentation.write(ImageRepresentation.java:130)
	at edu.illinois.library.cantaloupe.resource.ImageRequestHandler.handle(ImageRequestHandler.java:405)
	at edu.illinois.library.cantaloupe.resource.iiif.v2.ImageResource.doGET(ImageResource.java:128)
	at edu.illinois.library.cantaloupe.resource.HandlerServlet.handle(HandlerServlet.java:97)
	at edu.illinois.library.cantaloupe.resource.HandlerServlet.doGet(HandlerServlet.java:35)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
	at java.base/java.lang.Thread.run(Thread.java:832)

Some debugging/testing will be required. Maybe trying a later version? Compiling with different flags?

@aksm aksm added enhancement New feature or request esmero-cantaloupe IIIF goodness in docker package labels Oct 15, 2021
@aksm aksm self-assigned this Oct 15, 2021
@DiegoPino
Copy link
Member

@aksm could you share your full cmake command? Thanks!

@aksm
Copy link
Author

aksm commented Oct 18, 2021

@DiegoPino Sorry, forgot to mention that it was the release build that generated those errors. I'll try building it manually today.

@aksm
Copy link
Author

aksm commented Oct 19, 2021

@DiegoPino Ok, images seem to be loading now after building, but I still see the above in the logs. Thoughts? Should I open an issue with Cantaloupe?

@DiegoPino
Copy link
Member

@aksm that means it can not process JPEG2000 which is bad. There is probably a missing dependency in the make file.
Let me check

@DiegoPino
Copy link
Member

Ok, that was quick. Since I do not know how you are building it, but I do know that this here works, can you compare notes?

https://github.com/cantaloupe-project/cantaloupe/blob/develop/docker/Linux-JDK16/Dockerfile#L1-L67

Also, JDK-16... which one are we using??

There are other JDK examples there. But in all Grok passes compile... give it a try

@aksm
Copy link
Author

aksm commented Oct 19, 2021

@DiegoPino I'm just building locally with default flags and then copying the binaries into the container. I was trying to go for something from 9.2+ since the benchmarks are for 9.2 and since the buffer overflow bug seems to have been fixed from that point forward, but maybe 7.6.5 is not so much worse in performance? In any case, I'll give this a go and see.

We're on JDK-15.

@DiegoPino
Copy link
Member

@aksm
Copy link
Author

aksm commented Oct 22, 2021

Ok, I tried with a base image of Ubuntu 21.10 and the official apt package and ran into the same exact error so something else is going on. I'll try building from source with a newer version of grok since this base box should have all the dependencies.

@DiegoPino
Copy link
Member

Ok, it may be one of the dependencies then. Which is good because we do not need to change (once you figure out which one) the current base box

@aksm aksm removed their assignment Sep 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request esmero-cantaloupe IIIF goodness in docker package
Projects
None yet
Development

No branches or pull requests

2 participants