[C++] Should ThreadPool executor SpawnReal handle exception?

[mapleFU]

Describe the enhancement requested

apache arrow uses arrow::Status to handling error. This works well. However, when using ::arrow::ThreadPool, the internal task will not catch the exception when internal task throw exception, causing the coredump here.

There're many ways for solving this problem:

1. Make sure the caller doesn't throw any exceptions, and do nothing in arrow libraries
2. Capture the exception in arrow ThreadPool

Component(s)

C++

[pitrou]

Why did you close this as completed @mapleFU ?

[pitrou]

Sorry, my question was more: why close this? Is it not actually a bug?

[mapleFU]

Is it not actually a bug?

I'm thinking that:

1. This might a problem here if internal user throws an exception
2. However, when I fix this in my case, I found that at least I need handling the error in internal

So I mark this as not planned now

[pitrou]

But Apache Arrow is not your personal software, if there is a bug, the issue should remain open.

[pitrou]

Also, I do think we should indeed handle unexpected exceptions even though we generally expect Arrow routines to be exception-free.

[mapleFU]

But Apache Arrow is not your personal software, if there is a bug, the issue should remain open.

Yeah I understand, however when I'm adding this I found handling exception would be much more than "swallow exception in threadpool", at least, the caller ( like BackgroundGenerator ) might deadlocked after exception is thrown but not handled.

[pitrou]

I see. That said, I think we can still improve things in ThreadPool somehow.