Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi-threading causes hchebyshevcenter to trigger a segmentation fault #211

Open
ferrolho opened this issue Jun 1, 2020 · 4 comments
Open

Multi-threading causes hchebyshevcenter to trigger a segmentation fault #211

ferrolho opened this issue Jun 1, 2020 · 4 comments

Comments

@ferrolho
Copy link
Contributor

ferrolho commented Jun 1, 2020

I have been trying this with four threads:

export JULIA_NUM_THREADS=4

Minimal working example:

using GLPK
using JuMP
using Polyhedra
using QHull

@show Threads.nthreads()

thread_libs = [QHull.Library(GLPK.Optimizer) for i = 1:Threads.nthreads()]
thread_lp_solvers = [optimizer_with_attributes(GLPK.Optimizer, "presolve" => GLPK.ON) for i = 1:Threads.nthreads()]

Threads.@threads for i = 1:10
    threadᵢ = Threads.threadid()
    println("I am thread #$(threadᵢ).")

    lib = thread_libs[threadᵢ]
    lp_solver = thread_lp_solvers[threadᵢ]

    my_points = rand(10, 3)
    my_polyhedron = polyhedron(vrep(my_points), lib)
    hchebyshevcenter(my_polyhedron, lp_solver)
end

I am not even sure I need a distinct lib and lp_solver per thread, but I tried it anyway to check whether that was the cause of the problem...

Output log:

signal (11): Segmentation fault

signal (11): Segmentation fault
in expression starting at REPL[12]:1
in expression starting at REPL[12]:1
unknown function (ip: 0x7f3a890d7936)
PyUnicode_New at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyUnicodeWriter_PrepareInternal at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_DecodeUTF8Stateful at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_FromFormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyErr_FormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyErr_Format at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyObject_GetBuffer at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyObject_Malloc at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_New at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyUnicodeWriter_PrepareInternal at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_DecodeUTF8Stateful at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_FromFormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
sbuftype! at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:134 [inlined]
PyErr_FormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
isbuftype at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:148 [inlined]
pysequence_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:756
PyErr_Format at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyObject_GetBuffer at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
isbuftype! at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:134 [inlined]
isbuftype at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:148 [inlined]
pysequence_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:756
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:770
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:803 [inlined]
convert at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:828
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:770
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:803 [inlined]
convert at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:828
getproperty at /home/henrique/.julia/packages/PyCall/zqDXB/src/PyCall.jl:306 [inlined]
chull at /home/henrique/.julia/packages/QHull/ck7jQ/src/QHull.jl:34
getproperty at /home/henrique/.julia/packages/PyCall/zqDXB/src/PyCall.jl:306 [inlined]
chull at /home/henrique/.julia/packages/QHull/ck7jQ/src/QHull.jl:34
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:83
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:83
jl_apply_generic at /opt/julia-1.4.1/bin/../lib/libjulia.so.1 (unknown line)
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:62
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:61
getine at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:167
hrep at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:211
hyperplanetype at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/iterators.jl:175
_broadcast_getindex_evalf at ./broadcast.jl:631 [inlined]
_broadcast_getindex at ./broadcast.jl:604 [inlined]
#19 at ./broadcast.jl:1024 [inlined]
ntuple at ./ntuple.jl:41 [inlined]
copy at ./broadcast.jl:1024 [inlined]
materialize at ./broadcast.jl:820
hyperplanes at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/iterators.jl:183 [inlined]
hchebyshevcenter at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/center.jl:13
_jl_invoke at (null):0 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
macro expansion at ./REPL[12]:10 [inlined]
#12#threadsfor_fun at ./threadingconstructs.jl:61
#12#threadsfor_fun at ./threadingconstructs.jl:28
unknown function (ip: 0x7f3ab411c8ec)
_jl_invoke at (null):0 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:62
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:61
getine at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:167
hrep at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:211
hyperplanetype at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/iterators.jl:175
_broadcast_getindex_evalf at ./broadcast.jl:631 [inlined]
_broadcast_getindex at ./broadcast.jl:604 [inlined]
#19 at ./broadcast.jl:1024 [inlined]
ntuple at ./ntuple.jl:41 [inlined]
copy at ./broadcast.jl:1024 [inlined]
materialize at ./broadcast.jl:820
hyperplanes at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/iterators.jl:183 [inlined]
hchebyshevcenter at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/center.jl:13
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
macro expansion at ./REPL[12]:10 [inlined]
#12#threadsfor_fun at ./threadingconstructs.jl:61
#12#threadsfor_fun at ./threadingconstructs.jl:28
unknown function (ip: 0x7f3ab411c8ec)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322

signal (11): Segmentation fault
in expression starting at REPL[12]:1
PyErr_Restore at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyDict_GetItem at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
unknown function (ip: 0x7f3adb8c8a5e)
unknown function (ip: (nil))
Allocations: 132736478 (Pool: 132718107; Big: 18371); GC: 137
[1]    27961 segmentation fault (core dumped)  julia
@blegat
Copy link
Member

blegat commented Jun 1, 2020

The hchebyshevcenter needs the H-representation so it triggers computehrep! which calls the QHull library which seems to be what is segfaulting. Could you try replacing hchebyshevcenter(my_polyhedron, lp_solver) by computehrep!(my_polyhedron) and check whether it still fails ?

@ferrolho
Copy link
Contributor Author

ferrolho commented Jun 1, 2020

Hi, @blegat! For the MWE above, I have replaced the line

hchebyshevcenter(my_polyhedron, lp_solver)

with

Polyhedra.computehrep!(my_polyhedron)

The resulting output is below:

I am thread #2.
I am thread #3.
I am thread #4.
I am thread #1.
ERROR: TaskFailedException:
MethodError: no method matching computehrep!(::QHull.Polyhedron)
Closest candidates are:
  computehrep!(::DefaultPolyhedron) at /home/henrique/.julia/packages/Polyhedra/w0Toa/src/defaultlibrary.jl:84
Stacktrace:
 [1] macro expansion at ./REPL[11]:11 [inlined]
 [2] (::var"#3#threadsfor_fun#7"{UnitRange{Int64}})(::Bool) at ./threadingconstructs.jl:61
 [3] (::var"#3#threadsfor_fun#7"{UnitRange{Int64}})() at ./threadingconstructs.jl:28
Stacktrace:
 [1] wait(::Task) at ./task.jl:267
 [2] top-level scope at ./threadingconstructs.jl:69

I am testing this using Polyhedra v0.6.3.

@blegat blegat mentioned this issue Jun 1, 2020
Open
@blegat
Copy link
Member

blegat commented Jun 1, 2020

Arf, this is an issue with QHull.jl (opened in JuliaPolyhedra/QHull.jl#33), replace it with hrep(my_polyhedron) then

@ferrolho
Copy link
Contributor Author

ferrolho commented Jun 1, 2020

Thanks for opening the issue on the right place. I replaced Polyhedra.computehrep!(my_polyhedron) with hrep(my_polyhedron) and it went back to segfaulting; output follows below:

I am thread #2.
I am thread #4.
I am thread #3.
I am thread #1.

signal (11): Segmentation fault

signal (11): Segmentation fault
in expression starting at REPL[10]:1
in expression starting at REPL[10]:1
_PyObject_Malloc at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyObject_Malloc at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_New at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyUnicodeWriter_PrepareInternal at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_DecodeUTF8Stateful at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyObject_GetAttrString at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_New at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
_PyUnicodeWriter_PrepareInternal at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_DecodeUTF8Stateful at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyUnicode_FromFormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyErr_FormatV at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyErr_Format at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
PyObject_GetBuffer at /home/henrique/miniconda3/lib/libpython3.7m.so.1.0 (unknown line)
isbuftype! at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:134 [inlined]
isbuftype at /home/henrique/.julia/packages/PyCall/zqDXB/src/pybuffer.jl:148 [inlined]
pysequence_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:756
getproperty at /home/henrique/.julia/packages/PyCall/zqDXB/src/PyCall.jl:298
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:770
pytype_query at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:803 [inlined]
convert at /home/henrique/.julia/packages/PyCall/zqDXB/src/conversions.jl:828
getproperty at /home/henrique/.julia/packages/PyCall/zqDXB/src/PyCall.jl:306 [inlined]
chull at /home/henrique/.julia/packages/QHull/ck7jQ/src/QHull.jl:34
getproperty at /home/henrique/.julia/packages/PyCall/zqDXB/src/PyCall.jl:306 [inlined]
chull at /home/henrique/.julia/packages/QHull/ck7jQ/src/QHull.jl:34
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:83
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:83
jl_apply_generic at /opt/julia-1.4.1/bin/../lib/libjulia.so.1 (unknown line)
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:62
qhull at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:61
getine at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:167
hrep at /home/henrique/.julia/packages/QHull/ck7jQ/src/polyhedron.jl:211
jl_apply_generic at /opt/julia-1.4.1/bin/../lib/libjulia.so.1 (unknown line)
macro expansion at ./REPL[10]:11 [inlined]
#3#threadsfor_fun at ./threadingconstructs.jl:61
#3#threadsfor_fun at ./threadingconstructs.jl:28
unknown function (ip: 0x7f375a84d42c)
[1]    2167 segmentation fault (core dumped)  julia

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@blegat @ferrolho