Not working after updating pycall to 1.5.2

[yamapi-t]

I use snowflake connector through Pycall in Rails application.
Throwing queries asynchronously to Snowflake and then retrieving the status of the queries.

After updating pycall to 1.5.2, the screen froze when retrieving the status of the query.
However, looking at the logs, it appears that the retrieving of the query status completes successfully, and even the subsequent rendering of the view completes successfully.

Fixed Pycall version to 1.5.1 and did a bundle update and confirmed that it works correctly.
On the other hand, when I did a bundle update pycall, it didn't work properly.

No particular errors in the logs, even if they do not work properly.

What could be the cause?

Versions:

• ruby: 3.1.2
• rails: 7.0.8.1

[mscrivo]

we also saw hanging after updating to 1.5.2 using pycall in the context of ruby-spacy gem. We had to revert as well. Haven't had time to dig into what was happening.

These were all single threaded workers that just froze up and stopped processing anything, but without any errors or hints as to why.

[ankitmkt]

I had to revert as well and similarly, we haven't dug deeper into where it is failing within the pycall library. Reverting to 1.5.1 solved the problem for me.

Looking forward for the fix.

[kou]

Could you share a Ruby script that reproduces this problem?

[mscrivo]

in our case, it's enough to just:

require ruby-spacy

and then the whole process locks up entirely. Rather perplexing. I thought maybe updating our docker image to use python 3.13 over 3.11 would help, but nope, same problem. Only 1.5.1 works reliably.

[kou]

What is happen with require "ruby-spacy"?

Could you share Dockerfile for the problem require "ruby-spacy"?

[mscrivo]

What is happen with require "ruby-spacy"?

The process literally freezes, won't even accept input.

I'll try to get you a full repro, but essentially our Dockerfile has this:

FROM ruby:3.4.5-slim-trixie AS base

ENV VIRTUAL_ENV=/opt/venv \
    PATH="/opt/venv/bin:$PATH"

COPY requirements.txt .

RUN set -eux && \
    apt-get update && \
    # All but python3.13-venv are required to install dbt-postgres
    apt-get install -yqq --no-install-recommends python3-venv libpq-dev python3-dev build-essential && \
    python3 -m venv $VIRTUAL_ENV && \
    apt-get install -yqq --no-install-recommends libpq-dev python3-dev build-essential && \
    pip3 install --upgrade setuptools && \
    pip3 install -r requirements.txt && \
    python3 -m spacy download en_core_web_sm

and then we proceed to copy our app in, bundle install, etc.

requirements.txt has:

spacy==3.8.7

Then running any ruby code that requires ruby-spacy and attempts to use it completely hangs indefinitely.

[mscrivo]

This seems to be the problematic code. I created a fork without that and used it in our app and it no longer hangs.

That function assumes its being called from a thread that originated from the cpython interpreter.

[kou]

Thanks. But I couldn't reproduce it...:

FROM ruby:3.4.5-slim-trixie AS base

ENV VIRTUAL_ENV=/opt/venv \
    PATH="/opt/venv/bin:$PATH"

RUN set -eux && \
    apt-get update && \
    # All but python3.13-venv are required to install dbt-postgres
    apt-get install -yqq --no-install-recommends python3-venv libpq-dev python3-dev build-essential && \
    python3 -m venv $VIRTUAL_ENV && \
    apt-get install -yqq --no-install-recommends libpq-dev python3-dev build-essential && \
    pip3 install --upgrade setuptools && \
    pip3 install 'spacy==3.8.7' && \
    python3 -m spacy download en_core_web_sm && \
    gem install ruby-spacy && \
    ruby -e 'require "ruby-spacy"'

It's not frozen.

[mscrivo]

That's unfortunate. I really wish I could give you a reliable reproduction, but I don't really understand the confluence of things that causes it. I got tired of dealing with this issue and just replaced it all with a pure ruby implementation, so I'm not sure I will be of much help going forward 😞

[kou]

No problem. Thanks for sharing the information.

It's great that you can do what you want with a pure Ruby implementation. :-)

Others will provide a way to reproduce this problem.