Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve deprecation warning about multi-threaded use of fork #40

Open
AlanCoding opened this issue Jan 3, 2025 · 1 comment
Open

Resolve deprecation warning about multi-threaded use of fork #40

AlanCoding opened this issue Jan 3, 2025 · 1 comment
Labels
awx needed for AWX MVP minimum set of issues to fix

Comments

@AlanCoding
Copy link
Member

You can see the following warning in test output:

tests/integration/test_main.py: 42 warnings
  /usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=125488) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

This is related to a broader python issue, and the default method will actually change in python 3.14.

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Note The default start method will change away from fork in Python 3.14. Code that requires fork should explicitly specify that via get_context() or set_start_method().

Anecdotally, there are still a sense of fragility where deadlocks can happen for the subprocesses starting, stopping, and other things. Addressing this issue should go a very long way to establishing stability.

The outcome this issue seeks is that the deprecation warning is addressed.

@AlanCoding
Copy link
Member Author

Follow this for what it will change to python/cpython#84559

  • The default ProcessPoolExecutor start method (see
    :ref:multiprocessing-start-methods) changed from fork to forkserver on
    platforms other than macOS & Windows.

If we do go the route of forkserver, we would definitely need to use the .set_forkserver_preload() method, and even this leaves things a little unclear to me as to how we could load everything we really wanted. We also want to check the memory tradeoff involved.

@AlanCoding AlanCoding added awx needed for AWX MVP minimum set of issues to fix labels Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awx needed for AWX MVP minimum set of issues to fix
Projects
None yet
Development

No branches or pull requests

1 participant