torchdeploy

PyTorch Developer Podcast

torchdeploy is a way of running multiple Python interpreters inside the same process. It can be used to deploy Python PyTorch programs in situations where the GIL is a problem, not the CPython interpreter. How does it work, and what kind of challenges does it pose for people who want to write code that calls from C++ to Python?

Further reading.

• How the torchdeploy build system works https://dev-discuss.pytorch.org/t/torch-deploy-the-build/238
• Description of the single interpreter per Tensor invariant https://github.com/pytorch/pytorch/issues/57756
• Recent work on making it possible to load C extensions into torchdeploy https://dev-discuss.pytorch.org/t/running-multiple-python-interpreters-via-custom-dynamic-loading/241