gh-90815: Add mimalloc memory allocator#109914
Merged
vstinner merged 24 commits intopython:mainfrom Oct 30, 2023
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds mimalloc as an optional (but preferred when available) allocator to CPython. This is a bit of a mashup of the work from #109914 and the work of @colesbury to use mimalloc for no-gil and various updates to bring it up to current CPython.
The configuration logic added by @tiran is re-used and we keep pymalloc support unlike in the version from @colesbury. mimalloc is updated to 2.12 and along with a few changes @colesbury made to it.
This has run into some issues with subinterpreter support in that the allocator's are now stored in thread state and are per-thread. Sub interpreters in some scenarios will create a thread state on one thread and run that on another thread. Most of these are documented in the code base as being known issues. I've modified these so that we will find the right thread based upon the current thread ID and switch to it rather than getting the head thread. This seems pretty reasonable but looks pretty weird when we need to do it at interpreter shutdown.