Skip to content

GH-126491: Lower heap size limit with faster marking#127519

Merged
markshannon merged 24 commits intopython:mainfrom
faster-cpython:faster-marking
Dec 6, 2024
Merged

GH-126491: Lower heap size limit with faster marking#127519
markshannon merged 24 commits intopython:mainfrom
faster-cpython:faster-marking

Conversation

@markshannon
Copy link
Member

@markshannon markshannon commented Dec 2, 2024

With marking added to the cyclic GC (#127110) we spend a lot of the time in the GC forming transitive closures, both for marking and for the increments of the incremental GC.

Unfortunately the current algorithm has a couple of mistakes in it. One harmful, one beneficial.

  • The beneficial one is counting the initial mark twice. This helps because it reduces the cost of GC on heaps with little or no garbage
  • The harmful one is allowing the amount of work done to grow in proportion to the heap size.

These more or less cancel out.
This PR deliberately counts marking as twice as effective as normal collection, but limits the amount of work done.
To do so, we need to increase the typical amount of work done a bit.
This has the advantage of limiting the amount of garbage to (roughly) 1/3 of the heap.

This PR does two things:

  • Speeds up the marking and increment creation phases
  • Visits objects a bit faster to maintain a lower heap size.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants