Skip to content

Simplified LXR GC - #1508

Merged
qinsoon merged 1787 commits into
mmtk:masterfrom
wenyuzhao:lxr-x/simplified
Aug 19, 2026
Merged

qinsoon merged 1787 commits into
mmtk:masterfrom
wenyuzhao:lxr-x/simplified

Conversation

@wenyuzhao

@wenyuzhao wenyuzhao commented Jun 5, 2026 •

Copy link
Copy Markdown
Member

This pull request introduces a simplified version of the LXR collector. There are many features not included in this version of LXR, such as reference processing, and they will be ported to the master branch in the future.

In addition to the LXR plan, this PR also makes the following changes to mmtk-core

  • Field unlog bits metadata and field-logging barrier. They can be used by generational and concurrent Immix plans to implement remembered set and SATB.
  • Pause::RefCount: a new pause kind for reference counting.
  • RootsWorkFactory::create_process_roots_work_with_root_kind: an experimental API with RootKind.
  • Renamed pause-start and pause-end hooks to merge the two mechanisms from the master branch and the LXR branch to make the names consistent.
  • UnstraddlableRegion to prevent the invocation of region.contains on regions which objects can straddle, specifically Line.
  • Many changes to the ImmixSpace and LargeObjectSpace to support reference counting-based Immix plans.
  • Additional object metadata to support LXR.
  • Additional scheduler stages to support LXR.
  • Merged the implementations of ObjectReference::iterate_fields.
  • The definition of "chunk index" is changed.
  • Map32: 0 is now a valid entry of the linked list, and invalid entries are represented as -1.

Related PRs:

@qinsoon

qinsoon commented Aug 18, 2026 •

Copy link
Copy Markdown
Member

I started a performance run to compare simplified LXR and concurrent Immix from this PR, and full featured LXR from https://github.com/wenyuzhao/mmtk-core/tree/lxr. Weak reference and class unloading is disabled. All tests use 16 total GC threads (and 16 concurrent GC threads).

The experiment is still running. We may not see all the benchmarks until tomorrow. I found that I was not using a right build for full featured LXR (I used the jdk11 binding with jdk21, and surprisingly it ran fine without weak ref and class unloading).
https://squirrel.anu.edu.au/plotty/yilin/mmtk/#0|boar-2026-08-18-Tue-020623&benchmark^buildstring^invocation^iteration&time^time.other^time.stw&|10&iteration^1^2|20&1^invocation|30&1&benchmark&buildstring;jdk-mmtk-lxr-full.lxr.common.threads-16.dacapochopin|41&Histogram%20(with%20CI)^buildstring^benchmark&

The following is the new result. I included 4 configs:

  • Full featured LXR
  • Full featured LXR without weak ref and class unloading (however this didn't run, so we can't have a 'fair' comparison)
  • Simplified LXR without weak ref and class unloading
  • Concurrent Immix without weak ref and class unloading

Time
simplified-vs-full-time

Latency
simplified-vs-full-latency

@wks

wks commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

binding-refs
OPENJDK21_BINDING_REPO=wenyuzhao/mmtk-openjdk
OPENJDK21_BINDING_REF=lxr-x/jdk21-simplified
JIKESRVM_BINDING_REPO=qinsoon/mmtk-jikesrvm
JIKESRVM_BINDING_REF=update-mmtk-lxr-api-changes
RUBY_BINDING_REPO=qinsoon/mmtk-ruby
RUBY_BINDING_REF=update-mmtk-lxr-api-changes

@wks
wks marked this pull request as ready for review August 18, 2026 09:04
@wks wks added the PR-extended-testing Run extended tests for the pull request label Aug 18, 2026
`trace_object_without_moving` can still be called during RC.
@wks

wks commented Aug 19, 2026 •

Copy link
Copy Markdown
Collaborator

LXR crashes when using MemBalancer (OpenJDK without -Xms or -Xmx). The following log is from running GCBench, but it is also reproducible when running lusearch, too.

wks@luna ~/c/j/gcbench> MMTK_NO_REFERENCE_TYPES=true MMTK_PLAN=LXR taskset -c 0-15 /home/wks/projects/mmtk-github/parallels/lxr/lxr-x-simplified/openjdk/build/linux-x86_64-server-slowdebug/jdk/bin/java -XX:MetaspaceSize=500M -XX:+UseThirdPartyHeap -cp . GCBench
[2026-08-19T02:42:06Z INFO  mmtk::util::metadata::side_metadata::layout] Initializing side metadata base: vm_specs_registered=true core_end=15069496737792 vm_end=0xfb6a4000000
[2026-08-19T02:42:06Z INFO  mmtk::util::metadata::side_metadata::layout] Quarantine side metadata range: total_bytes=0xfb6a4000000, pages=0xfb6a4000, granularity=0x400000
[2026-08-19T02:42:07Z INFO  mmtk::util::metadata::side_metadata::layout] Side metadata base initialized at 0x6f46d8c00000 (range: 0x6f46d8c00000 - 0x7efd7cc00000)
[2026-08-19T02:42:07Z INFO  mmtk::memory_manager] Initialized MMTk with LXR (DynamicHeapSize(6815744, 8327790592))
Running in a single thread...
Garbage Collector Test
 Stretching memory with a binary tree of depth 20
 Total memory available=6815744 bytes  Free memory=2494464 bytes
[2026-08-19T02:42:07Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (1672/1664 pages)
[2026-08-19T02:42:07Z INFO  mmtk::plan::lxr::global] Scheduling RC collection...
[2026-08-19T02:42:07Z INFO  mmtk::scheduler::scheduler] End of GC (1626/1727 pages, took 24.25 ms)
[2026-08-19T02:42:07Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (1729/1727 pages)
[2026-08-19T02:42:07Z INFO  mmtk::plan::lxr::global] Scheduling emergency full-heap collection...
[2026-08-19T02:42:07Z INFO  mmtk::scheduler::scheduler] End of GC (823/1664 pages, took 69.32 ms)
[2026-08-19T02:42:07Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (1666/1664 pages)
[2026-08-19T02:42:07Z INFO  mmtk::plan::lxr::global] Scheduling RC collection...
[2026-08-19T02:42:07Z INFO  mmtk::scheduler::scheduler] End of GC (1789/2074 pages, took 50.53 ms)
[2026-08-19T02:42:07Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (2076/2074 pages)
[2026-08-19T02:42:07Z INFO  mmtk::plan::lxr::global] Scheduling emergency full-heap collection...
[2026-08-19T02:42:08Z INFO  mmtk::scheduler::scheduler] End of GC (2067/2607 pages, took 111.79 ms)
[2026-08-19T02:42:08Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (2613/2607 pages)
[2026-08-19T02:42:08Z INFO  mmtk::plan::lxr::global] Scheduling RC collection...
[2026-08-19T02:42:08Z INFO  mmtk::scheduler::scheduler] End of GC (2732/3258 pages, took 40.93 ms)
[2026-08-19T02:42:08Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (3263/3258 pages)
[2026-08-19T02:42:08Z INFO  mmtk::plan::lxr::global] Scheduling concurrent marking initial pause...
[2026-08-19T02:42:08Z INFO  mmtk::scheduler::scheduler] End of GC (3485/3258 pages, took 39.41 ms)
[2026-08-19T02:42:08Z INFO  mmtk::util::heap::gc_trigger] [POLL] immix: Triggering collection (3497/3258 pages)
[2026-08-19T02:42:08Z INFO  mmtk::plan::lxr::global] Scheduling concurrent marking final pause...

thread '<unnamed>' (29882) panicked at /home/wks/projects/mmtk-github/parallels/lxr/lxr-x-simplified/mmtk-core/src/plan/lxr/gc_work/mature_evac.rs:135:13:
Invalid slot: OpenJDKSlot { addr: 0x400e9378 } -> 0x827f004
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread '<unnamed>' (29886) panicked at /home/wks/projects/mmtk-github/parallels/lxr/lxr-x-simplified/mmtk-core/src/plan/lxr/gc_work/mature_evac.rs:135:13:
Invalid slot: OpenJDKSlot { addr: 0x40a54794 } -> 0x805000c

thread '<unnamed>' (29880) panicked at /home/wks/projects/mmtk-github/parallels/lxr/lxr-x-simplified/mmtk-core/src/plan/lxr/gc_work/mature_evac.rs:135:13:
Invalid slot: OpenJDKSlot { addr: 0x409f04cc } -> 0x8011002

thread '<unnamed>' (29883) panicked at /home/wks/projects/mmtk-github/parallels/lxr/lxr-x-simplified/mmtk-core/src/plan/lxr/gc_work/mature_evac.rs:135:13:
Invalid slot: OpenJDKSlot { addr: 0x40d31ff8 } -> 0x81e9e94
fish: Job 1, 'MMTK_NO_REFERENCE_TYPES=true MM…' terminated by signal SIGABRT (Abort)

The panic happens in the v & 0b111 branch:

        let v: u32 = unsafe { s.to_address().load::<u32>() };
        if v & 0b111 != 0 {
            panic!("Invalid slot: {s:?} -> {v:#x}");
        }

@qinsoon qinsoon left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

There are many issues related to the LXR impl and the other changes introduced in this PR that still need to be addressed properly. Besides, this PR is very large and includes many changes that should have been split into smaller PRs.

Normally, we would not consider a PR in this state ready to merge. However, we have decided to make an exception in this case and address the remaining issues after the merge:

  • This PR contains many years' worth of optimization work -- we don't want to accidentally revert important optimizations. Merging it and letting CI track the performance can reduce the risk of causing any regressions.
  • This PR introduces changes that may need proper redesign. Merging it and having future refactoring/redesign go through code review is also helpful.
  • We are currently under time pressure for LXR-related binding work. Merging the PR first also helps us parallelize future work on LXR.

Thus, I think this is okay to merge this PR right now, as we have done our best to make sure no regression for bindings. This is an exception to our usual review and merge criteria and should not be taken as a precedent for future PRs.

@qinsoon
qinsoon enabled auto-merge August 19, 2026 04:20
@qinsoon
qinsoon added this pull request to the merge queue Aug 19, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 19, 2026
@qinsoon qinsoon removed the PR-extended-testing Run extended tests for the pull request label Aug 19, 2026
@qinsoon
qinsoon added this pull request to the merge queue Aug 19, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Aug 19, 2026
@qinsoon
qinsoon added this pull request to the merge queue Aug 19, 2026
@wks

wks commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

OpenJDK 11 and 21 tests timed out while running sudo apt-get update -y. It is possible that the network was under load at that time.

@qinsoon

qinsoon commented Aug 19, 2026 •

Copy link
Copy Markdown
Member

OpenJDK 11 and 21 tests timed out while running sudo apt-get update -y. It is possible that the network was under load at that time.

Yeah. I noticed some tests failed at apt-get these few days.

Merged via the queue into mmtk:master with commit 2feeacc Aug 19, 2026
32 of 35 checks passed
mmtkgc-bot added a commit to mmtk/mmtk-openjdk that referenced this pull request Aug 19, 2026
The mmtk-core PR: mmtk/mmtk-core#1508

Important features that has been removed from the main lxr branch:
* class unloading
* weak ref processing
* various performance optimizations

---------

Co-authored-by: Hayley Patton <43771738+no-defun-allowed@users.noreply.github.com>
Co-authored-by: Yi Lin <qinsoon@gmail.com>
Co-authored-by: mmtkgc-bot <mmtkgc.bot@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Kunshan Wang <wks1986@gmail.com>
@qinsoon

qinsoon commented Aug 19, 2026 •

Copy link
Copy Markdown
Member

Updated results with full featured LXR

simplified-vs-full-time simplified-vs-full-latency logs: https://squirrel.anu.edu.au/plotty/yilin/mmtk/#0|boar-2026-08-19-Wed-025947&benchmark^buildstring^invocation^iteration&time^time.other^time.stw&|10&iteration^1^2|20&1^invocation|30&1&benchmark&buildstring;jdk-mmtk-lxr-full.lxr.common.threads-16.dacapochopin|41&Histogram%20(with%20CI)^buildstring^benchmark&

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants