Skip to content

rtc: Update and improve cyclic refresh (aq-mode=3) for AV2 - #5580

Merged
urvangjoshi merged 1 commit into
AOMediaCodec:av2-encfrom
jianj-g:rtc-aq-mode-3
Oct 23, 2026
Merged

urvangjoshi merged 1 commit into
AOMediaCodec:av2-encfrom
jianj-g:rtc-aq-mode-3

Conversation

@jianj-g

@jianj-g jianj-g commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Fix existing segmentation and quantizer state synchronization issues in
cyclic refresh (CYCLIC_REFRESH_AQ / aq-mode=3) and adapt its core
parameters, 1-pass CBR rate control, and variance-based partitioning to
AV2:

  • Remove unused last_coded_q_map allocation and fix the out-of-bounds
    write on keyframes in av2_cyclic_refresh_setup().
  • Adapt compute_deltaq(), qp_thresh, qp_max_thresh, and thresh_dist_sb
    to AV2's logarithmic qindex scale and bit-depth offset
    (MAXQ_FOR_GIVEN_BIT_DEPTH), and scale rate/distortion thresholds in
    candidate_refresh_aq() for BLOCK_128X128 and BLOCK_256X256 superblocks.
  • Update segment_id on skip_txfm, propagate segment_id to
    cur_frame->seg_map, enc_seg.map, and cr->map, and re-initialize plane
    quantizers inside av2_cyclic_refresh_update_segment().
  • Add av2_cyclic_reset_segment_skip() in encode_b() when skip_txfm is
    finalized after encode_superblock(), and accumulate per-thread cyclic
    refresh counters (actual_num_seg1_blocks, actual_num_seg2_blocks,
    cnt_zeromv) across worker threads.
  • Enable CYCLIC_REFRESH_AQ in encode_with_recode_loop(), dynamically
    adapt cyclic refresh percentage and delta-q in
    av2_cyclic_refresh_postencode(), and integrate cyclic refresh with
    1-pass CBR rate control in ratectrl.c.
  • Use segment-adjusted qindex in av2_choose_var_based_partitioning()
    when cyclic refresh segmentation is enabled so boosted superblocks are
    not over-merged by high-Q threshold relaxations.
  • Add unit tests in aq_segment_test.cc, rtc_test.cc, and
    rtc_datarate_test.cc covering aq-mode=3 across RTC speeds, superblock
    sizes (64x64, 128x128, 256x256, dynamic), screen content, row-MT, and
    CBR rate targeting.

5.6% bdrate gain / 8.8% speed up on RTC test set compared with aq mode 0
on speed 6.

@jianj-g
jianj-g requested a review from marpan-max October 20, 2026 21:00
@jianj-g
jianj-g marked this pull request as draft October 20, 2026 21:43
Comment thread av2/encoder/partition_search.c
@jianj-g
jianj-g force-pushed the rtc-aq-mode-3 branch 2 times, most recently from 364bfc0 to aa16b9f Compare October 23, 2026 20:46
@jianj-g
jianj-g marked this pull request as ready for review October 23, 2026 21:36
@jianj-g
jianj-g requested a review from urvangjoshi October 23, 2026 21:43
Fix existing segmentation and quantizer state synchronization issues in
cyclic refresh (CYCLIC_REFRESH_AQ / aq-mode=3) and adapt its core
parameters, 1-pass CBR rate control, and variance-based partitioning to
AV2:

- Remove unused last_coded_q_map allocation and fix the out-of-bounds
  write on keyframes in av2_cyclic_refresh_setup().
- Adapt compute_deltaq(), qp_thresh, qp_max_thresh, and thresh_dist_sb
  to AV2's logarithmic qindex scale and bit-depth offset
  (MAXQ_FOR_GIVEN_BIT_DEPTH), and scale rate/distortion thresholds in
  candidate_refresh_aq() for BLOCK_128X128 and BLOCK_256X256 superblocks.
- Update segment_id on skip_txfm, propagate segment_id to
  cur_frame->seg_map, enc_seg.map, and cr->map, and re-initialize plane
  quantizers inside av2_cyclic_refresh_update_segment().
- Add av2_cyclic_reset_segment_skip() in encode_b() when skip_txfm is
  finalized after encode_superblock(), and accumulate per-thread cyclic
  refresh counters (actual_num_seg1_blocks, actual_num_seg2_blocks,
  cnt_zeromv) across worker threads.
- Enable CYCLIC_REFRESH_AQ in encode_with_recode_loop(), dynamically
  adapt cyclic refresh percentage and delta-q in
  av2_cyclic_refresh_postencode(), and integrate cyclic refresh with
  1-pass CBR rate control in ratectrl.c.
- Use segment-adjusted qindex in av2_choose_var_based_partitioning()
  when cyclic refresh segmentation is enabled so boosted superblocks are
  not over-merged by high-Q threshold relaxations.
- Add unit tests in aq_segment_test.cc, rtc_test.cc, and
  rtc_datarate_test.cc covering aq-mode=3 across RTC speeds, superblock
  sizes (64x64, 128x128, 256x256, dynamic), screen content, row-MT, and
  CBR rate targeting.
@urvangjoshi
urvangjoshi merged commit 1461072 into AOMediaCodec:av2-enc Oct 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants