[v25.3.x] archival: add incremental batching to archive GC

[vbotbuildovich]

Backport #29870 to branch v25.3.x. This issue was created as a placeholder because the original PR's commit(s) could not be automatically cherry-picked.
Here are the commands to execute:

git checkout v25.3.x
git checkout -b manual-backport-29870-v25.3.x-484
git cherry-pick -x 50b857058e 4acaa35e63 61acfdbd7f

git push origin manual-backport-29870-v25.3.x-484
gh pr create \
  --title "[v25.3.x] archival: add incremental batching to archive GC" \
  --base "v25.3.x" \
  --label "kind/backport" \
  --head "manual-backport-29870-v25.3.x-484" \
  --draft \
  --repo "redpanda-data/redpanda" \
  --milestone "v25.3.x-next" \
  --body "Backport of PR https://github.com/redpanda-data/redpanda/pull/29870 "

[tyson-redpanda]

#29923