Skip to content

[ENG-11229] more fixes to migrate_osfmetrics_6to8#11744

Merged
aaxelb merged 7 commits into
CenterForOpenScience:developfrom
aaxelb:fix/cumulative-preprint-count-migration
May 19, 2026
Merged

[ENG-11229] more fixes to migrate_osfmetrics_6to8#11744
aaxelb merged 7 commits into
CenterForOpenScience:developfrom
aaxelb:fix/cumulative-preprint-count-migration

Conversation

@aaxelb
Copy link
Copy Markdown
Collaborator

@aaxelb aaxelb commented May 18, 2026

Ticket

ENG-11229

Purpose

(fix data-migration problems missed in #11741, oops)

Changes

  • when migrating monthly usage reports, get fresh cumulative counts independently for each (stop trying to optimize by sorting and adding counts together)
  • handle PublicItemUsageReport.item_type more consistently
    • in migrate, handle list of strings (as it sometimes/often is), so preprints are correctly special-cased (historical data stored separately)
    • when generating reports, keep the legacy item type (lower-cased django model name) in the es6 report
  • more consistently treat osfid and osfid_v1 as synonyms
    • when scheduling tasks to migrate usage reports, dedupe
    • when getting cumulative counts for either osfid or osfid_v1, query for both
  • move usage-report task scheduling (one task per item) to another background task, so the command to start the migration can finish quickly
    • reorder to handle usage reports first, so the scheduling task doesn't get buried and queue depth can be a helpful indicator of progress
  • unswap --clear-es8-data --usage-reports and --clear-es8-data --usage-events

Side Effects

QE Notes

CE Notes

Documentation

@aaxelb aaxelb force-pushed the fix/cumulative-preprint-count-migration branch from 9019fa5 to 1f97c9a Compare May 19, 2026 11:52
@aaxelb aaxelb changed the title [ENG-11229] fix: osfmetrics 6to8, item_type might be a list [ENG-11229] more fixes to migrate_osfmetrics_6to8 May 19, 2026
@aaxelb aaxelb force-pushed the fix/cumulative-preprint-count-migration branch from 95e762e to 09bb51d Compare May 19, 2026 15:43
@aaxelb aaxelb marked this pull request as ready for review May 19, 2026 16:46
@aaxelb aaxelb force-pushed the fix/cumulative-preprint-count-migration branch from 09bb51d to 207d504 Compare May 19, 2026 16:52
@aaxelb aaxelb requested a review from felliott May 19, 2026 17:20
Copy link
Copy Markdown
Member

@felliott felliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is good!

@aaxelb aaxelb merged commit c298b54 into CenterForOpenScience:develop May 19, 2026
8 checks passed
@aaxelb aaxelb deleted the fix/cumulative-preprint-count-migration branch May 19, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants