Add ShowDiffraction analysis modules, update UI, phase identification

[henryhng]

Summary

Introduce a complete ShowDiffraction workflow and matching UI (Auto pipeline, Phase/Exclude menus, profile/azimuthal/quality panels) with mobile support

• Crystallographic phase model library
• Center refinement: symmetry autoconvolution and phase correlation
• Ring/spot detection
• Phase calibration, hkl indexing
• Elliptical-distortion correction, masking, frame merging

Checklist

• The change includes focused tests for Python state/export behavior and frontend build coverage where possible; start with PYTHONPATH=src pytest -q and npm run build, or run scripts/widget_local_signoff.sh.
• Before committing, inspect git status --short and git diff --stat; do not commit generated HTML, docs builds, screenshots, local notebooks, private data, or machine-specific notes.
• Only the sections below that this PR touches are kept; the rest are deleted from this description.

Python API and docs — new widget, loader, or API change

• The widget has a small, stable Python API with NumPy-style docs, helpful errors, and (row, col) coordinate wording where positions are shown.
• Documentation includes a minimal tutorial notebook under docs/tutorials and an API page under docs/api when a public widget or loader is added.
• Tutorial notebooks avoid unnecessary display(...) and extra display imports; let the returned widget render naturally.

UI design and theming — frontend / viewer changes

• The frontend follows the local viewer patterns instead of inventing a new design system; compare against Show2D, Show3D, Show3DSlices, Show4DSTEM, and ShowEDS. Follow the widget UI protocol.
• Controls are compact and content-sized: use icon/text buttons for commands, switches for binary options, sliders for numeric values, menus for option sets, and avoid stretched empty control bars.
• Compact labels stay grouped with the control they name on mobile and narrow layouts. A row may wrap, but Auto, Smooth, Zoom, Pan, Contrast, fps, avg, and similar labels must not separate from their switch, menu, slider, or button.
• Compact widget control labels do not use decorative colons. Prefer Scale, Color, Auto, Link, Zoom, Pan, and ROI in dense toolbar rows; keep colons for explanatory prose and tooltips.
• Command buttons use Title Case, for example Copy, Export, Reset, Add, Clear, and Undo. Keep scientific acronyms and file formats uppercase, for example FFT, ROI, BF, ADF, HTML, PNG, and MP4.
• The widget supports both light and dark notebook/docs themes: all labels, borders, controls, plots, histograms, ROI handles, status text, and export UI remain readable.
• The widget has no hardcoded dark-only or light-only assumptions in plots, canvas backgrounds, tooltips, menus, or exported HTML.
• Histogram UI matches the existing Show2D-style interaction: compact panel, no extra whitespace, draggable min/max handles, fast center drag, and no visible lag.

Performance and real-time interaction — drag, live controls, big data paths

• Any draggable selector has live preview separate from committed widget state; use refs/CSS transforms or an equivalent fast path during drag. See performance notes.
• Use Show4DSTEM detector dragging and ShowEDS energy-band dragging as the real-time UX benchmark: aim for 60 FPS when feasible and keep live controls at 30 FPS or better.
• Real-time interactions are browser-driven and verified by actually dragging controls in JupyterLab or exported HTML, not only by reading code or unit tests.
• Show4DSTEM live-scope workflows append new ready *_master.h5 acquisitions into the same live viewer without rebuilding the notebook. Verify new masters appear in the Dataset slider, partial files are skipped until ready, and detector/scan interactions remain real time after append. See Show4DSTEM live scope folders. n/a — this PR does not touch Show4DSTEM.
• Performance reports separate load time, widget build time, first browser paint, and interaction FPS/latency. Include data shape, dtype, raw size, backend, and any crop/bin/downsample/quantization. Prefer verbose=True output that users and agents can copy; use quantem.widget.profile_widget for profiling notebooks when possible. See performance notes.
• For interaction-sensitive changes, run scripts/widget_local_signoff.sh --quick --browser for exported HTML/UI paths, fix issues immediately, rebuild, refresh, and redrive before claiming the widget is ready. See Automation and Agent signoff.
• Expensive work avoids Python/kernel round trips during pointer movement; use WebGPU, typed arrays, cached indexes, workers, or throttled schedulers where the widget interaction requires live feedback.

Measured on the real Fe3O4 SAED (512x512 float32, 1.0 MB, CPU backend, exact full-precision export, no crop/bin/quantize): data load 7 ms, widget build 85 ms, Auto pipeline 303 ms, export 37 ms (1.9 MB single-file HTML), first contentful paint 1.98 s; drag FPS in exported HTML: canvas pan 49, wheel zoom 59, histogram thumbs 60; live JupyterLab canvas drag 53 FPS; saved-state check: interact, Ctrl+S, reload without rerun restores the full view (metadata.widgets present). Zero page errors in all drives. widget_local_signoff.sh --quick --browser browser smoke: showdiffraction.html passed at 61.7 FPS; the two failing pages (Show3D reorder drag, Show3DSlices WebGPU-unavailable) are in widgets this PR does not touch, in a headless container without WebGPU.

Data honesty and repo size — data loaders, tutorial data, CI fixtures

• Large scientific data stays honest about precision and size: do not silently crop, bin, downsample, quantize, or materialize sparse zeros.
• Any binning/downsampling is explicit in the API and documentation, with the reducer named clearly, for example mean, sum, or display-scaled uint8.
• Keep main lightweight: small real rendered examples are fine, but large tutorial arrays or HTML payloads should be generated during docs builds or downloaded from public data hosting only when the size justifies it.
• Keep clone and install size small for microscope PCs. Real tutorial data belongs in public data hosting such as Hugging Face datasets, Zenodo, or release assets, then gets downloaded and cached by tutorial helpers at run time. Do not commit large real arrays, generated HTML, or rendered docs branches to this repository just to make examples work.
• CI should test data-loading protocol with tiny deterministic fixtures or monkeypatched downloads. Full real-data downloads are reserved for docs builds, release signoff, or local performance checks that explicitly opt in.

Export and saved state — HTML export, widget state, sharing

• The widget exposes export_html(path=None, title=None, mode="single", encoding="full", downsample=None) when it can be exported. Follow the HTML export protocol.
• If the widget has an in-widget Export button, it uses the standard export traits and reports filename, mode, encoding/downsample choice, and output size.
• Saved Jupyter widget state works: after interacting, Cmd+S, close/reopen in JupyterLab, and confirm the view restores without rerunning cells when the environment supports saved widget state.
• Standalone HTML works without a live Python kernel, and the exported page preserves the intended theme, viewport, interaction state, and scale/contrast state.
• GitHub sharing is treated separately from live HTML: GitHub notebook previews should use static compressed widget pictures, never heavy live widget state. See GitHub preview. n/a — this PR adds no GitHub-preview notebook.

---

This PR workflow follows the packaging standards for reproducible scientific software described in scikit-package: S. Lee, C. Myers, A. Yang, T. Zhang, Y. Xiao, and S. J. L. Billinge, Digital Discovery (2026), DOI 10.1039/d6dd00121a.

[bobleesj]

ready for review?

[henryhng]

Yup, this is ready to review thanks!

[bobleesj]

So ideally under the widget folder, we don't want to have non-widget code as much as you can. So do you think it is possible for you to create like a folder, put it there under there? ideally no seprate file if possible at all, this is to ensure we don't create files that aren't used by EVERY widget.

colormap, fft, these can have separeate files.

[bobleesj]

Thanks! A few structural points before the detailed review:

Module layout — we don't want to create a separate Python file unless it is used by every other widget. centering.py, crystal.py, diffraction.py, and phasedb.py are only used by ShowDiffraction, so please fold them into showdiffraction.py. It's okay to keep a single file even if it gets long — AI tools can easily parse it. Since the tutorial uses from quantem.widget.crystal import library_phase, re-export the public names (library_phase, Phase) from quantem.widget so users never depend on the internal file layout.

Tutorial notebook — good scope. Keep updating the existing docs/tutorials/showdiffraction.ipynb in place (I verified it builds and renders on the docs site with all three widgets live).

PR checklist — we just added a PR template: .github/PULL_REQUEST_TEMPLATE.md. Take a look. It has a small core checklist plus sections (API/docs, UI/theming, performance, data, export) that you use selectively — complete only the sections your change touches and delete the rest, so reviewers see only relevant boxes. Since this PR predates the template, please copy the core checklist and the relevant sections into the PR description. Since this PR is LLM-written, the same agent can work through the checklist and verify each box for you. T

[bobleesj]

for tests, let's also have it more explciit so that we know hwere these are used

test_showdiffracion.py, test_showdiffraction_center.py, explicit is better for humans to have an overall undrsetanding in this case.

[bobleesj]

Also, whenever you're done, you can let me know in the comment tagging me, it's ready for review so that I get notified.

So that we can reduce round trip if needed Also, I'm wondering about force push. Well, we don't want to rewrite the histories of previous users, right? Do you think it would be possible for you to use agents to perhaps resolve those conflicts and get pull and push again?

[henryhng]

@bobleesj thank you for the feedback! not quite ready for review as still figuring out how to reconcile the conflicts and figuring out the force-push for now. The reason I force-push is because it's the only solution I am aware of catching up on branches, since commits are often made to the main repository before the PR is ready to merge. I'll try to figure out another way to account for this

So ideally under the widget folder, we don't want to have non-widget code as much as you can. So do you think it is possible for you to create like a folder, put it there under there? ideally no seprate file if possible at all, this is to ensure we don't create files that aren't used by EVERY widget.

Yup, put all of the ShowDiffraction modules under showdiffraction.py, thanks for the suggestion

PR checklist

Updated in the original post of the pull request and will change it as the PR progresses

for tests, let's also have it more explicit so that we know hwere these are used

Split the test into different modules:

1. test_showdiffraction_center.py
2. test_showdiffraction_crystal.py
3. test_showdiffraction_matching.py
4. test_showdiffraction.py

[henryhng]

@bobleesj ready to review!