fix(whatsapp): store history sync delivered as JoinedGroup events - #74
fix(whatsapp): store history sync delivered as JoinedGroup events#74jqueguiner wants to merge 1 commit into
Conversation
wa-rs 0.2 never dispatches `Event::HistorySync`. It streams the backfill one conversation at a time through `Event::JoinedGroup(LazyConversation)` (see wa-rs `history_sync.rs`: "Receive and dispatch lazy conversations as they come in"). The variant still exists in the enum, so the arm matching it kept compiling while receiving nothing, and every conversation WhatsApp pushed after pairing was dropped. Measured on a fresh link: wa-rs logged "History sync progress: 775 conversations processed" while only 4 rows reached the database, and the handler's own log line never appeared once. Split the per-conversation body out of `handle_history_sync` into `store_conversation` and call it from the `JoinedGroup` arm, so history is persisted as it streams in. The `HistorySync` arm is kept: it costs nothing and resumes working if wa-rs dispatches it again. Progress is reported as a cumulative counter every 250 messages rather than one line per conversation, since a backfill carries hundreds.
MaximeGaudin
left a comment
There was a problem hiding this comment.
PR Review — #74: fix(whatsapp): store history sync delivered as JoinedGroup events
Recommendation: Request changes
Author: jqueguiner · +63/−10 across 3 files · CI: all pass
1. Intent & fit
Fits. Well-written bug fix for a serious silent failure: wa-rs 0.2 streams history sync as Event::JoinedGroup(LazyConversation) instead of Event::HistorySync, so the existing handler was dead code and 775 conversations were dropped on a fresh link. Single focused commit, conventional commit style, CHANGELOG entry added.
2. Security · Verdict: clean
No dependency changes, no CI/workflow changes, no dangerous patterns. Only control flow changes inside the existing WhatsApp event handler + a pure refactor in sync.rs. AtomicU64 for progress counter is benign.
3. Code review
Blocker: lazy_conv.conversation() clears messages — the fix stores zero messages
connector_trait.rs:193 calls lazy_conv.conversation(). In wa-rs-core (types/events.rs:87-95), this method clears conv.messages after decoding as a memory optimization:
// wa-rs-core/src/types/events.rs
pub fn conversation(&self) -> &wa::Conversation {
self.parsed.get_or_init(|| {
let mut conv = wa::Conversation::decode(&self.raw_bytes[..])
.expect("Failed to decode conversation");
conv.messages.clear(); // ← messages stripped
conv.messages.shrink_to_fit();
conv
})
}store_conversation then iterates conv.messages (now empty) and stores nothing. The Ok(0) branch silently swallows this. The fix compiles and passes CI but is functionally identical to the current broken state — conversation metadata is stored, but zero messages.
Fix: Use lazy_conv.get() instead, which preserves messages and returns Option<&WaConversation>:
Event::JoinedGroup(lazy_conv) => {
let own_identity = own_identity_holder.lock().expect("mutex").clone();
if let Some(conv) = lazy_conv.get() {
match store_conversation(&db, &config_id, &own_identity, conv) {
Ok(0) => {}
Ok(n) => {
let hist = history_count.fetch_add(n, Ordering::Relaxed) + n;
if hist % 250 < n {
eprintln!(
"[whatsapp:{config_id}] history sync: {hist} messages imported"
);
}
}
Err(e) => warn!("Failed to store history conversation: {e}"),
}
}
}get() also returns None instead of panicking on malformed protobuf, which is a robustness bonus.
Should-fix
- French comment —
connector_trait.rs:97-98. All existing comments are in English. Replace with: "Cumulative counter of imported history messages, shared across handler calls (one per conversation during a backfill)."
Nits
- Vestigial
{ }block instore_conversation—sync.rs:67. The extra braces wrapped the originalforloop body and serve no purpose after extraction. - "history sync" progress label —
connector_trait.rs:202.Event::JoinedGroupmay also fire for actual group joins, not just history backfill. Low impact since the 250-message threshold filters out small group joins.
Summary
Excellent root-cause analysis — the diagnosis of wa-rs 0.2's event dispatch change is precise and well-documented. But LazyConversation::conversation() strips conv.messages as a memory optimization, so the fix as written stores conversation metadata but zero messages. Switching to lazy_conv.get() fixes both the correctness bug and adds robustness against malformed protobuf. This must be fixed and ideally end-to-end verified before merge.
What
WhatsApp history sync is never stored. This wires it back up.
wa-rs0.2 does not dispatchEvent::HistorySync. It streams the backfill oneconversation at a time through
Event::JoinedGroup(LazyConversation)— seewa-rs/src/history_sync.rs:Event::HistorySyncstill exists in the enum, so the arm matching it inconnector_trait.rskept compiling — it just never fired. Everything WhatsApppushed after pairing was dropped on the floor.
Why it went unnoticed
The failure is silent in both directions: no error, no warning, and the
connector reports healthy. The only visible trace is a gap between what the
library logs and what the connector logs.
On a fresh pairing, before the fix:
wa-rsreportedHistory sync progress: 775 conversations processed...andProcessing history sync ... (Size: 1087297, Type: Recent)across threeRecentblobs plus oneFull. The database ended up with 4 conversationsand 14 messages, all from the live stream after the backfill finished.
This matters more than a normal dropped event: WhatsApp only sends the full
history once, right after a device is linked. Missing it means unlinking and
relinking the device to get another chance.
How
handle_history_syncintostore_conversation(db, connection_id, own_identity, conv).LazyConversation::conversation()yields awa::Conversation, which isexactly the type that loop already consumed, so the storage logic is
unchanged — it just runs one conversation at a time.
Event::JoinedGrouparm.Event::HistorySyncarm. It costs nothing and resumes working if afuture
wa-rsdispatches it again.line per conversation, since a backfill carries hundreds.
Verified
./scripts/check.sh(fmt + clippy-D warnings+ tests):Built
--releaseand confirmed the new arm compiles into the binary. I have notyet been able to verify a full end-to-end backfill against a live account: that
requires unlinking and relinking the device, which destroys the session I am
currently running on. Happy to do it if you would rather have that before
merging — the reasoning above is from reading
wa-rsand from the logs of thefailed run, not from a successful one.
Not in this PR
store_conversation. Messages withneither text nor media are still skipped, which is correct for receipts and
system messages, but it does mean the stored count is lower than the count
wa-rsreports.example the daemon is not running during pairing), relinking the device is
still the only way to get it again.