Rowversion decorator

[EdvinRygh-OBOS]

Problem statement

My team and I are building a lightweight masterdata management tool using Rayfin. In order to have a fallback to loading data from the fabric app sql server using cdf into our append-only raw lakehouse, we need a reliable watermark column in each table. That way we can easily incrementally load and store all revisions of each managed entity in our raw lakehouse even if we for some reason end up out of sync with the change data feed.

Proposed solution

Add a rowversion decorator that can be applied to tables that need a reliable watermark column for incremental downstream load/transformation

Alternatives considered

No response

Additional context

No response

[kdubau]

Thanks — this is a clear and well-motivated ask. A decorator that maintains a reliable rowversion/watermark column for incremental downstream loads makes sense, and we are tracking it. We will follow up here as it is scoped.