🐛 Catch up server bucket revisions on site energy totals (#51) #74

Merged
mat merged 3 commits from fix-site-bucket-revisions into main 2026-08-25 09:50:23 +00:00
Owner

Context

Fixes the site-energy half of GitHub issue #51: https://github.com/MateoGreil/homeassistant-comwatt/issues/51

The Comwatt server sometimes revises a completed hourly QUANTITY/HOUR bucket upward after publishing it. The integration folds each bucket exactly once (high-water mark), so upward revisions were never caught up, leaving the site *_total_energy sensors permanently below the server's current values.

Measured on a production instance (site 3349, 8-day audit, Aug 2026): 2 revised buckets caused a permanent drift of −42 Wh (production), −214 Wh (consumption), −173 Wh (withdrawal).

What it does

  • Per-bucket ledger on _SiteEnergyState (folded_buckets: bucket UTC ISO → metric → folded Wh), recording exactly what was folded.
  • Catch-up pass on every site fetch: buckets present in the response with a ledger entry reconcile per (bucket, metric); only positive deltas are applied (folded value is a floor), so site totals never decrease — total_increasing stays truthful.
  • Idempotent (ledger follows the server value once caught up); 100 → 80 → 110 folds exactly +10; downward revisions ignored permanently.
  • Ledger persisted in comwatt.energy_state (store minor bump); old state without the ledger key loads with an empty ledger and unchanged totals — pre-existing drift stays frozen by design.
  • Ledger pruned to the server response window, but only when the response contains at least one parseable bucket (an empty/unparseable 200 response leaves it untouched).

Not changed

  • Device *_total_energy path (PR #73), stream.py, sensor.py untouched.
  • First-fetch 8-day seeding, negative-bucket rejection, and fold-once semantics unchanged.

Verification

  • 121 tests passed (10 new: ledger fold/recording, idempotent catch-up, floor on downward revisions, per-metric independence, pruning, pre-existing totals untouched, reload round-trip + catch-up-once, old-state migration, JSON shape, empty-response guard)
  • ruff, mypy (CI config), git diff --check: clean
## Context Fixes the site-energy half of GitHub issue #51: https://github.com/MateoGreil/homeassistant-comwatt/issues/51 The Comwatt server sometimes revises a completed hourly QUANTITY/HOUR bucket upward after publishing it. The integration folds each bucket exactly once (high-water mark), so upward revisions were never caught up, leaving the site *_total_energy sensors permanently below the server's current values. Measured on a production instance (site 3349, 8-day audit, Aug 2026): 2 revised buckets caused a permanent drift of −42 Wh (production), −214 Wh (consumption), −173 Wh (withdrawal). ## What it does - Per-bucket ledger on _SiteEnergyState (folded_buckets: bucket UTC ISO → metric → folded Wh), recording exactly what was folded. - Catch-up pass on every site fetch: buckets present in the response with a ledger entry reconcile per (bucket, metric); only positive deltas are applied (folded value is a floor), so site totals never decrease — total_increasing stays truthful. - Idempotent (ledger follows the server value once caught up); 100 → 80 → 110 folds exactly +10; downward revisions ignored permanently. - Ledger persisted in comwatt.energy_state (store minor bump); old state without the ledger key loads with an empty ledger and unchanged totals — pre-existing drift stays frozen by design. - Ledger pruned to the server response window, but only when the response contains at least one parseable bucket (an empty/unparseable 200 response leaves it untouched). ## Not changed - Device *_total_energy path (PR #73), stream.py, sensor.py untouched. - First-fetch 8-day seeding, negative-bucket rejection, and fold-once semantics unchanged. ## Verification - 121 tests passed (10 new: ledger fold/recording, idempotent catch-up, floor on downward revisions, per-metric independence, pruning, pre-existing totals untouched, reload round-trip + catch-up-once, old-state migration, JSON shape, empty-response guard) - ruff, mypy (CI config), git diff --check: clean
The server sometimes revises a completed QUANTITY/HOUR bucket upward
after publication; those revisions were never caught up (#51). Record
what each fold actually added in a per-site folded_buckets ledger
(bucket UTC ISO key -> metric -> folded Wh) and, after the fold loop,
re-examine every response bucket that has ledger entries: a strictly
positive delta is added to the total and the ledger entry raised to the
new server value (idempotent next fetch). Non-positive deltas leave
total and ledger untouched, so each folded value is a floor; skipped
metrics (None/negative) get no entry and never reconcile from nothing.
Ledger entries outside the response window are pruned. In-memory only;
persistence of the ledger is deferred to a follow-up.
🐛 keep site ledger when response has no parseable buckets
All checks were successful
Validate / lint-ruff (pull_request) Successful in 7s
Validate / test-pytest (pull_request) Successful in 3m16s
Validate / type-check-mypy (pull_request) Successful in 3m16s
Validate / lint-ruff (push) Successful in 7s
Validate / test-pytest (push) Successful in 3m13s
Validate / type-check-mypy (push) Successful in 3m15s
d76ded7844
mat force-pushed fix-site-bucket-revisions from d76ded7844
All checks were successful
Validate / lint-ruff (pull_request) Successful in 7s
Validate / test-pytest (pull_request) Successful in 3m16s
Validate / type-check-mypy (pull_request) Successful in 3m16s
Validate / lint-ruff (push) Successful in 7s
Validate / test-pytest (push) Successful in 3m13s
Validate / type-check-mypy (push) Successful in 3m15s
to 160b58e3e7
All checks were successful
Validate / lint-ruff (push) Successful in 7s
Validate / test-pytest (push) Successful in 3m13s
Validate / type-check-mypy (push) Successful in 3m15s
Validate / lint-ruff (pull_request) Successful in 7s
Validate / test-pytest (pull_request) Successful in 3m13s
Validate / type-check-mypy (pull_request) Successful in 3m15s
2026-08-24 14:44:19 +00:00
Compare
mat changed title from WIP: 🐛 Catch up server bucket revisions on site energy totals (#51) to 🐛 Catch up server bucket revisions on site energy totals (#51) 2026-08-24 17:12:03 +00:00
mat merged commit 07d0aea8cc into main 2026-08-25 09:50:23 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
mat/homeassistant-comwatt!74
No description provided.