mat/homeassistant-comwatt
1
0
Fork 0
Code Issues 1 Pull requests 1 Projects Releases Packages Activity Actions

WIP: ✨ Re-sync live energy vs server QUANTITY/HOUR buckets — closes #5 #42 (streaming 5/5) #60

Closed
mat wants to merge 1 commit from feat/stream-resync into feat/stream-energy
pull from: feat/stream-resync
merge into: mat:feat/stream-energy
Conversation 1 Commits 1 Files changed 2 +309 -30
mat commented 2026-07-14 15:14:31 +00:00
Owner
Copy link

Slice 5 of 5 (stacked on #59). Closes #5 and #42. The energy is already real-time (slice 4); this bounds its long-term accuracy against the server's authoritative hourly buckets.

What

When the stream owns the live total, each new server QUANTITY/HOUR bucket for a completed UTC hour RECONCILES the total instead of accumulating:

drift = server_val - live_by_hour[hour]
live_total_wh += drift
live_by_hour[hour] = server_val   # snap → re-fetch never re-corrects

The existing bucket_dt > last_bucket_ts high-water-mark guard ensures each completed+published hour reconciles exactly once and the current in-progress hour is never touched. When the stream is not active, buckets accumulate into total_wh (unchanged fallback).

Self-healing / gap recovery

If the stream missed a whole hour, live_by_hour lacks it → drift = server_val - 0 = server_val fills the gap from the authoritative bucket.

Spike finding (validates the design)

A live spike across two hour rollovers (14:00 and 15:00 UTC) observed zero QUANTITY measurements on the stream — the WebSocket feed pushes only FLOW (power) + STATE (switch), never energy buckets. So the REST QUANTITY/HOUR poll is the only source of authoritative energy, and this REST re-sync is necessary (the stream cannot provide it).

Bucket-labeling assumption

Server bucket_dt assumed START-of-hour (matches the live accumulator's sample-hour key). Documented in the method docstring; one-line fix if real data proves END-labeled.

Known follow-ups (Minor, for a later fix PR)

  • Thread-safety: the executor's live_total_wh write-back can clobber event-loop deltas arriving during the QUANTITY HTTP fetch (<0.1%/yr, self-healed at next reconciliation). Fix: apply only the reconciliation drift as += instead of overwriting.
  • live_by_hour grows unbounded (one entry/hour). Add pruning of hours older than the high-water mark.

Tests

6 tests (T1-T6) with exact reconciliation arithmetic + repurposed slice-4 test. 69/69 passing, ruff + mypy clean.

Stacking

Base: feat/stream-energy (PR #59). Rebase down the stack to main as each merges.

Slice 5 of 5 (stacked on #59). **Closes #5 and #42.** The energy is already real-time (slice 4); this bounds its long-term accuracy against the server's authoritative hourly buckets. ## What When the stream owns the live total, each new server `QUANTITY/HOUR` bucket for a **completed** UTC hour RECONCILES the total instead of accumulating: ``` drift = server_val - live_by_hour[hour] live_total_wh += drift live_by_hour[hour] = server_val # snap → re-fetch never re-corrects ``` The existing `bucket_dt > last_bucket_ts` high-water-mark guard ensures each completed+published hour reconciles exactly once and the current in-progress hour is never touched. When the stream is not active, buckets accumulate into `total_wh` (unchanged fallback). ## Self-healing / gap recovery If the stream missed a whole hour, `live_by_hour` lacks it → `drift = server_val - 0 = server_val` fills the gap from the authoritative bucket. ## Spike finding (validates the design) A live spike across two hour rollovers (14:00 and 15:00 UTC) observed **zero QUANTITY measurements on the stream** — the WebSocket feed pushes only FLOW (power) + STATE (switch), never energy buckets. So the REST QUANTITY/HOUR poll is the only source of authoritative energy, and this REST re-sync is necessary (the stream cannot provide it). ## Bucket-labeling assumption Server `bucket_dt` assumed START-of-hour (matches the live accumulator's sample-hour key). Documented in the method docstring; one-line fix if real data proves END-labeled. ## Known follow-ups (Minor, for a later fix PR) - Thread-safety: the executor's `live_total_wh` write-back can clobber event-loop deltas arriving during the QUANTITY HTTP fetch (<0.1%/yr, self-healed at next reconciliation). Fix: apply only the reconciliation drift as `+=` instead of overwriting. - `live_by_hour` grows unbounded (one entry/hour). Add pruning of hours older than the high-water mark. ## Tests 6 tests (T1-T6) with exact reconciliation arithmetic + repurposed slice-4 test. 69/69 passing, ruff + mypy clean. ## Stacking Base: `feat/stream-energy` (PR #59). Rebase down the stack to `main` as each merges.
✨ Re-sync live energy against server QUANTITY/HOUR buckets (streaming 5/5, closes #5 #42)
All checks were successful
Validate / lint-ruff (pull_request) Successful in 7s
Details
Validate / test-pytest (pull_request) Successful in 3m5s
Details
Validate / type-check-mypy (pull_request) Successful in 3m11s
Details
Validate / lint-ruff (push) Successful in 7s
Details
Validate / test-pytest (push) Successful in 3m5s
Details
Validate / type-check-mypy (push) Successful in 3m12s
Details
e0a2e94893 
When the stream owns the live total (live_total_wh is not None), each new
server QUANTITY/HOUR bucket for a completed UTC hour RECONCILES the total
instead of accumulating: drift = server_val - live_by_hour[hour], applied as
live_total += drift, then live_by_hour[hour] is snapped to the authoritative
value so a re-fetch of the same hour never re-corrects. When the stream has
not taken over, buckets accumulate into total_wh as before (unchanged
fallback for accounts/devices without streaming).

The existing bucket_dt > last_bucket_ts high-water-mark guard ensures each
completed+published hour is reconciled exactly once; the current in-progress
hour is never touched (the server hasn't published it). If the stream missed
a whole hour, live_by_hour lacks that hour so drift = server_val, filling the
gap from the authoritative bucket — self-healing recovery.

Bucket-labeling assumption (documented): server bucket_dt is START-of-hour,
matching the live accumulator's sample-hour key. One-line fix if real data
proves END-labeled.

Closes #5 and #42: energy is real-time (slice 4) AND long-term-accurate
against the server (this slice).

6 tests (T1-T6) + repurposed slice-4 test. 69/69 passing, ruff + mypy clean.
mat commented 2026-07-14 17:12:48 +00:00
Author
Owner
Copy link

Landed in main via the linear stack fast-forward (commit 9d016f7). This slice is part of the streaming feature merged as #56…#62 — closing (its commits are already in main).

Landed in `main` via the linear stack fast-forward (commit `9d016f7`). This slice is part of the streaming feature merged as #56…#62 — closing (its commits are already in `main`).
mat closed this pull request 2026-07-14 17:12:48 +00:00
All checks were successful
Validate / lint-ruff (pull_request) Successful in 7s
Details
Validate / test-pytest (pull_request) Successful in 3m5s
Details
Validate / type-check-mypy (pull_request) Successful in 3m11s
Details
Validate / lint-ruff (push) Successful in 7s
Details
Validate / test-pytest (push) Successful in 3m5s
Details
Validate / type-check-mypy (push) Successful in 3m12s
Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
mat/homeassistant-comwatt!60
No description provided.