✨ fix: convert kWh QUANTITY/HOUR buckets to Wh at reconciliation

mat commented

2026-07-15 15:44:07 +00:00

Owner

Slice 2 of 2 — stacks on #63 (fix/reconcile-skip-unreliable). Review the combined diff against main as the complete unit-aware reconciliation fix.

Context

Slice 1 made the hourly reconciliation skip QUANTITY/HOUR buckets whose unit cannot be trusted (kWh, no live reference, incoherent ratio). That stopped the hundred-Wh downward jumps but left grid devices (injection/soutirage) with no drift correction — skipped every hour. This slice adds the kWh→Wh conversion so those grid devices get a bounded drift correction too.

The Comwatt endpoint returns mixed units per device with no unit field and no metadata predictor (confirmed — see slice 1's docstring). The unit is inferred from the ratio of the server value to the live ∫W·dt for the hour: a Wh device lands near ratio 1.0, a kWh device near 0.001. Slice 2 adds the kWh ratio band [0.0005, 0.002] to _server_bucket_to_wh: a bucket in that band is converted ×1000 to Wh before reconciling. Everything outside both bands is still skipped (slice 1's safety net stays intact).

Impact

Grid child-devices (INJECTION/WITHDRAWAL — the handoff's open soutirage 13.23 case) now reconcile correctly instead of being skipped: server kWh is converted to Wh and snapped against the live ∫W·dt as a bounded drift correction (a few Wh), never a hundred-Wh unit-conversion jump.
The reconciliation branch call site is unchanged (slice 1 already wired _server_bucket_to_wh); this slice only extends the helper with a second ratio band + its docstring.
Live ∫W·dt path, the fallback branch, the throttle, and stream.py are untouched.

Considerations

The kWh band [0.0005, 0.002] is centered on 0.001 with the same ~0.5×–2× relative tolerance as the Wh band, and is separated from it by a 250× gap (0.002, 0.5) that serves as the deliberate "incoherent ratio → skip" region. No overlap, no ambiguous boundary.
Anomalous virtual-device aggregations (e.g. the EV one-off 62.83 bucket) still fall outside both bands and are skipped — drift uncorrected for those, which is safer than a wrong snap.

Tests

3 new: kWh bucket converted before snapping (no ~0.9 snap), kWh conversion corrects live drift (server 1.1 kWh vs live 1000 Wh → +100 Wh), and the withdrawal case (server 13.23 → 13230 Wh, covering the handoff's soutirage question).
1 slice-1 test renamed/re-docstringed (it now asserts conversion instead of skip).
The two other slice-1 skip tests (no-live-reference, anomalous-ratio) and all pre-existing tests stay green unchanged.
80/80 passing, ruff clean, mypy clean.

Box verification (live, ~2h15m across multiple hourly reconcile cycles)

Deployed slices 1+2 to the HA box and monitored the *_total_energy sensors every 2 min through several real reconciliation cycles (~20:34 and ~21:29 UTC). Result:

No downward snap on any device (min per-interval Δ = 0.0 Wh for all sensors across the whole window) — the old bug's signature (injection −508 Wh, soutirage −595 Wh per hour) is gone.
A real reconcile was observed firing (solar +74.4 Wh at ~21:22, a bounded upward Wh-device drift correction where the stream had under-measured) — confirming reconciles fire and produce bounded corrections, not snaps.
Grid kWh devices (injection/soutirage) never snapped down across the cycles.

**Slice 2 of 2 — stacks on #63** (`fix/reconcile-skip-unreliable`). Review the combined diff against `main` as the complete unit-aware reconciliation fix. ## Context Slice 1 made the hourly reconciliation **skip** `QUANTITY/HOUR` buckets whose unit cannot be trusted (kWh, no live reference, incoherent ratio). That stopped the hundred-Wh downward jumps but left grid devices (injection/soutirage) with **no** drift correction — skipped every hour. This slice adds the kWh→Wh conversion so those grid devices get a bounded drift correction too. The Comwatt endpoint returns mixed units per device with no unit field and no metadata predictor (confirmed — see slice 1's docstring). The unit is inferred from the ratio of the server value to the live ∫W·dt for the hour: a Wh device lands near ratio 1.0, a kWh device near 0.001. Slice 2 adds the kWh ratio band `[0.0005, 0.002]` to `_server_bucket_to_wh`: a bucket in that band is converted `×1000` to Wh before reconciling. Everything outside both bands is still skipped (slice 1's safety net stays intact). ## Impact - Grid child-devices (INJECTION/WITHDRAWAL — the handoff's open `soutirage 13.23` case) now reconcile correctly instead of being skipped: server kWh is converted to Wh and snapped against the live ∫W·dt as a bounded drift correction (a few Wh), never a hundred-Wh unit-conversion jump. - The reconciliation branch call site is unchanged (slice 1 already wired `_server_bucket_to_wh`); this slice only extends the helper with a second ratio band + its docstring. - Live ∫W·dt path, the fallback branch, the throttle, and `stream.py` are untouched. ## Considerations - The kWh band `[0.0005, 0.002]` is centered on 0.001 with the same ~0.5×–2× relative tolerance as the Wh band, and is separated from it by a 250× gap `(0.002, 0.5)` that serves as the deliberate "incoherent ratio → skip" region. No overlap, no ambiguous boundary. - Anomalous virtual-device aggregations (e.g. the EV one-off `62.83` bucket) still fall outside both bands and are skipped — drift uncorrected for those, which is safer than a wrong snap. ## Tests - 3 new: kWh bucket converted before snapping (no ~0.9 snap), kWh conversion corrects live drift (server 1.1 kWh vs live 1000 Wh → +100 Wh), and the withdrawal case (server 13.23 → 13230 Wh, covering the handoff's soutirage question). - 1 slice-1 test renamed/re-docstringed (it now asserts conversion instead of skip). - The two other slice-1 skip tests (no-live-reference, anomalous-ratio) and all pre-existing tests stay green unchanged. - `80/80 passing`, `ruff` clean, `mypy` clean. ## Box verification (live, ~2h15m across multiple hourly reconcile cycles) Deployed slices 1+2 to the HA box and monitored the `*_total_energy` sensors every 2 min through several real reconciliation cycles (~20:34 and ~21:29 UTC). Result: - **No downward snap on any device** (min per-interval Δ = 0.0 Wh for all sensors across the whole window) — the old bug's signature (injection −508 Wh, soutirage −595 Wh per hour) is gone. - A real reconcile was observed firing (solar +74.4 Wh at ~21:22, a bounded upward Wh-device drift correction where the stream had under-measured) — confirming reconciles fire and produce bounded corrections, not snaps. - Grid kWh devices (injection/soutirage) never snapped down across the cycles.

mat added 1 commit

2026-07-15 15:44:07 +00:00

✨ fix: convert kWh QUANTITY/HOUR buckets to Wh at reconciliation

Validate / lint-ruff (push) Successful in 8s

Details

Validate / test-pytest (push) Successful in 3m6s

Details

Validate / type-check-mypy (push) Successful in 3m7s

Details

Validate / lint-ruff (pull_request) Successful in 7s

Details

Validate / test-pytest (pull_request) Successful in 3m1s

Details

Validate / type-check-mypy (pull_request) Has been cancelled

Details

523d4dd148

mat changed target branch from fix/reconcile-skip-unreliable to main

2026-07-15 15:51:04 +00:00

mat changed title from ~~WIP: ✨ fix: convert kWh QUANTITY/HOUR buckets to Wh at reconciliation~~ to ✨ fix: convert kWh QUANTITY/HOUR buckets to Wh at reconciliation

2026-07-15 15:51:21 +00:00

mat force-pushed fix/reconcile-convert-kwh from 523d4dd148

Validate / lint-ruff (push) Successful in 8s

Details

Validate / test-pytest (push) Successful in 3m6s

Details

Validate / type-check-mypy (push) Successful in 3m7s

Details

Validate / lint-ruff (pull_request) Successful in 7s

Details

Validate / test-pytest (pull_request) Successful in 3m1s

Details