🔒️ Drop non-finite values before they reach the recorder #20

Open

mat wants to merge 1 commit from fix/46-value-guard into main

pull from: fix/46-value-guard

merge into: mat:main

mat:main

mat:fix/5-energy-hour-offset

mat:feat/c7-reauth

mat:docs/api-exploration

mat:docs/47-setup-tutorial

mat:feat/40-site-metrics

mat:chore/release-0.4.0

mat:7-problème-de-connexion-version-anterieure-a-gen4-2

mat:update-python-comwatt

mat:7-problème-de-connexion-version-anterieure-a-gen4

mat:3-adjust-update-frequency-for-comwattenergysensor

mat:9-enhancement-request-additional-device-control-modes-for-comwatt-integration

mat:fix/power-sensor

mat:fix/device-child-not-containing-id#10

mat:4-optimize-api-client-handling-for-multiple-sensors

mat:use-go.comwatt.com

Conversation 0 Commits 1 Files changed 3 +67 -7

mat commented 2026-04-24 17:14:55 +00:00

Owner

Copy link

Summary

Closes MateoGreil/homeassistant-comwatt#46.

Second layer of defence against the "negative / aberrant values" problem. #14 already maps failed fetches to UpdateFailed (entity unavailable for that cycle) instead of silently re-authing and publishing stale state. This PR adds a value-level guard: a _clean_numeric helper coerces every raw API reading to float and returns None unless the result is finite. Applied to:

• Site rates (auto_production_rate today; extends to every site metric added by #19 once it merges).
• Per-device power.
• Per-device energy delta — on a bad bucket we skip the accumulator update so the running total never absorbs a NaN / string / null.

What this does NOT fix

Pre-existing bad statistic rows in your recorder database still need manual correction via Developer Tools → Statistics. This PR only prevents new occurrences.

Test plan

• New test_non_numeric_values_are_dropped feeds NaN, Inf and the string "not-a-number" through the three ingestion paths; coordinator surfaces None and _energy_state stays untouched.
• pytest tests/ — 25 passed; ruff + mypy clean.

## Summary Closes [MateoGreil/homeassistant-comwatt#46](https://github.com/MateoGreil/homeassistant-comwatt/issues/46). Second layer of defence against the "negative / aberrant values" problem. [#14](https://git.greil.fr/mat/homeassistant-comwatt/pulls/14) already maps failed fetches to `UpdateFailed` (entity unavailable for that cycle) instead of silently re-authing and publishing stale state. This PR adds a value-level guard: a `_clean_numeric` helper coerces every raw API reading to `float` and returns `None` unless the result is finite. Applied to: - Site rates (`auto_production_rate` today; extends to every site metric added by [#19](https://git.greil.fr/mat/homeassistant-comwatt/pulls/19) once it merges). - Per-device `power`. - Per-device `energy` delta — on a bad bucket we skip the accumulator update so the running total never absorbs a NaN / string / null. ## What this does NOT fix Pre-existing bad statistic rows in your recorder database still need manual correction via **Developer Tools → Statistics**. This PR only prevents new occurrences. ## Test plan - [x] New `test_non_numeric_values_are_dropped` feeds `NaN`, `Inf` and the string `"not-a-number"` through the three ingestion paths; coordinator surfaces `None` and `_energy_state` stays untouched. - [x] `pytest tests/` — 25 passed; `ruff` + `mypy` clean.

mat added 1 commit 2026-04-24 17:14:55 +00:00

🔒️ Drop non-finite values before they reach the recorder ... e1c171eb61

Closes #46.

Issue #46 reports that the Comwatt integration occasionally records
"negative and aberrant" values into HA's long-term statistics. The
working hypothesis in that thread is that upstream API errors (HTTP
400s during the Comwatt service update on 17 June) leaked into sensor
state and contaminated the Energy dashboard.

PR #14 already eliminated the biggest class of this by mapping failed
fetches to `UpdateFailed` (entity unavailable for that cycle) instead
of silently re-authenticating and publishing stale state. This PR adds
a second layer: a `_clean_numeric` helper that coerces each raw API
value to `float` and returns `None` unless the result is finite. It is
applied wherever we pull the latest bucket:

- Site rates (`auto_production_rate` today; extensible to the rest of
  #40's metrics when that PR lands).
- Per-device `power`.
- Per-device `energy` delta — if the new bucket is non-numeric we skip
  the accumulator update entirely, so the running total never absorbs
  a NaN / string / null.

Existing stats already polluted in a user's database still need a
manual correction via Developer Tools → Statistics; this fix only
prevents new occurrences from being recorded.

## Test

- New `test_non_numeric_values_are_dropped` feeds `NaN`, `Inf` and the
  string `"not-a-number"` through the three ingestion paths and asserts
  the coordinator surfaces `None` and leaves `_energy_state` untouched.
- Power-sensor assertion normalised to `"350.0"` because
  `_clean_numeric` promotes ints to float for consistency.

mat force-pushed fix/46-value-guard from e1c171eb61 to 55b4348e89 2026-04-24 21:53:04 +00:00 Compare

All checks were successful

Hide all checks

Validate / validate-hacs (pull_request) Has been skipped

Details

Validate / validate-hassfest (pull_request) Has been skipped

Details

Validate / lint-ruff (pull_request) Successful in 7s

Details

Validate / test-pytest (pull_request) Successful in 1m44s

Details

Validate / type-check-mypy (pull_request) Successful in 1m48s

Details

Validate / validate-hacs (push) Has been skipped

Details

Validate / validate-hassfest (push) Has been skipped

Details

Validate / lint-ruff (push) Successful in 7s

Details

Validate / test-pytest (push) Successful in 1m44s

Details

Validate / type-check-mypy (push) Successful in 1m48s

Details

This pull request can be merged automatically.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin fix/46-value-guard:fix/46-value-guard

git switch fix/46-value-guard

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch main

git merge --no-ff fix/46-value-guard

git switch fix/46-value-guard

git rebase main

git switch main

git merge --ff-only fix/46-value-guard

git switch fix/46-value-guard

git rebase main

git switch main

git merge --no-ff fix/46-value-guard

git switch main

git merge --squash fix/46-value-guard

git switch main

git merge --ff-only fix/46-value-guard

git switch main

git merge fix/46-value-guard

git push origin main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

mat/homeassistant-comwatt!20

No description provided.