Commit Graph

91 Commits (9354df975c131e0c6d233ab25c86d73c9404bef6)

Author SHA1 Message Date
Bram Kragten fcc6ea7497
Add energy integration (#52001)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Erik <erik@montnemery.com>
2021-07-26 09:37:37 -07:00
Erik Montnemery ddc788bf8e
Include future statistics in history/list_statistic_ids (#52942)
* Include future statistics in history/list_statistic_ids

* Improve tests
2021-07-14 11:54:55 +02:00
Erik Montnemery c5556a091e
Return empty when listing statistic_ids for unsupported statistic (#52954) 2021-07-13 10:35:55 +02:00
Erik Montnemery d450cda385
Report target unit in statistics meta data (#52341) 2021-06-30 16:53:51 +02:00
Erik Montnemery 0476c7f9ee
Normalize temperature statistics to °C (#52297)
* Normalize temperature statistics to °C

* Fix tests

* Support temperature conversion to and from K, improve tests

* Fix test

* Add tests, pylint
2021-06-30 14:17:58 +02:00
Erik Montnemery 0ab999738b
Add statistics meta data table (#52331)
* Add statistics meta data table

* Tweak meta data generation
2021-06-30 13:32:17 +02:00
Erik Montnemery 0ca199d8d0
Add WS API for listing available statistic ids (#51984)
* Add WS API for listing available statistic ids

* Update homeassistant/components/history/__init__.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Bram Kragten <mail@bramkragten.nl>
2021-06-18 21:32:30 +02:00
Paulus Schoutsen 805ef3f90b
Allow fetching multiple statistics (#51996) 2021-06-18 21:03:13 +02:00
Erik Montnemery 27b9d7fed0
Fix flaky statistics tests (#51242) 2021-05-29 09:00:36 -05:00
Bram Kragten 26563e3ea4
Add statistics websocket endpoint (#51044)
Co-authored-by: Erik <erik@montnemery.com>
2021-05-25 08:03:37 -07:00
Erik Montnemery e16a8063a5
Compile statistics for energy sensors (#50829)
* Compile statistics for energy sensors

* Update tests

* Rename abs_value to state

* Tweak

* Recreate statistics table

* Pylint

* Try to fix test

* Fix statistics for multiple energy sensors

* Fix energy statistics when last_reset is not set
2021-05-20 13:05:15 +02:00
Erik Montnemery 973f59e423
Refactor history component (#50287)
* Refactor history component

* Update tests

* Address review comments

* Correct deprecated functions
2021-05-11 09:21:57 +02:00
Marc Mueller 8e2b5b36b5
Bump pyupgrade to 2.12.0 (#48943) 2021-04-09 09:58:27 -07:00
Matthew Donoughe 9011a54e7f
Switch history tests to pytest (#42318)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2021-03-17 15:32:44 +01:00
Franck Nijhof 65cf2fcb6f
Drop asynctest (#44746) 2021-01-01 22:31:56 +01:00
J. Nick Koston 9649525fe6
Increase coverage and optimize history skip_initial_state (#42014)
Avoids a database query when all states have not
changed in the time window when skip_initial_state
is set.
2020-10-19 09:53:47 +02:00
J. Nick Koston 68e1c70b14
Update history tests to use async_add_executor_job (#41526) 2020-10-09 08:36:59 +02:00
J. Nick Koston aada6a1d88
Fix logbook with empty filter and remove unused code (#40565) 2020-09-30 13:10:11 +02:00
J. Nick Koston a19e10c57a
Add entity glob matching support to history (#40387) 2020-09-28 15:58:55 +02:00
Franck Nijhof 1c2ebdf307
Upgrade black to 20.8b1 (#39287) 2020-08-27 13:56:20 +02:00
J. Nick Koston f8d547f2d3
Adjust history as all scripts can now be canceled (#37820) 2020-07-14 22:31:34 +02:00
Paulus Schoutsen 8ed1a29c82
Drop white blacklist pt1 (#37816) 2020-07-13 17:43:11 +02:00
J. Nick Koston b64ae55c66
Prebake common history queries (#37496)
* Prebake common history queries

The python overhead of to construct
the queries exceeded the database overhead.  We now
prebake the queries that get frequently polled.

This reduces the time it takes to update history_stats
sensors and can make quite a difference if there
are a lot of them.

When using the mini-graph-card card, all the entities
on the card being graphed are queried every few seconds
for new states. Previously this would tie up the database if there
are lot of these graphs in the UI.

* Update homeassistant/components/history/__init__.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Update homeassistant/components/history/__init__.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* cache entity_filter in the lambda

* switch to yield

* Revert "switch to yield"

This reverts commit f8386f4940.

* get_states always returns a list

* query wasnt actually reusable so revert part of the breakout

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2020-07-05 11:03:23 -05:00
J. Nick Koston f4528d0db2
Ensure history states can be copied (#37081)
The filter integration makes a copy of a
state object obtained from history.
2020-06-24 22:43:08 -05:00
J. Nick Koston 5b79c1f9ef
Remove support for deprecated hidden attribute from logbook and history (#37039) 2020-06-23 13:40:39 -07:00
J. Nick Koston e19c97af7c
Fix history api with no constraints (#36979) 2020-06-22 10:32:46 -05:00
J. Nick Koston 9cc20fc6b8
Fix history graphs with mysql/mariadb (#36769) 2020-06-13 21:22:16 -07:00
Paulus Schoutsen c33edbe5bb
Use TestCase.addCleanup (#36560) 2020-06-08 12:26:40 -07:00
J. Nick Koston 0a6deeb49b
Improve history api performance (#35822)
* Improve history api performance

A new option "minimal_response" reduces the amount of data
sent between the first and last history states to only the
"last_changed" and "state" fields.

Calling to_native is now avoided where possible and only
done at the end for rows that will be returned in the response.

When sending the `minimal_response` option, the history
api now returns a json response similar to the following
for an entity

Testing:

History API Response time for 1 day
Average of 10 runs with minimal_response

Before: 19.89s. (content length : 3427428)
After: 8.44s (content length: 592199)

```
[{
	"attributes": {--TRUNCATED--},
	"context": {--TRUNCATED--},
	"entity_id": "binary_sensor.powerwall_status",
	"last_changed": "2020-05-18T23:20:03.213000+00:00",
	"last_updated": "2020-05-18T23:20:03.213000+00:00",
	"state": "on"
},
...
{
	"last_changed": "2020-05-19T00:41:08Z",
	"state": "unavailable"
},
...
{
	"attributes": {--TRUNCATED--},
	"context": {--TRUNCATED--},
	"entity_id": "binary_sensor.powerwall_status",
	"last_changed": "2020-05-19T00:42:08.069698+00:00",
	"last_updated": "2020-05-19T00:42:08.069698+00:00",
	"state": "on"
}]
```

* Remove impossible state check

* Remove another impossible state check

* Update homeassistant/components/history/__init__.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Reorder to save some indent per review

* Make query response make sense with to_native=False

* Update test for 00:00 to Z change

* Update homeassistant/components/recorder/models.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2020-05-26 21:53:56 -05:00
J. Nick Koston ebed1de581
Avoid creating multiple sqlalchemy sessions in a single history call (#35721)
* Avoid a context switch in the history api

The history api was creating a job to fetch the
states and another job to convert the states to
json. This can be done in a single job which
decreases the overhead of the operation.

* Ensure there is only one sqlalchemy session created per history
query.

Most queries created three sqlalchemy sessions which was
especially slow with sqlite since it opens and closes the
database.

In testing the UI is noticeably faster at generating history
graphs for entites.

* Add additional coverage

* pass hass first to _states_to_json and _get_significant_states
2020-05-19 07:52:38 +02:00
Paulus Schoutsen 2af984917e
Use asynctest-mock in most places (#35109)
* Use asynctest-mock in most places

* Fix broken patch in pilight
2020-05-03 11:27:19 -07:00
Franck Nijhof 906385172a
String formatting improvements for tests (#33663) 2020-04-05 00:26:08 +02:00
Santobert 0186ce7896
Filter the history of device_tracker by location attributes (#33356)
* Add include_location_attributes

* Add check for different state

* Fix things

* Drop filter

* significant changes only

* Change default behavior

* Remove trailing commas
2020-03-30 08:13:22 -07:00
J. Nick Koston 2360fd4141
Switch legacy calls to init_recorder_component using async_add… (#33185) 2020-03-23 11:27:45 -07:00
J. Nick Koston 3a680bf7b7
Add a commit interval setting to recorder (#32596)
* Add a commit interval setting to recorder

* Make the default every 1s instead of immediate

* See attached py-spy flamegraphs for why 1s

* This avoids disk thrashing during event storms

* Make Home Assistant significantly more responsive on busy systems

* remove debug

* Add commit forces for tests that expect commits to be immediate

* Add commit forces for tests that expect commits to be immediate

* make sure _trigger_db_commit is in the right place (all effective "wait_recording_done" calls)

* De-duplicate wait_recording_done code
2020-03-09 17:43:26 -07:00
Bas Nijholt 8b39957c56 Sort imports according to PEP8 for components starting with "H" (#29768) 2019-12-09 14:19:48 +01:00
Paulus Schoutsen 3649a1b5e9
Filter out empty results in history API (#25633) 2019-08-01 11:52:57 -07:00
Paulus Schoutsen 93c0db2328 Lint 2019-07-31 12:53:07 -07:00
Paulus Schoutsen 4de97abc3a Black 2019-07-31 12:25:30 -07:00
Thomas Lovén 671cb0d092 Return history for entities in the order they were requested (#25560)
* Return history for entities in the order they were requested

* Fix problems. Add tests

* Update __init__.py
2019-07-31 11:59:25 -07:00
Paulus Schoutsen b8cc547fa3
Move components to folders (#20774)
* Move all components into folders

* Move component tests into folders

* Fix init moving

* Move tests

* Lint

* Update coverage

* Fix service descriptions

* Update CODEOWNERS
2019-02-05 19:31:15 -08:00