Commit Graph

84 Commits (dd862595a3dc3c0715cb7527eb45ef5136228de1)

Author SHA1 Message Date
Erik Montnemery fd6ffef52f
Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery 606d544157
Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
Thibault Cohen 1d7d2875e1
Add websocket command recorder/import_statistics (#73937)
* Expose ws_add_external_statistics in websocket API

* Refactor

* Add tests

* Improve test coverage

Co-authored-by: Thibault Cohen <titilambert@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2022-07-21 12:36:49 +02:00
J. Nick Koston 00a79635c1
Revert "Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191)" (#73917) 2022-06-23 13:59:55 -05:00
J. Nick Koston 7ae8bd5137
Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191) 2022-06-07 16:15:50 -07:00
Erik Montnemery 5f2b4001f3
Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston c66b000d34
Reduce branching in generated lambda_stmts (#73042) 2022-06-05 21:13:31 -07:00
J. Nick Koston 8e8fa0399e
Fix statistics_during_period being incorrectly cached (#72947) 2022-06-03 13:04:46 -07:00
Erik Montnemery 23bd64b7a2
Prevent duplication of statistics metadata (#71637)
* Prevent duplication of statistics metadata

* Add models_schema_28.py

* Handle entity renaming as a recorder job

* Improve tests
2022-05-24 15:34:46 +02:00
Erik Montnemery edd7a3427c
Remove support for databases without ROW_NUMBER (#72092) 2022-05-18 21:52:38 -05:00
J. Nick Koston f3c582815c
Convert statistics to use lambda_stmt (#71903)
* Convert stats to use lambda_stmt

- Since baked queries are now [deprecated in 1.4](https://docs.sqlalchemy.org/en/14/orm/extensions/baked.html#module-sqlalchemy.ext.baked) the
  next step is to convert these to `lambda_stmt`

https://docs.sqlalchemy.org/en/14/core/connections.html#quick-guidelines-for-lambdas

* Update homeassistant/components/recorder/statistics.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-05-18 16:22:21 +02:00
Erik Montnemery 68c2b63ca1
Fix issue creation links in log messages (#71638) 2022-05-10 15:22:12 +02:00
J. Nick Koston a8aa0e1cca
Add Estimated Database Size to the recorder system health (#71463) 2022-05-07 21:02:54 -07:00
J. Nick Koston eb77f8db85
Complete strict typing for recorder (#71274)
* Complete strict typing for recorder

* update tests

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Update tests/components/recorder/test_migrate.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

* Remove the asserts

* remove ignore comments

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-05-04 12:22:50 -05:00
J. Nick Koston 27a4a9eed4
Adjust get_latest_short_term_statistics query to be postgresql compatible (#70953) 2022-04-27 21:19:36 -07:00
J. Nick Koston 3737b58e85
Avoid fetching metadata multiple times during stat compile (#70397) 2022-04-22 00:25:42 -10:00
J. Nick Koston 89807f0d2d
Reduce the number of queries needed to compile statistics (#69731)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-04-15 09:13:29 -07:00
Erik Montnemery ba07663e7d
Fix adjusting statistics in ft³ (#69913)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2022-04-12 14:08:38 -07:00
Erik Montnemery 80ff1ecb2a
Fix adjusting 5-minute statistics (#69921) 2022-04-12 18:10:17 +02:00
Erik Montnemery def305cf46
Improve the use of bakeries in recorder (#69418) 2022-04-06 08:00:10 -10:00
Erik Montnemery 8e090a8c04
Minor tweak of recorder typing (#69415) 2022-04-06 08:08:00 -07:00
Marc Mueller 53110f8cb7
Update pylint to 2.13.2 (#68704) 2022-03-27 16:08:24 +02:00
Marc Mueller 911b159281
Cleanup after pylint update (#68657) 2022-03-26 00:34:12 +01:00
J. Nick Koston 3777fa52f0
Ensure recorder statistics process registry updates in the db executor (#68633) 2022-03-24 09:48:09 -10:00
Erik Montnemery 61cc8e32f3
Include has_mean + has_sum in statistics metadata WS response (#68546)
* Include has_mean + has_sum in statistics metadata WS response

* Don't include has_mean/has_sum in history/list_statistic_ids

* Adjust tests

* Do include has_mean/has_sum in history/list_statistic_ids
2022-03-24 10:12:01 +01:00
J. Nick Koston 8c10963bc0
Small cleanups for recorder (#68551) 2022-03-23 12:12:37 -10:00
Erik Montnemery b5c5da96ac
Add WS API to adjust incorrect energy statistics (#65147)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2022-03-22 15:18:30 -07:00
Pawel bcb8c7ec3c
Add API endpoint get_statistics_metadata (#68471)
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-03-21 21:14:47 -07:00
Marc Mueller cb736eaeaf
Add type ignore error codes [recorder] (#66780) 2022-02-18 10:37:38 +01:00
Erik Montnemery 910b1f1ec8
Speed up deletion of duplicated statistics (#66014) 2022-02-07 09:11:52 -08:00
Erik Montnemery ac7662c82d
Remove limit of amount of duplicated statistics (#65641) 2022-02-04 14:49:45 +01:00
Franck Nijhof 7a2b699371
Update codespell to 2.1.0 (#64661) 2022-01-21 22:05:24 +01:00
Erik Montnemery 2f8e65a9b0
Store deleted duplicated statistics in .storage (#62574) 2021-12-22 14:27:56 +01:00
Erik Montnemery 48b3d6e1c0
Save original + duplicate pairs when deleting duplicated statistics (#62498) 2021-12-21 15:24:36 +01:00
Erik Montnemery cab2a74b5f
Don't pollute config dir with deleted duplicated statistics (#62489) 2021-12-21 14:27:35 +01:00
Erik Montnemery bceeaec2f8
Remove duplicated statistics rows (#61146)
* Remove duplicated statistics

* Fix misleading docstring

* Pylint knows best

* Correct test

* Oops

* Prevent insertion of duplicated statistics

* Tweak

* pylint

* Add models_schema_23.py

* Tweak
2021-12-13 14:15:36 +01:00
Erik Montnemery dc5888ab4a
Correct recorder.statistics.get_last_statistics (#61421) 2021-12-10 10:09:29 -08:00
Erik Montnemery 4a5238efa5
Add support for calculating daily and monthly fossil energy consumption (#59588) 2021-11-22 09:10:54 -08:00
Erik Montnemery bcd9f3c05f
Correct end time for monthly statistics summary (#59551)
* Correct end time for monthly statistics summary

* Add tests
2021-11-12 15:01:40 +01:00
Erik Montnemery 65b1f0d9eb
Minor refactor of energy validator (#58209) 2021-11-10 22:38:15 -08:00
Marc Mueller 84618fa831
Use assignment expressions 30 (#58714) 2021-10-30 16:33:42 +02:00
Erik Montnemery ac5e32d648
Corrections for external statistics (#58469) 2021-10-26 14:05:45 +02:00
Erik Montnemery f594bc353b
Add support for external statistics (#56607)
* Support external statistics

* Update tests

* Lint

* Adjust code after rebase

* Separate external statistic_id with :, add name to metadata

* Adjust tests

* Simplify get_metadata_with_session

* Address review comments

* Allow updating external statistics

* Validate input

* Adjust tests after rebase

* Pylint

* Adjust tests

* Improve test coverage
2021-10-26 10:26:50 +02:00
Erik Montnemery e3534eec87
Report orphaned statistics in statistic validation (#57324) 2021-10-20 15:57:22 +02:00
Erik Montnemery 9a26a8cfd8
Add support for daily and monthly statistics (#57576)
* Add support for daily and monthly statistics

* Remove debug code

* Format code

* Don't use dateutil package

* Remove 2 TODOs

* Remove TODO

* Add comments
2021-10-19 08:29:23 +02:00
Marc Mueller aa7dc78a1e
Use assignment expressions 11 (#57792) 2021-10-17 20:15:48 +02:00
Erik Montnemery ee98849360
Always include start point for statistics (#57182) 2021-10-12 10:39:36 -07:00
Erik Montnemery 0139bfa749
Detect if mysql and sqlite support row_number (#57475) 2021-10-11 21:17:18 -07:00
Erik Montnemery be61009030
Correct SQL query generated by get_metadata_with_session (#57225)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2021-10-07 13:23:23 -07:00
Erik Montnemery 6af1a835e6
Optimize statistics generation (#56821)
* Optimize statistics generation

* pylint
2021-09-30 17:14:36 +02:00