Commit Graph

148 Commits (415042f356cb77f136d7abbcc93c8bec9fb85c1d)

Author SHA1 Message Date
Erik Montnemery 0dc21504f5
Remove support for excluding attributes in recorder platforms (#100679) 2023-09-24 14:45:06 +02:00
Ville Skyttä 34b47a2597
Remove unnnecessary pylint configs from components [m-r]* (#98924) 2023-08-24 00:56:50 +02:00
Marc Mueller cd8d6ecd81
Fix recorder DeprecationWarnings (#97738) 2023-08-04 11:32:51 +02:00
J. Nick Koston b2e708834f
Add slots to the StateMachine class (#95849) 2023-07-05 14:00:37 +02:00
Allen Porter 12129e9d21
Update service call return values and error handling (#94657)
* Update return signature of service calls

* Add timeout error handling in websocket api for service calls

* Update recorder tests to remove assertion on service call

* Remove timeout behavior and update callers that depend on it today

* Fix tests

* Add missing else

* await coro directly

* Fix more tests

* Update the intent task to use wait instead of timeout

* Remove script service call limits and limit constants

* Update tests that depend on service call limits

* Use wait instead of wait_for and add test

* Update homeassistant/helpers/intent.py

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>

---------

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2023-06-16 07:01:40 -07:00
J. Nick Koston 2ec1359063
Dynamically size recorder max backlog based on available memory (#90894)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2023-04-18 12:35:49 +02:00
J. Nick Koston fdc6cf3472
Continue recording events until final write (#91260) 2023-04-16 09:22:47 -10:00
J. Nick Koston 1379ad60c8
Ensure recorder always attempts clean shutdown if recorder thread raises (#91261)
* Ensure recorder run shutdown if the run loop raises

If anything goes wrong with the recorder we should
still try to shutdown cleanly

* tweak

* tests

* tests

* handle migraiton failure

* tweak comment

* naming

* order

* order

* order

* reword

* adjust test

* fixes

* threading

* failure case

* fix test

* have to wait for stop because the task blocks on thread join
2023-04-14 21:03:24 -04:00
epenet 2f7c5a56eb
Use tmp_path in recorder tests (#91202) 2023-04-11 09:18:16 +02:00
J. Nick Koston 49079691d4
Reduce overhead of legacy database columns on new installs (#90246)
* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* Reduce overhead of legacy database columns on new installs

* not working as expected

* override the type compiler

* override the type compiler

* override the type compiler

* override the type compiler

* Apply suggestions from code review

* pgsql char1

* make entity filter test setup with old schema

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix some more tests that were mutating state

* fix more dbstate mutations

* add shim for older tests

* split migration tests

* add coverage for purging legacy data

* tweak

* more fixes

* drop some legacy

* fix another test

* fix a few more

* add casts for postgresql in case someone deletes the schema changes table

* dry

* dry

* dry
2023-04-10 10:08:46 -04:00
J. Nick Koston 96c5e845e5
Guard against invalid ULIDs in contexts while recording events (#90889) 2023-04-06 10:34:54 +02:00
J. Nick Koston 4f1574b859
Prevent overly large event data from being stored in the database (#90747)
This is the same change as #87105 for events
2023-04-04 00:02:49 -04:00
J. Nick Koston d49fbc17df
Fix recorder attribute excludes not being effective until after startup (#90198)
* Fix attribute excludes not being effective until after startup

fixes #90016

* reduce
2023-03-23 20:52:37 -04:00
J. Nick Koston 96225bb287
Rename recorder run_history to table_managers.recorder_runs_manager (#90070) 2023-03-21 18:38:33 -10:00
epenet ab4a726e6c
Add tmpdir to known fixtures in pylint (#89844) 2023-03-17 10:22:02 +01:00
J. Nick Koston f6f3565796
Reduce latency to find stats metadata (#89824) 2023-03-16 19:00:02 -10:00
epenet ff8b91aeea
Add freezer to known test fixtures in pylint (#89825)
Add freezer to known fixtures in pylint
2023-03-16 22:39:41 -04:00
J. Nick Koston e379aa23bd
Migrate StateAttributes to use a table manager (#89760)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2023-03-15 15:26:29 -10:00
J. Nick Koston b906d67c1e
Fix filtered purge not removing newer events (#89721) 2023-03-14 16:33:19 -10:00
J. Nick Koston c41f91be89
Deduplicate entity_id in the states table (#89557) 2023-03-12 10:01:58 -10:00
J. Nick Koston 8bd43760b6
Deduplicate event_types in the events table (#89465)
* Deduplicate event_types in the events table

* Deduplicate event_types in the events table

* more fixes

* adjust

* adjust

* fix product

* fix tests

* adjust

* migrate

* migrate

* migrate

* more test fixes

* more test fixes

* fix

* migration test

* adjust

* speed up

* fix index

* fix more tests

* handle db failure

* preload

* tweak

* adjust

* fix stale docs strings, remove dead code

* refactor

* fix slow tests

* coverage

* self join to resolve query performance

* fix typo

* no need for quiet

* no need to drop index already dropped

* remove index that will never be used

* drop index sooner as we no longer use it

* Revert "remove index that will never be used"

This reverts commit 461aad2c52.

* typo
2023-03-11 14:54:55 -05:00
J. Nick Koston a9731a7b26
Make the recorder LRU adjustment threadsafe (#88443) 2023-02-19 19:30:08 +01:00
J. Nick Koston 9df00bd285
Adjust recorder LRU cache to align with the number of entities (#88350)
* Adjust size of recorder LRU based on number of entities

If there are a large number of entities the cache would
get thrashed as there were more state attributes being
recorded than the size of the cache. This meant we had
to go back to the database to do lookups frequently when
an instance has more than 2048 entities that change
frequently

* add a test

* do not actually record 4096 states

* patch target

* patch target
2023-02-19 00:02:36 -05:00
J. Nick Koston 3ca9f3c0d5
Only build the recorder SupportedDialect enum once (#88402)
Every call to dialect_name was creating a new enum object
in the recorder
2023-02-18 20:21:14 -05:00
Erik Montnemery 83e5bf7ae8
Use entity_sources to determine integration in recorder platforms (#88382) 2023-02-18 07:21:41 -06:00
J. Nick Koston 289bab6f87
Make recorder platform attribute exclude integration aware (#88357) 2023-02-18 10:08:59 +01:00
epenet 185cd61cbd
Add type hints to integration tests (recorder) (#88313) 2023-02-17 18:50:09 +01:00
Franck Nijhof ed79265843
Enable Ruff PT006 (#88165)
* Enable Ruff PT006

* Adjust existing cases

* Fix tests

* Remove unneeded parentheses
2023-02-15 14:09:50 +01:00
J. Nick Koston cbaf4764e7
Ensure MySQL tests cleanup connections and raise an exception if they do not (#87767)
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2023-02-09 16:27:53 -06:00
Erik Montnemery 2fae82a9ef
Really ensure recorder test fixture is setup before hass fixture (#87796) 2023-02-09 11:43:14 -06:00
J. Nick Koston c05a7b29e6
Terminate stale MySQL connections at the end of test runs (#87794) 2023-02-09 11:13:13 -06:00
J. Nick Koston 483b0cd017
Skip db locking test with postgresql/MySQL (#87760)
* Skip db locking test with postgresql/MySQL

https://github.com/home-assistant/core/pull/87756#issuecomment-1423828389

* switch to the guards instead
2023-02-09 13:58:32 +01:00
J. Nick Koston 77cdde94f3
Turn back off test_shutdown_before_startup_finishes test on MySQL (#87761) 2023-02-09 03:12:10 -06:00
Erik Montnemery 58618b2978
Improve recorder tests to use MariaDB / PostgreSQL (#87756) 2023-02-09 02:18:20 -06:00
epenet ba85fdcd61
Add type hints to integration tests (q-s) (#87706) 2023-02-08 08:12:54 -08:00
Erik Montnemery 94519de8dd
Upgrade SQLAlchemy to 2.0.2 (#86436)
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00
Erik Montnemery 720f51657d
Add CI job which runs recorder tests on PostgreSQL (#80614)
Co-authored-by: Franck Nijhof <git@frenck.dev>
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-07 09:46:14 -06:00
epenet 59ca7780fa
Add typing to tests with single hass argument (#87631) 2023-02-07 15:01:16 +01:00
J. Nick Koston 389fc515a1
Prevent overly large attributes from being stored in the database (#87105) 2023-02-02 11:49:33 -06:00
J. Nick Koston 1fd58b6cb7
Fix a few cartesian products in recorder tests (#87106)
Fixes
```
recorder/test_init.py:251: SAWarning: SELECT statement has a cartesian product between FROM element(s) "states" and FROM element "state_attributes".  Apply join condition(s) between each element to resolve.
```
2023-02-02 07:47:09 +01:00
J. Nick Koston c612a92cfb
Use python defaults for comparing State, LazyState, and Event objects (#86856)
* Speed up comparing State and Event objects

Use default python implementation for State and Event __hash__ and __eq__

The default implementation compared based on the id() of the object
which is effectively what we want here anyways. These overrides are
left over from the days when these used to be attrs objects

By avoiding implementing these ourselves all of the equality checks
can happen in native code

* tweak

* adjust tests

* write out some more

* fix test to not compare objects

* more test fixes

* more test fixes

* correct stats tests

* fix more tests

* fix more tests

* update sensor recorder tests
2023-01-29 13:31:43 -05:00
Erik Montnemery fea30c1ce9
Terminate strings at NUL when recording states and events (#86687) 2023-01-26 11:11:03 +01:00
epenet 30bf0634fe
Add per-file-ignore to pylint (#86289) 2023-01-22 17:26:24 +01:00
Erik Montnemery b94e1e9ef8
Fire events when long term statistics is updated (#82492)
* Fire events when long term statistics is updated

* Allow the new events to be subscribed to by anyone

* Address review comments

* Finish renaming events

* Finish renaming events

* Fix do_adhoc_statistics

* Adjust tests

* Adjust tests
2022-11-24 22:01:36 +01:00
Erik Montnemery f4951a4f31
Add CI job which runs recorder tests on MariaDB (#80586)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2022-10-19 13:04:28 +02:00
Erik Montnemery 31a787558f
Ensure recorder test fixture is setup before hass fixture (#80528)
* Ensure recorder test fixture is setup before hass fixture

* Adjust more tests
2022-10-19 07:58:47 +02:00
Erik Montnemery 2c206ad050
Fix flaky recorder test (#80246)
* Fix flaky recorder test

* Update tests/components/recorder/test_init.py

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>

Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
2022-10-14 09:10:38 +02:00
Erik Montnemery 04cc2ae264
Correct initialization of new databases (#80234) 2022-10-13 13:01:27 +02:00
Erik Montnemery 4462f2fc46
Fix recorder tests related to mysql (#80238) 2022-10-13 11:44:48 +02:00
Erik Montnemery 466c4656ca
Refactor recorder migration (#80175)
* Refactor recorder migration

* Improve test coverage
2022-10-13 08:11:54 +02:00