Commit Graph

32 Commits (9ab95b6348e79b2f0529994f13228c25002a77b2)

Author SHA1 Message Date
Erik Montnemery 83e5bf7ae8
Use entity_sources to determine integration in recorder platforms (#88382) 2023-02-18 07:21:41 -06:00
J. Nick Koston 289bab6f87
Make recorder platform attribute exclude integration aware (#88357) 2023-02-18 10:08:59 +01:00
epenet 185cd61cbd
Add type hints to integration tests (recorder) (#88313) 2023-02-17 18:50:09 +01:00
J. Nick Koston 03b47f1931
Disable test_entity_ids on MariaDB (#87766)
Dropping the database after this test will fail on
MySQL and hang forever because it causes an InnoDB deadlock

```
| 2042 | root   | localhost:52698 | NULL               | Query   |   41 | Waiting for table metadata lock | DROP DATABASE `homeassistant-test` |    0.000 |
```
2023-02-09 15:30:35 +01:00
Erik Montnemery 58618b2978
Improve recorder tests to use MariaDB / PostgreSQL (#87756) 2023-02-09 02:18:20 -06:00
epenet 4142f0d15d
Add return type to tests without arguments (#87613)
* Add return type to tests without arguments

* Black

* Cancel fixture amends
2023-02-07 14:20:06 +01:00
epenet aa00114c2f
Add return type to async tests without arguments (#87612) 2023-02-07 10:26:56 +01:00
Erik Montnemery 53c5f02ca2
Remove some dead code from recorder (#86697) 2023-01-30 18:37:48 +01:00
J. Nick Koston c612a92cfb
Use python defaults for comparing State, LazyState, and Event objects (#86856)
* Speed up comparing State and Event objects

Use default python implementation for State and Event __hash__ and __eq__

The default implementation compared based on the id() of the object
which is effectively what we want here anyways. These overrides are
left over from the days when these used to be attrs objects

By avoiding implementing these ourselves all of the equality checks
can happen in native code

* tweak

* adjust tests

* write out some more

* fix test to not compare objects

* more test fixes

* more test fixes

* correct stats tests

* fix more tests

* fix more tests

* update sensor recorder tests
2023-01-29 13:31:43 -05:00
J. Nick Koston 52ea64d1d0
Fix repr for States and Events without a timestamp (#86391) 2023-01-22 08:11:42 -10:00
J. Nick Koston b8a1537b58
Improve performance of fetching and storing history and events with the database (#84870) 2023-01-02 13:26:08 -10:00
Erik Montnemery 5f2b4001f3
Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston 8c2743bb67
Avoid storing last_changed in the database if its the same as last_updated (#71843) 2022-05-14 12:06:31 -07:00
J. Nick Koston 1d9fb4bca8
Fix process_datetime_to_timestamp and add test coverage (#71755) 2022-05-12 17:12:50 -07:00
J. Nick Koston e2cef55162
Add history/history_during_period websocket endpoint (#71688) 2022-05-11 17:52:22 -05:00
J. Nick Koston 5db014666c
Avoid recording state_changed events in the events table (#71165)
* squash

fix mypy

* Update homeassistant/components/recorder/models.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* pass all columns

* fix commented out code

* reduce logbook query complexity

* merge

* comment

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-02 02:10:34 -05:00
J. Nick Koston c23866e5e5
De-duplicate event data into a new event_data table (#71135) 2022-05-01 21:01:17 -05:00
J. Nick Koston 3150915cb7
Convert unindexed domain queries to entity_id queries (#68404) 2022-03-20 01:28:17 -10:00
J. Nick Koston 9215702388
Separate attrs into another table (reduces database size) (#68224) 2022-03-18 00:23:13 -10:00
J. Nick Koston bf2d40adfe
Migrate from pytz to python-dateutil (#49643)
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
2021-05-07 22:46:26 -07:00
Ariana Hlavaty 1e9e40bf71
Rewrite recorder unittest tests to pytest style test function (#41264) 2020-10-06 21:24:13 +02:00
J. Nick Koston 20a136e2a1
Avoid event data serialization during recorder that we throw away (#41217)
We currently serialize the event data for state change events
and then replace it because we save the state in the states table.
Since the old state and new state are both contains in the event
the cost of serializing the data has a noticable impact when there
are many state changed events.
2020-10-05 15:08:47 +02:00
J. Nick Koston 91e0395c1c
Optimize database indexes for existing queries (#37036)
Cleanup indexes as >50% of the db size was indexes,
many of them unused in any current query

Logbook search was having to filter event_types without
an index:

  Created ix_events_event_type_time_fired
  Dropped ix_events_event_type

States had a redundant keys on composite index:

  Dropped ix_states_entity_id
  Its unused since we have ix_states_entity_id_last_updated

De-duplicate storage of context in states as
its always stored in events and can be found
by joining the state on the event_id.

  Dropped ix_states_context_id
  Dropped ix_states_context_parent_id
  Dropped ix_states_context_user_id

After schema v9:

STATES............................................ 10186       40.9%
EVENTS............................................ 5502        22.1%
IX_STATES_ENTITY_ID_LAST_UPDATED.................. 2177         8.7%
IX_EVENTS_EVENT_TYPE_TIME_FIRED................... 1910         7.7%
IX_EVENTS_CONTEXT_ID.............................. 1592         6.4%
IX_EVENTS_TIME_FIRED.............................. 1383         5.6%
IX_STATES_LAST_UPDATED............................ 1079         4.3%
IX_STATES_EVENT_ID................................ 375          1.5%
IX_EVENTS_CONTEXT_PARENT_ID....................... 347          1.4%
IX_EVENTS_CONTEXT_USER_ID......................... 346          1.4%
IX_RECORDER_RUNS_START_END........................ 1            0.004%
RECORDER_RUNS..................................... 1            0.004%
SCHEMA_CHANGES.................................... 1            0.004%
SQLITE_MASTER..................................... 1            0.004%
2020-06-23 10:57:52 -07:00
J. Nick Koston 53a91ece4e
Improve isoformat timestamp performance (#36991)
* adj

* time_fired_isoformat

* remove unused code

* tests for processing timestamps

* restore missing import lost in merge conflict

* test for None case
2020-06-22 12:06:02 -05:00
J. Nick Koston edad387b12
Make recorder execute avoid native conversion by default (#36938) 2020-06-21 21:58:57 -07:00
Paulus Schoutsen c33edbe5bb
Use TestCase.addCleanup (#36560) 2020-06-08 12:26:40 -07:00
Bas Nijholt 3d64405896 Sort imports according to PEP8 for recorder (#29652) 2019-12-08 18:48:18 +01:00
Paulus Schoutsen 4de97abc3a Black 2019-07-31 12:25:30 -07:00
Paulus Schoutsen c7f5beb794
history allowed to load states with invalid entity IDs (#20399) 2019-01-24 17:53:01 -08:00
Paulus Schoutsen 9512bb9587
Add and restore context in recorder (#15859) 2018-08-10 18:09:01 +02:00
Johann Kellerman 4cf618334c Update recorder. (#2549)
* Update recorder.

models.py:
 - Use scoped_session in models.py to fix shutdown error
__init__.py:
 - Session _commit & retry method
 - Single session var for purge_data
 - Ensure single _INSTANCE
 - repeat purge every 2 days
 - show correct time in log_error

* _commit

* Restore models to old functionality, swap purge, remove _INSTANCE cleanup from tests, typing ignore Base class

* pylint

* Remove recorder from model unit test
2016-07-23 11:25:17 -07:00
Paulus Schoutsen e0dd5a8558 Tweak Recorder 2016-07-11 08:56:07 -07:00