* Add additional coverage to history websocket api
related issue #93258
* Add additional coverage to history websocket api
related issue #93258
* Fix results when union query ends up at the end instead of front
* Apply suggestions from code review
* resort
* zero instead
* fix exception
* fix tests
* delete more code
* tweak
* tweak
* wrappers
* restore lost performance
* restore lost performance
* restore lost performance
* compact
* reduce
* fix refactor
* DRY
* tweak
* delete the start time state injector
* move away the legacy code
* tweak
* adjust
* adjust
* tweak
* ignore impossible
* fix a bug where the first start was changed to the start time when there was no previous history recorded before
* avoid the empty scan most cases
* postgresql
* fixes
* workaround for mariadb < 10.4
* remove unused
* remove unused
* adjust
* bail early
* tweak
* tweak
* fix more tests
* fix recorderrun being init in the future in the test
* run history tests on schema 30 as well
* Revert "run history tests on schema 30 as well"
This reverts commit d798b100ac.
* reduce
* cleanup
* tweak
* reduce
* prune
* adjust
* adjust
* adjust
* reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code
* Revert "reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code"
This reverts commit bf974e103e.
* fix test
* Revert "Revert "reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code""
This reverts commit 119354499e.
* more coverage
* adjust
* fix for table order
* impossible for it to be missing
* remove some more legacy from the all states
* Deduplicate event_types in the events table
* Deduplicate event_types in the events table
* more fixes
* adjust
* adjust
* fix product
* fix tests
* adjust
* migrate
* migrate
* migrate
* more test fixes
* more test fixes
* fix
* migration test
* adjust
* speed up
* fix index
* fix more tests
* handle db failure
* preload
* tweak
* adjust
* fix stale docs strings, remove dead code
* refactor
* fix slow tests
* coverage
* self join to resolve query performance
* fix typo
* no need for quiet
* no need to drop index already dropped
* remove index that will never be used
* drop index sooner as we no longer use it
* Revert "remove index that will never be used"
This reverts commit 461aad2c52.
* typo
Dropping the database after this test will fail on
MySQL and hang forever because it causes an InnoDB deadlock
```
| 2042 | root | localhost:52698 | NULL | Query | 41 | Waiting for table metadata lock | DROP DATABASE `homeassistant-test` | 0.000 |
```
* Speed up comparing State and Event objects
Use default python implementation for State and Event __hash__ and __eq__
The default implementation compared based on the id() of the object
which is effectively what we want here anyways. These overrides are
left over from the days when these used to be attrs objects
By avoiding implementing these ourselves all of the equality checks
can happen in native code
* tweak
* adjust tests
* write out some more
* fix test to not compare objects
* more test fixes
* more test fixes
* correct stats tests
* fix more tests
* fix more tests
* update sensor recorder tests
* Separate recorder database schema from other classes
* fix logbook imports
* migrate new tests
* few more
* last one
* fix merge
Co-authored-by: J. Nick Koston <nick@koston.org>
We currently serialize the event data for state change events
and then replace it because we save the state in the states table.
Since the old state and new state are both contains in the event
the cost of serializing the data has a noticable impact when there
are many state changed events.
Cleanup indexes as >50% of the db size was indexes,
many of them unused in any current query
Logbook search was having to filter event_types without
an index:
Created ix_events_event_type_time_fired
Dropped ix_events_event_type
States had a redundant keys on composite index:
Dropped ix_states_entity_id
Its unused since we have ix_states_entity_id_last_updated
De-duplicate storage of context in states as
its always stored in events and can be found
by joining the state on the event_id.
Dropped ix_states_context_id
Dropped ix_states_context_parent_id
Dropped ix_states_context_user_id
After schema v9:
STATES............................................ 10186 40.9%
EVENTS............................................ 5502 22.1%
IX_STATES_ENTITY_ID_LAST_UPDATED.................. 2177 8.7%
IX_EVENTS_EVENT_TYPE_TIME_FIRED................... 1910 7.7%
IX_EVENTS_CONTEXT_ID.............................. 1592 6.4%
IX_EVENTS_TIME_FIRED.............................. 1383 5.6%
IX_STATES_LAST_UPDATED............................ 1079 4.3%
IX_STATES_EVENT_ID................................ 375 1.5%
IX_EVENTS_CONTEXT_PARENT_ID....................... 347 1.4%
IX_EVENTS_CONTEXT_USER_ID......................... 346 1.4%
IX_RECORDER_RUNS_START_END........................ 1 0.004%
RECORDER_RUNS..................................... 1 0.004%
SCHEMA_CHANGES.................................... 1 0.004%
SQLITE_MASTER..................................... 1 0.004%
* adj
* time_fired_isoformat
* remove unused code
* tests for processing timestamps
* restore missing import lost in merge conflict
* test for None case
* Update recorder.
models.py:
- Use scoped_session in models.py to fix shutdown error
__init__.py:
- Session _commit & retry method
- Single session var for purge_data
- Ensure single _INSTANCE
- repeat purge every 2 days
- show correct time in log_error
* _commit
* Restore models to old functionality, swap purge, remove _INSTANCE cleanup from tests, typing ignore Base class
* pylint
* Remove recorder from model unit test
Erik Montnemery
Ville Skyttä
J. Nick Koston
epenet
Ariana Hlavaty
Paulus Schoutsen
Bas Nijholt
Paulus Schoutsen
Paulus Schoutsen
Johann Kellerman