* Reduce overhead to compile statistics
statistics uses LazyState for compatibility with State when pulling
data from the database.
After the previous round of refactoring to modern history, the setters
are never called and can be removed.
* reduce
Almost 99% of items that are put into the recorder queue are
Events. Avoid wrapping them in tasks since we have to unwrap
them right away and its must faster to check for both RecorderTask
and Events since events are the common case.
* Handle statistics columns being unmigrated from previous downgrades
If the user downgraded HA from 2023.3.x to an older version without
restoring the database and they upgrade again with the same database
they will have unmigrated statistics columns since we only migrate them
once.
As its expensive to check, we do not want to check every time
at startup, so we will only do this one more time since the
risk that someone will downgrade to an older version is very
low at this point.
* add guard to sqlite to prevent re-migrate
* test
* move test to insert with old schema
* use helper
* normalize timestamps
* remove
* add check
* add fallback migration
* add fallback migration
* commit
* remove useless logging
* remove useless logging
* do the other columns at the same time
* coverage
* dry
* comment
* Update tests/components/recorder/test_migration_from_schema_32.py
* Update return signature of service calls
* Add timeout error handling in websocket api for service calls
* Update recorder tests to remove assertion on service call
* Remove timeout behavior and update callers that depend on it today
* Fix tests
* Add missing else
* await coro directly
* Fix more tests
* Update the intent task to use wait instead of timeout
* Remove script service call limits and limit constants
* Update tests that depend on service call limits
* Use wait instead of wait_for and add test
* Update homeassistant/helpers/intent.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Add additional coverage to history websocket api
related issue #93258
* Add additional coverage to history websocket api
related issue #93258
* Fix results when union query ends up at the end instead of front
* Apply suggestions from code review
* resort
* zero instead
* fix exception
* fix tests
* Support calculating changes between consecutive sum statistics
* Add support for unit conversion when calculating change
* Don't include sum in WS response unless requested
* Improve tests
* Break out calculating change to its own function
* Improve test coverage
* Auto repair incorrect collation on MySQL schema
As we do more union queries in 2023.5.x if there is a mismatch
between collations on tables, they will fail with an error
that is hard for the user to figure out how to fix
`Error executing query: (MySQLdb.OperationalError) (1271, "Illegal mix of collations for operation UNION")`
This was reported in the #beta channel and by PM from others
so the problem is not isolated to a single user
https://discord.com/channels/330944238910963714/427516175237382144/1100908739910963272
* test with ascii since older maraidb versions may not work otherwise
* Revert "test with ascii since older maraidb versions may not work otherwise"
This reverts commit 787fda1aefcd8418a28a8a8f430e7e7232218ef8.t
* older version need to check collation_server because the collation is not reflected if its the default
* Speed up logbook and history queries where ORM rows are not needed
This avoids having sqlalchemy wrap Result in ChunkedIteratorResult
which has additional overhead we do not need for these cases
* more places
* anything that uses _sorted_statistics_to_dict does not need orm rows either
* Fallback to generating a new ULID on migraiton if context is missing or invalid
It was discovered that postgresql will do a full scan if
there is a low cardinality on the index because of missing
context ids. We will now generate a ULID for the timestamp
of the row if the context data is missing or invalid
fixes#91514
* tests
* tweak
* tweak
* preen
* Ensure recorder run shutdown if the run loop raises
If anything goes wrong with the recorder we should
still try to shutdown cleanly
* tweak
* tests
* tests
* handle migraiton failure
* tweak comment
* naming
* order
* order
* order
* reword
* adjust test
* fixes
* threading
* failure case
* fix test
* have to wait for stop because the task blocks on thread join
Marc Mueller
Erik Montnemery
J. Nick Koston
Ville Skyttä
Aarni Koskela
Allen Porter