* Add support for streaming (push) history
Currently we poll for history data, like logbook we can stream
this data to avoid database overhead
* Update homeassistant/components/history/__init__.py
* merge filter
* expose new api
* expose new api
* expose new api
* expose new api
* coverage
* tests
* fixes
* tweak
* tweak
* tweak
* DRY
* leaky
* test for specific entities
* test for specific entities
* test for specific entities
* test for specific entities
* test for specific entities
* cover
* cover
* more cover
* tweak
* make sure it works before history starts
* fix test
* cover
* tweak
* make sure we unsub on overflow
* Update homeassistant/components/history/__init__.py
* Update homeassistant/components/history/__init__.py
* fix race in test
* fix db executor access
* relo above task creation
* Initial orjson support take 2
Still need to work out problem building wheels
--
Redux of #72754 / #32153 Now possible since the following is solved:
ijl/orjson#220 (comment)
This implements orjson where we use our default encoder. This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently. If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).
Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)
Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups). I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes
If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`
Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">
* use for views as well
* handle UnicodeEncodeError
* tweak
* DRY
* DRY
* not needed
* fix tests
* Update tests/components/http/test_view.py
* Update tests/components/http/test_view.py
* black
* templates
* Add WS API for listing available statistic ids
* Update homeassistant/components/history/__init__.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Bram Kragten <mail@bramkragten.nl>
If LazyState cannot parse the attributes of its row as JSON, it prints
a message to the logger. Unfortunately, it passes `self` as a format
argument to that message, which causes its `__repr__` method to be
called, which then tries to retrieve `self.attributes` in order to
display them. This leads to an infinite recursion and a crash of the
entire core.
To fix, send the database row to be printed in the log message, rather
than the LazyState object that wraps around it.
J. Nick Koston
Erik Montnemery
epenet
Franck Nijhof
Pawel
Bram Kragten
Marc Mueller
Ville Skyttä
Jaroslav Hanslík
Joakim Sørensen
Paulus Schoutsen
Justin Paupore
Philip Allgaier
