* Initial orjson support take 2
Still need to work out problem building wheels
--
Redux of #72754 / #32153 Now possible since the following is solved:
ijl/orjson#220 (comment)
This implements orjson where we use our default encoder. This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently. If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).
Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)
Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups). I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes
If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`
Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">
* use for views as well
* handle UnicodeEncodeError
* tweak
* DRY
* DRY
* not needed
* fix tests
* Update tests/components/http/test_view.py
* Update tests/components/http/test_view.py
* black
* templates
* Add WS API for listing available statistic ids
* Update homeassistant/components/history/__init__.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Bram Kragten <mail@bramkragten.nl>
If LazyState cannot parse the attributes of its row as JSON, it prints
a message to the logger. Unfortunately, it passes `self` as a format
argument to that message, which causes its `__repr__` method to be
called, which then tries to retrieve `self.attributes` in order to
display them. This leads to an infinite recursion and a crash of the
entire core.
To fix, send the database row to be printed in the log message, rather
than the LazyState object that wraps around it.
* Prebake common history queries
The python overhead of to construct
the queries exceeded the database overhead. We now
prebake the queries that get frequently polled.
This reduces the time it takes to update history_stats
sensors and can make quite a difference if there
are a lot of them.
When using the mini-graph-card card, all the entities
on the card being graphed are queried every few seconds
for new states. Previously this would tie up the database if there
are lot of these graphs in the UI.
* Update homeassistant/components/history/__init__.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* Update homeassistant/components/history/__init__.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* cache entity_filter in the lambda
* switch to yield
* Revert "switch to yield"
This reverts commit f8386f4940.
* get_states always returns a list
* query wasnt actually reusable so revert part of the breakout
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Erik Montnemery
epenet
Franck Nijhof
J. Nick Koston
Pawel
Bram Kragten
Marc Mueller
Ville Skyttä
Jaroslav Hanslík
Joakim Sørensen
Paulus Schoutsen
Justin Paupore
Philip Allgaier
