* Fix memory churn in state templates
The LRU for state templates was limited to 512 states. As soon
as it was exaused, system performance would tank as each template
that iterated all states would have to create and GC any state
> 512
* does it scale?
* avoid copy on all
* comment
* preen
* cover
* cover
* comments
* comments
* comments
* preen
* preen
* Adds a loader to enable jinja imports.
* Switch to in-memory
* Move loading custom_jinja off of the event loop
* Raise TemplateNotFound if template doesn't exist
* Fix docstring
* Adds a service to reload custom jinja
* Remove IO from test setup
* Improve coverage and small refactor
* Incorporate feedback and use .jinja extension
* Check the loaded sources in test.
* Incorporate PR feedback.
* Update homeassistant/helpers/template.py
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
When a template render raised an exception we would start listening for
all states until the template did not raise an exception anymore. This
was not needed since the entity that is causing the exception was
already in the tracker.
Re-rendering on all state changes can be extremely expensive and can
bring an instance into a sluggish or unresponsive state when updating
from a much older version that did not raise ValueError when a default
was missing.
* Add type to template hassfunction decorator
* Adjust to use EvalContext
* Use runtime.Context
* Use TypeVar for context
* Use jinja2.runtime.Context
* Reverse declarations
* Use Any
* Update homeassistant/helpers/template.py
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
* Speed up generation of template states
* tweak
* cache
* cache hash
* weaken
* Revert "weaken"
This reverts commit 4856f50080.
* lower cache size as it tends to be the same ones over and over
* lower cache size as it tends to be the same ones over and over
* lower cache size as it tends to be the same ones over and over
* cover
* Update homeassistant/helpers/template.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* id reuse is possible
* account for iterting all sensors
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
- These were using orjson directly, its a bit cleaner
to use the helper so everything is easier to adjust
in the future if we need to change anything about
the loading
* Initial orjson support take 2
Still need to work out problem building wheels
--
Redux of #72754 / #32153 Now possible since the following is solved:
ijl/orjson#220 (comment)
This implements orjson where we use our default encoder. This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently. If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).
Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)
Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups). I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes
If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`
Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">
* use for views as well
* handle UnicodeEncodeError
* tweak
* DRY
* DRY
* not needed
* fix tests
* Update tests/components/http/test_view.py
* Update tests/components/http/test_view.py
* black
* templates
Erik Montnemery
J. Nick Koston
Petro31
ehendrix23
David Poll
Lucas Mindêllo de Andrade
epenet
Franck Nijhof
Vaarlion
Marc Mueller
Michaël Arnauts
Mike Degatano
On Freund
Jan Bouwhuis
Rami Mosleh
Joakim Sørensen
[pʲɵs]
Raphaël Beamonte
