* Disable cleanup_closed for aiohttp.TCPConnector with cpython 3.11.2+
There is currently a relatively fast memory leak when using
cpython 3.11.2+ and cleanup_closed with aiohttp
For my production instance it was leaking ~450MiB per day
of `MemoryBIO`, `SSLProtocol`, `SSLObject`, `_SSLProtocolTransport`
`memoryview`, and `managedbuffer` objects
see https://github.com/aio-libs/aiohttp/issues/7252
see https://github.com/python/cpython/pull/98540
* Update homeassistant/helpers/aiohttp_client.py
* Initial orjson support take 2
Still need to work out problem building wheels
--
Redux of #72754 / #32153 Now possible since the following is solved:
ijl/orjson#220 (comment)
This implements orjson where we use our default encoder. This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently. If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).
Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)
Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups). I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes
If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`
Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">
* use for views as well
* handle UnicodeEncodeError
* tweak
* DRY
* DRY
* not needed
* fix tests
* Update tests/components/http/test_view.py
* Update tests/components/http/test_view.py
* black
* templates
* Patch aiohttp client session close
* Add test
* Restore close regardless of auto_cleanup
* Close session instead of detaching and do not restore
* Delint test
* Add frame helper
* Use frame helper
* Test warning log when closing session
* Clean up
* Correct docstring
* Do not change shutdown
* Fix tests
* [recorder] Use orjson to parse json faster
* Remove from http manifest
* Bump to orjson 2.5.1
* Empty commit to trigger CI
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* Fix for network issues
Fix for network issues
* Further changes to network connection
* websession is created in pymyq
websession is created in pymyq instead.
Added call on stop event to close web session.
* Updated requirements file
* Added comment
* Changed back to use aiohttp_client
* Cleanup closed sockets in aiohttp
Enable automatic cleanup of closed sockets in aiohttp client helper.
* Updated manifest & requirements
* Updated comment block
J. Nick Koston
Marc Mueller
epenet
Franck Nijhof
Raman Gupta
Ruslan Sayfutdinov
Paulus Schoutsen
jlvaillant
Philip Allgaier
Martin Hjelmare
Bas Nijholt
Franck Nijhof
ehendrix23
Ville Skyttä
Paulus Schoutsen
Martin Mois
Pascal Vizeli
