Handle cancelation in wait_for_ble_connections_free
If `wait_for_ble_connections_free` was canceled due to timeout or
the esp disconnecting from Home Assistant the future would get
canceled. When we reconnect and get the next callback we need
to handle it being done.
fixes
```
2023-03-21 02:34:36.876 ERROR (MainThread) [homeassistant] Error doing job: Fatal error: protocol.data_received() call failed.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 868, in _read_ready__data_received
self._protocol.data_received(data)
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/_frame_helper.py", line 195, in data_received
self._callback_packet(msg_type_int, bytes(packet_data))
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/_frame_helper.py", line 110, in _callback_packet
self._on_pkt(Packet(type_, data))
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/connection.py", line 688, in _process_packet
handler(msg)
File "/usr/local/lib/python3.10/site-packages/aioesphomeapi/client.py", line 482, in on_msg
on_bluetooth_connections_free_update(resp.free, resp.limit)
File "/usr/src/homeassistant/homeassistant/components/esphome/entry_data.py", line 136, in async_update_ble_connection_limits
fut.set_result(free)
asyncio.exceptions.InvalidStateError: invalid state
```
* Adds base code for matter lock
* Adds basic matter door lock support
* Adds matter lock fixture
* Adds tests for matter lock
* Addresses feedback
* Added logic to handle inter states of matter lock
* Addesses feedback
* Introduce a delay between update entity calls
* Update homeassistant/components/zwave_js/update.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* move delay to constant and patch
* rename constant
* Switch to async_call_later
* Remove failing test
* Reimplement to solve task problem
* comment
* pass count directly so that value doesn't mutate before we store it
* lines
* Fix logic and tests
* Comments
* Readd missed coverage
* Add test for delays
* cleanup
* Fix async_added_to_hass logic
* flip conditional
* Store firmware info in extra data so we can restore it along with latest version
* Comment
* comment
* Add test for is_running check and fix bugs
* comment
* Add tests for various restore state scenarios
* move comment so it's less confusing
* improve typing
* consolidate into constant and remove unused one
* Update update.py
* update test to unknown state during partial restore
* fix elif check
* Fix type
* clean up test docstrings and function names
---------
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
If a user manually migrated their database to MySQL or PostgresSQL
and incorrectly created the timestamp columns as float we would
fail to correct them to double because when we migrated to use
timestamps for the columns I missed that we needed to change the
columns and types for µs precision
- If the user had previously duplicated data we could end up
picking the next metadata_id and there could be stale rows
in the database that have that metadata_id. This can only happen
from bad manual migrations (which is what this is function
is validating in the first place). To solve this we now insert
data with a future date and look at the latest inserted row
instead of the first.
Example
```
['stored_statistics',
defaultdict(<class 'list'>,
{'recorder.db_test_schema': [{'end': 948589200.0,
'last_reset': None,
'max': None,
'mean': 2021.0,
'min': None,
'start': 948585600.0,
'state': None,
'sum': 394.5068},
{'end': 1601946000.000001,
'last_reset': 1601942400.000001,
'max': 1.000000000000001,
'mean': 1.000000000000001,
'min': 1.000000000000001,
'start': 1601942400.000001,
'state': 1.000000000000001,
'sum': 1.000000000000001}]})]
```
Update the calander event trigger logic to have more exhaustive coverage. The
trigger will now use a timespan to create an explicit window for considering
upcoming events. The start/end of the time span is now more explicit, rather
than getting it from the alarm time.
The trigger is now broken into composable pieces:
- A timespan object for more explicitly managing the time window
- A function to get events during a time span
- A function to process upcoming events and determine the trigger times
The existing listener is now just responsible for scheduling alarms and glue.
This fixes bug with DST handling where the conversion back and forth between
UTC and timezone ends up dropping events during the jump forward. In practice,
an event was returned from the scanning, but it was never fired by the trigger
because (1) it was filtered out of the interval and (2) the event list was
previously cleared every iteration so it would get dropped.
Future improvements can bake more invariant checking into this structure.
* shield Reolink webhook callback from cancelation
* Update homeassistant/components/reolink/host.py
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
* fix styling
* fix black
* Revert to using asyncio.shield
---------
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Set unique on StatesMeta and EventTypes
These should have been marked unique originally to prevent
collision bugs from going unnoticed. These have not been
to beta yet so this is not a breaking change
* Fix cpu thrashing during purge after all legacy events were removed
We now remove the the index of of event ids on the states table when its
all NULLs to save space. The purge path needs to avoid checking for legacy
rows to purge if the index has been removed since it will result in a full
table scan each purge cycle that will always find no legacy rows to purge
* one more place
* drop the key constraint as well
* fixes
* more sqlite
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* remove exception catch since the threading.excepthook will actually catch this in production
* fix a few missed ones
* threadsafe
* Update homeassistant/components/recorder/table_managers/statistics_meta.py
* coverage and optimistic caching
* Add state attribute translations for media players
* Process review comments
* Process review comments
* Fix and extend
* Add yes/no as generic state
* Warn if the previous push wait task it taking longer than the update interval
* refactor
* Call _async_fetch_number_of_messages first
* Add cleanup in case fetching fails
* mypy
* Set sensor to unknown if an error occured.
* Handling invalid auth an reraise when needed
* Handle invalid folder as setup error
* Close IMAP stream before logout at cleanup
---------
Co-authored-by: J. Nick Koston <nick@koston.org>
* Add local calendar diagnostics platform
* Use redaction from ical
* Update diagnostics for new ical version
* Apply suggestions from code review
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
* Use snapshot tests for local calendar diagnostics
* Setup diagnostics directly in tests rather than via dependencies
---------
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
* Initial commit
* Add websocket test tool
* Small tweak
* Tiny cleanup
* Make pipeline work with frontend branch
* Add some more info to start event
* Fixes
* First voice assistant tests
* Remove run_task
* Clean up for PR
* Add config_flow.py
* Remove CLI tool
* Simplify by removing stt/tts for now
* Clean up and fix tests
* More clean up and API changes
* Add quality_scale
* Remove data from run-finish
* Use StrEnum backport
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add filters to climate and light service descriptions
* Allow specifying enums directly
* Update service descriptions
* Adjust test
* Cache entity features
* Lint
* Improve error handling, add list of known base components
* Don't allow specifying an entity feature as int
* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* reduce
* comments
* Debounce and group mqtt subscriptions
* Cleanup
* Do not cooldown on resubscribe
* Remove lock from task
Co-authored-by: Erik Montnemery <erik@montnemery.com>
* ruff
* Longer initial cool down. Manages unsubscribes
* Own lock for access to self._pending_subscriptions
* adjust
* Subscribe to highest QoS when sharing subscription
* do not block _pending_subscriptions_lock with io
* Test the highest qos is subscribed at
* Cleanup max qos
* Follow up comments part 1
* Make docstr more generic
* Make max qos update thread safe
* Add lock on clearing _max_qos when resubscribing
* Wait for linger task
* User copy
* Check for key before cleaning up
* Fix lingering task
* Do not use a lock
* do not await _async_queue_subscriptions
* Replace copy with assignment
* Update max qos before returning
* Do not iterate if max_qos == 0
* Do not ieterate subs if max qos == 0
* Set initial cooldown correctly
* Ensure discovery cooldown ends after subscribing
* plan last subscribe with debouncer timeout
* cooldown if self._pending_subscriptions is set
* Revert format changes
* Remove stale assingnment self._last_subscribe
* Remove not used property
* Also check while for pending subscriptions
* revert first added sleep()
* Optimize
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Co-authored-by: J. Nick Koston <nick@koston.org>
* Adds a loader to enable jinja imports.
* Switch to in-memory
* Move loading custom_jinja off of the event loop
* Raise TemplateNotFound if template doesn't exist
* Fix docstring
* Adds a service to reload custom jinja
* Remove IO from test setup
* Improve coverage and small refactor
* Incorporate feedback and use .jinja extension
* Check the loaded sources in test.
* Incorporate PR feedback.
* Update homeassistant/helpers/template.py
Co-authored-by: Erik Montnemery <erik@montnemery.com>
---------
Co-authored-by: Erik Montnemery <erik@montnemery.com>
Drop duplicated indices from schema
https://docs.percona.com/percona-toolkit/pt-duplicate-key-checker.html
```
% pt-duplicate-key-checker --databases fresh
ALTER TABLE `fresh`.`events` DROP INDEX `ix_events_event_type_id`;
ALTER TABLE `fresh`.`states` DROP INDEX `ix_states_metadata_id`;
ALTER TABLE `fresh`.`statistics` DROP INDEX `ix_statistics_metadata_id`;
ALTER TABLE `fresh`.`statistics_short_term` DROP INDEX `ix_statistics_short_term_metadata_id`;
```
* Switch history stats to report in seconds
Because hours were previously used, the data would always be off because
of the loss of resolution when the time being tracked was in a window
of more than 12s
* Apply suggestions from code review
* Update homeassistant/components/history_stats/sensor.py
* tweak
* Split context id migration into states and events tasks
Since events can finish much earlier than states we
would keep looking at the table because states as not
done. Make them seperate tasks
* add retry dec
* fix migration happening twice
* another case
epenet
jan iversen
Franck Nijhof
Allen Porter
Tom Harris
J. Nick Koston
Arturo
Raman Gupta
Jon Caruana
Jan Bouwhuis
Steven Looman
Malte Franken
Joakim Plate
Michael
micha91
Jesse Moody
starkillerOG
Matthias Alphart
Oliver
Vincent Knoop Pathuis
Kevin Worrel
Martin Hjelmare
lunmay
Jean-François Roy
Erik Montnemery
Michael Hansen
Bram Kragten
Kevin Siml
Jack Boswell
zhangshengdong29
Marcel van der Veldt
Maciej Bieniek
PatrickGlesner
StefanIacobLivisi
Penny Wood
Marcio Granzotto Rodrigues
Ernst Klamer
Aidan Timson
amitfin
G Johansson
puddly
Mick Vleeshouwer
Paulus Schoutsen
David Poll
Nick Borgers
Joakim Sørensen
tomrennen
Thijs Walcarius
Klaas Schoute
anotherthomas
Arjan
Jan Rieger
cnico
dougiteixeira
Eugenio Panadero
Stephan Uhle
Barry Loong