* Move logbook continuous domain filtering to sql
sensors tend to generate a significant amount of states
that are filtered out by logbook. In testing 75% of
states can be filtered away in sql to avoid the
sqlalchemy ORM overhead of creating objects that will
be discarded.
* remove un-needed nesting
* Added GLOB capability to entityfilter and every place that uses it. All existing tests are passing
* added tests for components affected by glob change
* fixed flake8 error
* mocking the correct listener
* mocking correct bus method in azure test
* tests passing in 3.7 and 3.8
* fixed formatting issue from rebase/conflict
* Checking against glob patterns in more performant way
* perf improvments and reverted unnecessarily adjusted tests
* added new benchmark test around filters
* no longer using get with default in entityfilter
* changed filter name and removed logbook from filter benchmark
* simplified benchmark tests from feedback
* fixed apache tests and returned include exclude schemas to normal
* fixed azure event hub tests to properly go through component logic
* fixed azure test and clean up for other tests
* renaming test files to match standard
* merged mqtt statestream test changes with base
* removed dependency on recorder filter schema from history
* fixed recorder tests after merge and a bunch of lint errors
Cleanup indexes as >50% of the db size was indexes,
many of them unused in any current query
Logbook search was having to filter event_types without
an index:
Created ix_events_event_type_time_fired
Dropped ix_events_event_type
States had a redundant keys on composite index:
Dropped ix_states_entity_id
Its unused since we have ix_states_entity_id_last_updated
De-duplicate storage of context in states as
its always stored in events and can be found
by joining the state on the event_id.
Dropped ix_states_context_id
Dropped ix_states_context_parent_id
Dropped ix_states_context_user_id
After schema v9:
STATES............................................ 10186 40.9%
EVENTS............................................ 5502 22.1%
IX_STATES_ENTITY_ID_LAST_UPDATED.................. 2177 8.7%
IX_EVENTS_EVENT_TYPE_TIME_FIRED................... 1910 7.7%
IX_EVENTS_CONTEXT_ID.............................. 1592 6.4%
IX_EVENTS_TIME_FIRED.............................. 1383 5.6%
IX_STATES_LAST_UPDATED............................ 1079 4.3%
IX_STATES_EVENT_ID................................ 375 1.5%
IX_EVENTS_CONTEXT_PARENT_ID....................... 347 1.4%
IX_EVENTS_CONTEXT_USER_ID......................... 346 1.4%
IX_RECORDER_RUNS_START_END........................ 1 0.004%
RECORDER_RUNS..................................... 1 0.004%
SCHEMA_CHANGES.................................... 1 0.004%
SQLITE_MASTER..................................... 1 0.004%
* adj
* time_fired_isoformat
* remove unused code
* tests for processing timestamps
* restore missing import lost in merge conflict
* test for None case
* Squeezebox add config flow and player discovery
* Fixes to config flow
* Unavailable player detection and recovery
* Improved error message for auth failure
* Testing for squeezebox config flow
* Import configuration.yaml
* Support for discovery integration
* Internal server discovery
* Fix bug restoring previously detected squeezebox player
* Tests for user and edit steps in config flow
* Tests for import config flow
* Additional config flow tests and fixes
* Linter fixes
* Check that players are found before iterating them
* Remove noisy logger message
* Update requirements_all after rebase
* Use asyncio.Event in discovery task
* Use common keys in strings.json
* Bump pysqueezebox to v0.2.2 for fixed server discovery using python3.7
* Bump pysqueezebox version to v0.2.3
* Don't trap AbortFlow exception
Co-authored-by: J. Nick Koston <nick@koston.org>
* Refactor validate_input
* Update squeezebox tests
* Build data flow schema using function
* Fix linter error
* Updated en.json
* Update homeassistant/components/squeezebox/media_player.py
Co-authored-by: J. Nick Koston <nick@koston.org>
* Update homeassistant/components/squeezebox/media_player.py
Co-authored-by: J. Nick Koston <nick@koston.org>
* Update homeassistant/components/squeezebox/media_player.py
Co-authored-by: J. Nick Koston <nick@koston.org>
* Update homeassistant/components/squeezebox/media_player.py
Co-authored-by: J. Nick Koston <nick@koston.org>
* Update .coveragerc for squeezebox config flow test
* Mock TIMEOUT for faster testing
* More schema de-duplication and testing improvements
* Apply suggestions from code review
Co-authored-by: J. Nick Koston <nick@koston.org>
* Testing and config flow improvements
* Remove unused exceptions
* Remove deprecated logger message
* Update homeassistant/components/squeezebox/media_player.py
Co-authored-by: J. Nick Koston <nick@koston.org>
* Implement suggestions from code review
* Add async_unload_entry
* Use MockConfigEntry in squeezebox tests
* Remove unnecessary config schema
* Stop server discovery task when last config entry unloaded
* Improvements to async_unload_entry
* Fix bug in _discovery arguments
* Do not await server discovery in async_setup_entry
* Do not await start server discovery in async_setup
* Do not start server discovery from async_setup_entry until homeassistant running
* Re-detect players when server removed and re-added without restart
* Use entry.entry_id instead of unique_id
* Update unittests to avoid patching homeassistant code
Co-authored-by: J. Nick Koston <nick@koston.org>
* Refactor / update Awair integration
This commit does a few things, all in service of making the Awair
integration more modern and reliable. Specifically we do the following:
- Update to python_awair 0.1.1
- Begin using config entries / flow for setting up the integration.
- YAML support is completely removed.
- The integration now allows adding multiple Awair accounts, should a
user wish to do so (I found it _very_ useful in development).
- Group various Awair sensors into devices, using the device registry.
- Renames various sensors and treats the "dust" sensor as a particulate sensor.
- Device update rate-limits are no longer dynamically calculated; the
Awair API now separates rate-limits on a per-device basis.
- Supports sound pressure and illuminance sensors found on some Awair devices.
- We report the "awair index" for certain sensors as part of device_state_attributes.
The "index" is a subjective measure of whether or not a sensor reading
is "good" or "bad" (and to what extent). It's a component of the Awair
score, and it is useful on its own as an input for those who wish to
do things like "display this color if the value is 'bad'".
This is a breaking change, and requires updates to documentation and a
warning in the README. The breaking changes in detail, are:
- Support for all YAML configuration is removed, and users will need to
re-add the integration via the UI.
- We no longer support overriding device discovery via manual
configuration of device UUIDs. This was previously supported because
the Awair API had severe limits on the device list endpoints; however
those have since been removed.
- Gen 1 devices no longer show a "dust" sensor; rather we create a PM2.5
sensor and a PM10 sensor and just keep the values in sync. This better
reflects the sensor capabilities: it can detect particles in a range
from 2.5 -> 10, but cannot differentiate between sizes.
- Sensors are renamed as follows:
- "sensor.devicename_co2" -> "sensor.devicename_carbon_dioxide"
- "sensor.devicename_voc" -> "sensor.devicename_volatile_organic_compounds"
- "sensor.devicename_score" -> "sensor.devicename_air_quality_index"
- I've chosen to call the "Awair Score" an "air quality index" sensor,
because fundamentally the "Awair Score" and other air quality indices
(such as CAQI) do the same thing: they calculate a value based on a
variety of other inputs.
Under the hood, the integration has seen some improvements:
- We use the DataUpdateCoordinator class to handle updates, rather than
rolling our own update class.
- The code no longer tracks availability based on a timestamp returned
from the Awair service; we assert that if we have received a response
and the response has data for our device, then we are available (and
otherwise, not available). We don't need to test the actual Awair API
so heavily.
- Test coverage has been expanded to handle a variety of products that
Awair produces, not just the one I happen to own.
- Test coverage no longer concerns itself with testing behavior that is
now handled by the DataUpdateCoordinator; nor is it concerned with
ensuring that the overall component sets up and registers properly.
These are assumed to be well-tested functionaity of the core and not
things we need to re-test ourselves.
Finally - between library updates and integration updates, this
integration is well-positioned to support future updates. I have a
proof-of-concept patch for device automations, and the underlying
library now supports subclassing authentication (which clears the way
for us to use OAuth authentication for Awair).
* Wrap test fixture in mock_coro
Truthfully I'm not sure why this was passing on my local dev
environment, but I was developing with python 3.8 before. After
installing python 3.7, I was able to reproduce the CI failures and fix
them.
* Fix broken tests after #34901 and/or #34989
* Do not rename sensors so broadly
We're going to keep the sensors named as they were before, pending the
outcome of any decisions around the air_quality component and what names
should be standardized for air-quality-like devices.
If standardized names are selected (which does seem likely), then we
will update this integration to match them - at which point, it would be
a breaking change.
But for now, we'll keep names mostly identical to what users had before.
Notable in this commit is that we generate the entity_id ourselves,
rather than just allowing it to be auto-generated from the name
attribute. This allows us to provide more human friendly names, while
keeping the old format for entity ids. For example, given an Awair
device called "Living Room", we'll generate an entity id of
"sensor.living_room_voc" but show set the name of the device to "Living
Room Volatile organic compounds".
* Support import from config.yaml
We'll create a config entry from config.yaml the first time we're
loaded, and then defer to it from then on.
We ignore all keys other than the access_token, since we no longer need
to deal with per-account rate-limits (rather, everything is per-device
now).
* Add myself to CODEOWNERS
Since I wrote the initial integration, and now this re-write, it feels
appropriate for me to take care of the integration along with `danielsjf`.
* Remove name mangling
* Update homeassistant/components/awair/manifest.json
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/awair/config_flow.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/awair/sensor.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Update homeassistant/components/awair/sensor.py
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Address some review feedback
* Set up reauth flow in a job, rather than awaiting
* Remove unnecessary title string
* Remove unnecessary config schema checking
As pointed out in review, because this comes in via import from
`configuration.yaml`, we can rely on the `PLATFORM_SCHEMA` validation instead.
* Fix tests
* Set unique_id appropriately for legacy devices
For users who have had this integration already installed (and who have
updated their home assistant installation sometime in recent history),
we want to ensure that unique_id's are set to the same thing as before,
to facilitate the upgrade process.
To do that, we add an additional property to the `SENSOR_TYPES` dict
(`ATTR_UNIQUE_ID`) which allows us to map modern sensor names from
python_awair to what older versions called them - ie: `humidity` ->
`HUMID`. We then use that value when constructing the unique ID. This
should allow users to upgrade and not lose configuration even if entity
IDs would otherwise change (because we have changed the name format that
generates entity IDs).
One note is that for the gen1 `DUST` sensor, we have to treat it
differently. This integration used to call that a "PM2.5" sensor, but
the unique_id generated would be something like `awair_12345_DUST`. So
we special-case that sensor, and do the same thing. We do not need to
special-case the PM10 sensor for gen1 devices, because we didn't create
a PM10 sensor in the past (we do now, because the "DUST" sensor is
really a hybrid PM2.5/PM10 sensor).
* Patch async_setup_entry for two tests
* Update awair config_flow to require / use an email address for unique_id
Also, only start one re-auth flow.
* Add a few more tests, try to get coverage up.
* Add another test
* Move attribution to device_state_attributes
* Don't require email
* Switch from Union[dict, None] to Optional[dict]
* Use a mock where requested
* Fix missing constant rename
* Use async_create_task
* Bump test coverage a bit for config_flow
* s/CONF_UNIQUE_ID/UNIQUE_ID/g
* Add warning about deprecated platform config
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
* Prefilter more logbook events in sql
Prefilter sensor events in _keep_event before humanify
Cache static attribute lookup
Reduces logbook execution time by ~35%
* fix mocking in benchmark
* Update tests for logbook users
* TTS seems unsupported on heos media player #32862
TTS seems unsupported on heos media player #32862
The type media_type music which is required by TTS was not covered.
* Update homeassistant/components/heos/media_player.py
Co-authored-by: Andrew Sayre <6730289+andrewsayre@users.noreply.github.com>
* Update to test_media_player.py
Test for TTS support to Heos #35386
* Update test
Add TTS support to Heos #35386
* Update to test_play_media_music
assert set to "Unable to play music: Failure (1)" seems to cause an issue.
* test_play_media_music
syntax
Co-authored-by: Andrew Sayre <6730289+andrewsayre@users.noreply.github.com>
* Allow activity change on start of switch
Allow activity to be updated when a switch to a new activity is initiated instead of when it is completed.
* Updates based on feedback
Some items are not required to be done as YAML is not used anymore.
Cleaned-up some code.
* Fix for change on how to set callbacks
How callbacks are set now one has to set the new_activity and new_activity_starting as well, even just with None.
* Added callback update
Added so that when it is changed in the UI the callbacks will be changed as well.
* Added test cases for notify setting
Added test cases for config flow to test new setting for activity notifications.
* Add discovery to NUT integration
* implement async_step_zeroconf
* Update test to make sure unique id not set
* Remove host/port import when coming from discovery, add title placeholders
* fix mis-paste
Co-authored-by: J. Nick Koston <nick@koston.org>
* Add expiration timestamp to cert_expiry sensors
* Clear timestamp if cert becomes invalid
* Use timezone-aware timestamps
* Use DataUpdateCoordinator, split timestamp to separate sensor
* Use UTC, simpler add/remove handling
* Review fixes
* Fix incomplete mock that fails in 3.8
* Use static timestamps, improve helper method name
* Add old_state_id to states, remove old/new state data from events since it can now be found by a join
* remove state lookup on restart
* Ensure old_state is set for exisitng states
* Add precipitation probability attribute
* bump env_canada version to 0.0.38 in manifest
* bump env_canada version to 0.0.38 in requirements
* Add support for percipiation probability
* formated project with black
* make sure prob is an int
* fix build break
* update usage to correct naming
* revert bad change and apply fix build break
* add tests
* Only process logbook timestamps for events we will keep
Since we group by minute we were previously processing
every timestamp. We can avoid this by making all the
minute checks use the unprocessed datetime since
the groupings will be the same regardless of timezone.
This reduces the number of datetime object recreations
by at least an order of magnitude.
* Vera now polls for all status data, no only incremental.
Vera polling is not handled using hass event loops with proper backoffs.
* Using long polling.
* Addressing PR feedback.
* Addressing PR feedback.
Adding controller stop on config unload.
* Add config_flow to the dunehd integration
* Add tests
* Run gen_requirements_all
* Fix pylint error
* Better hostname validation
* Build device info in the class
* Remove zeroconf options from homekit
homekit uses the system shared zeroconf instance which
made the interface choice option controlled by the
zeroconf integration setting.
* change to cv.deprecated
* adj
* fix remaining tests from original merge conflict
* remove invalidation_version
* Part 1 of 2 (no breaking changes in part 1).
When integrations configured via the UI block startup or fail to start,
the webserver can remain offline which make it is impossible
to recover without manually changing files in
.storage since the UI is not available.
This change is the foundation that part 2 will build on
and enable a listener to start the webserver when the frontend
is finished loading.
Frontend Changes (home-assistant/frontend#6068)
* Address review comments
* bump timeout to 1800s, adjust comment
* bump timeout to 4h
* remove timeout failsafe
* and the test
* Support stream profile and configuring it through options flow
* Options flow test
* Allow configuration of not using a stream profile
* Shorten default stream profile string
Franck Nijhof
J. Nick Koston
jjlawren
Indu Prakash
Paulus Schoutsen
Aaron Bach
uvjustin
mdegat01
Shulyaka
Hedgehog57
Erik Montnemery
Chris Talkington
Thomas Hollstegge
Martin Hjelmare
Xiaonan Shen
Chris
Jonas Thuresson
zvldz
Robert Van Gorkom
rajlaud
starkillerOG
Andrew Hayworth
Fredrik Erlandsson
avocadio
Kit Klein
ehendrix23
Robert Svensson
bsmappee
marawan31
Kevin Fronczak
etheralm
Zac West
Ian Harcombe
Tom Schneider
Alexei Chetroi
MarBra
Janitha Karunaratne
Rami Mosleh
SukramJ
mreiling
StevusPrimus
Jeff Irion
definitio
Joakim Plate
Tim van Cann
Thomas Lovén
Andre Lengwenus
Austin Mroczek
Maciej Bieniek
Boris Kaplounovsky
Markus Bong
Brynley McDonald