* Core POC support for polymer i18n
* Add gulp to build_frontend
* Remove frontend build
* Updated translations format
* Eliminate translation namespace from panel names
* Only register translations path in dev mode
* Add support for multi-entity recent fetch of history. Add graph component
* Rename graph to history_graph. Support fast fetch without current state.
* Address comments
* Add DEBUG-level log for db row to native object conversion
This is now the bottleneck (by a large margin) for big history queries, so I'm leaving this log feature in to help diagnose users with a slow history page
* Rewrite of the "first synthetic datapoint" query for multiple entities
The old method was written in a manner that prevented an index from being used in the inner-most GROUP BY statement, causing massive performance issues especially when querying for a large time period.
The new query does have one material change that will cause it to return different results than before: instead of using max(state_id) to get the latest entry, we now get the max(last_updated). This is more appropriate (primary key should not be assumed to be in order of event firing) and allows an index to be used on the inner-most query. I added another JOIN layer to account for cases where there are two entries on the exact same `last_created` for a given entity. In this case we do use `state_id` as a tiebreaker.
For performance reasons the domain filters were also moved to the outermost query, as it's way more efficient to do it there than on the innermost query as before (due to indexing with GROUP BY problems)
The result is a query that only needs to do a filesort on the final result set, which will only be as many rows as there are entities.
* Remove the ORDER BY entity_id when fetching states, and add logging
Having this ORDER BY in the query prevents it from using an index due to the range filter, so it has been removed.
We already do a `groupby` in the `states_to_json` method which accomplishes exactly what the ORDER BY in the query was trying to do anyway, so this change causes no functional difference.
Also added DEBUG-level logging to allow diagnosing a user's slow history page.
* Add DEBUG-level logging for the synthetic-first-datapoint query
For diagnosing a user's slow history page
* Missed a couple instances of `created` that should be `last_updated`
* Remove `entity_id` sorting from state_changes; match significant_update
This is the same change as 09b3498f41 , but applied to the `state_changes_during_period` method which I missed before. This should give the same performance boost to the history sensor component!
* Bugfix in History query used for History Sensor
The date filter was using a different column for the upper and lower bounds. It would work, but it would be slow!
* Update Recorder purge script to use more appropriate columns
Two reasons: 1. the `created` column's meaning is fairly arbitrary and does not represent when an event or state change actually ocurred. It seems more correct to purge based on the event date than the time the database row was written.
2. The new columns are indexed, which will speed up this purge script by orders of magnitude
* Updating db model to match new query optimizations
A few things here: 1. New schema version with a new index and several removed indexes
2. A new method in the migration script to drop old indexes
3. Added an INFO-level log message when a new index will be added, as this can take quite some time on a Raspberry Pi
* Bugfix: remove superfluous domain filter
This filter is already applied later in the function by the `filters` object, where it is conditionally applied when appropriate. This fixes the problem where we get a domain filter even when searching for a single entity_id, which needlessly harms the query's performance.
* Performance: build different query when only getting single entity
When querying the history of a single entity, we can use an entirely different method for the "synthetic zero data point" by simply sorting by date and doing a LIMIT 1. This performs thousands of times better than the multi-entity query when the current recorder_run has been going for a while.
* Add entity_id filter to single-entity request
The entity_id filter was handled inside the `filters.apply` logic which is used in most cases, BUT didn't work when no `filters` was passed in to the method. Now it'll work even if no `filters` object is passed in.
* Fix linting errors in history.py
* Undo removal of domain filter
Putting back the domain filter that was removed in 76a6371705 - there are use-cases where get_states is called without a filter object, so we need the domain filter to work in those cases as well.
* Fix truncated comment
* Remove setting up an hbmqtt broker
* Don't pass loop to web.Application in tests
* Use .query instead of deprecated .GET for aiohttp requests
* Fix closing file resource
* Do not use asyncio mark
* Notify.html5 - PyJWT: Use options to disable verify
* Yamaha: Test was still using deprecated ip
* Remove pytest-asyncio
* Restore states
* feedback
* Remove component move into recorder
* space
* helper
* Address my own comments
* Improve test coverage
* Add test for light restore state
* Move HTTP to own folder
* Break HTTP into middlewares
* Lint
* Split tests per middleware
* Clean up HTTP tests
* Make HomeAssistantViews more stateless
* Lint
* Make HTTP setup async
* added include and exclude functionality to history component
* fixed summary lines in test method doc.
* cleanup of query filter creation
* o improved config validation
o move move IGNORE_DOMAINS to Filter.apply()
o removed config from Last5StatesView
o Filters instance is now created on setup
o config values are processed in setup and set to the Filters instance
o function _set_filters_in_query() moved to Filters class and renamed to apply()
* added unittests for more include/exclude filter combinations
* make pylint happy
* Use local timezone for log and history dates
* home-assistant-js fix
* Submodule updates not included so travis can build
* Separate Date and DateTime http validators
* Include submodule reference
* Update frontend
* Switch to SQLAlchemy for the Recorder component. Gives the ability to use MySQL or other.
* fixes for failed lint
* add conversion script
* code review fixes and refactor to use to_native() model methods and execute() helper
* move script to homeassistant.scripts module
* style fixes my tox lint/flake8 missed
* move exclusion up
This adds a new function to history module which returns significant
states. For most domains this is the list of state changes. For the
thermostat domain this also includes attribute changes, so that
changes in the current_temperature are exposed to the graphing layer.
Closes#881
Adam Mills
Paulus Schoutsen
milanvo
Alan Fischer
Andrey
Greg Laabs
Fabian Affolter
Johann Kellerman
Matthew Garrett
wokar
rhooper
Fabian Affolter
Sean Dague
Jeff Schroeder
jamespcole