- Create separate method for regisetering mycroft system event handlers
- Fix some string concatenations
- Group acknowledge together with speak and speak_dialog
This will recurse down to MycroftSkill class but not include it. for
most skills this will mean only the methods / variables added by the
skill creator
An exception in skill's initialize method would be reraised and cause
the main loading routine to halt. This does not reraise the exception,
instead the instance reference is removed.
Split skill_manager into three separate classes, SkillManager, SkillUpdater and SkillLoader splitting the responsibility into logical units
* Split the SkillManager.__init__ code to determine the download times into a new method
* Make docstrings consistent and PEP257 compliant. Also fixed a couple of spelling errors
* fixed two issues introduced in the previous refactoring
* removed unnecessary assignment of an instance attribute to a local variable
* updated the unit test to mock out code that reaches outside of core, like MSM and the configuration manager.
* add several unittests and refactored load_priority method.
* add a test for the _get_last_modified_date function.
* add "quick" argument to docstring
* removed unused import
* new class containing the logic to periodically update/install skills and send skill manifests to the backend.
* import MsmException from where it is defined, not from the skill manager.
* add some logging to the skill updater
* remove code now in SkillUpdater from SkillManager
* added imports to __init__.py to define the API into the message bus package
* new base class for unit tests and module for reusable mocks
* new skill loader class that will replace the _load_or_reload_skill() method in the SkillManager class.
* moved skill loading logic from core.py into skill_loader.py, resulting in some refactoring of skill loader and skill manager. change unit tests to match.
* added back some spacing that was inadvertently removed.
* change skill tester to use new SkillLoader class.
* Separate reload required check from performing reload to make logic easier
to follow
* Track skills that failed to load to handle infinite loop at first load
if skill fails to load
* Allow reloading skills that has failed to load
* Simplify first load of skills
- create activate, deactivate and unload methods for skill_loader
objects
- add sanity checks before activating and deactivating skills
- Update activation/deactivation test cases
In some instances, the backend returns number setting values as an integer rather than a string.
This has been seen in the wild but has been difficult to replicate. The circumstances under which it happens are still unclear. I was able to semi-consistently have a number returned as int from the [Severe Weather Skill](https://github.com/domcross/severe-weather-information-skill) when editing an unrelated setting. To test, remove the 3 character country code (`[A-Z]{3} - `) from the service options of this skill .
This seems to be the quickest fix for it, but worth investigating further from the backend.
This small change adds a check that the intent has been registered
before removing it.
If an unregistered intent was removed padatious would silently throw an
exception due to a list operation error. But when run synchronously from
the skill tester this silent exception was actually loud causing skills
to fail to load.
Major refactoring of the skills startup sequence
- Restructure to a less nested structure
- Remove usage of globals by wrapping a lot of state variables into a class this allows for things like caching a negative pairing status throughout the startup process
- MycroftSkill and related core functions to mycroft_skill.py
- FallbackSkill to fallback_skill.py
- Add important classes decorators and functions to __init__.py
- move SkillGUI class to the enclosure along with the other enclosure
interfaces
core.py retains the same members as previously by means of imports to
retain backwards compatibility. When most of the available skills starts
using this new structure properly
Message bus config loading is now shared by service and client.
messagebus.client.ws file is still available in case skills are using it. It is a backport that inherits from the new MessageBusClient class. Adds depreciation warning.
Checks if the skills haven't been updated for more than 2 weeks or if
the last_download is undefined. last_download may be undefined if
the .msm file is missing or if the list of installed requirements is
missing so skill requirements may be missing.
This also makes sure the skills are normally loaded before the an update
attemt is tried.
Upgrade msm to 0.7.8
Using the new max_threads parameters to MycroftSkillsManager.apply() the
skill loading is reduced to 2 threads if it's during runtime while a
fast update (20 threads) are used if not all default skills are
installed to speed up getting core into a 100% operational state.
The MycroftSkill.stop() method does not need to be implemented by the majority of skills, no reason to 'require' its implementation with an abstractmethod.
* Handle DelayRequests when uploading blank settingsmeta
* Move Api methods to DeviceApi
This includes delete, put and get skill settings/meta
* Cache skill settings for 30 seconds
The skill settings are cached to reduce number of requests to the backend and increase skill loading speed.
* Remove _request_other_settings method
The endpoint is not used anymore
* Fix issue when the settingsmeta fails to load
Remove an old reference to "BLANK_META" left behind in the exception handler for failing to load settingsmeta.
* Catch the new system.gui.user.interaction
This will in turn trigger the gui.page_interaction similar to the page flip but page_number will be None
* Fix issue when adding pages
Seems like appending would fail at times
* Add lock around namespace modifications
* Improve argument description for override_idle
* Log error when creating Padatious
* Update msm to v0.7.6
This handles connection errors during the startup procedure and improves
the skill_gid when no network is available
Padatious is registered as a FallbackSkill and was pushing settings to
the backend.
This change allows Padatious to flag to the MycroftSkill class
that no settings should be created. This is done via the new optional
MycroftSkill parameter use_settings.
Mycroft-core won't retry for 5 minutes on a 422, 500 or 501 error.
This adds a DelayRequests Exception which will cause the settings
fetching Timer thread to sleep for 5 minutes
The message returned is a dict of all available audio backends as keys
containing 'supported_uris' listing supported uri types (file:// http://
etc.), 'remote' (bool) True if remote), 'default' (bool) true if the
default audio backend.
Revert "Fix a couple of minor issues intruduced by skill_gid (#2079)"
This reverts commit e046377ce1.
Revert "Merge pull request #2075 from forslund/bugfix/msm_wrapper-license"
This reverts commit 18cfbce0ca, reversing
changes made to 82fa314ce9.
Revert "Feature/skillsmeta gid (#2074)"
This reverts commit 82fa314ce9.
* Fix skill_gid for modified skills
Used the old '_' after device uuid
* Identify skill using gid
When receiving data from server the "identifier" entry is now removed and in it's stead the skill_gid should be used.
This also removes the "fetch other settings" functionallity since that is no longer relevant.
* Add global id basics to settings meta
- All skills will upload a blank settingsmeta
- a skill_gid will be appended to all settingsmeta upload-data
- Added basic function for generating skill_gid
* Use new skill_gid field.
Populate skill_gid directly from metadata
* Separate travis tmp-dirs
- Update travis script to use tempdir for each python version
- Update test script to handle nonstandard tempdirs
- Generate msm folder using tempdir when running create_msm test
* Add title field with pretty name
* Collect and expand "title" as needed
For title use market-place title or name in settings meta or skillname
* Switch skill_manager create_msm test to 19.02
* Remove leading / trailing Skill in display name
Also rename title displayname to match new mycroft-skills-data
* Lock msm_create and mock the name info test_settings
If no match was found for the non-normalized utterance it would jump to
the exception handler for StopIteration skipping the normalized step
altogether
This changes the next/StopIteration system for list comprehensions
Previously Padatious intent matches were performed on non-normalized text, meaning that things like "what's the weather" wouldn't match a Padatious intent but
"what is the weather" would.
The "utterance" in Adapt intent data will still be non-normalized even if the intent match occurred on a normalized utterance. Retaining the existing behavior.
The "intent_failure" data. In there, "utterance" is always the raw version, "norm_utt" is the normalized one.
Also added better debugging info for intent matching to the log.
Also addresses a rare issue with the old code where the Adapt context could
have been updated even if the Adapt intent wasn't actually invoked due to
a higher Padatious intent match.
* Add the SkillGUI method send_event()
Sends raw mycroft.events.triggered messages to the gui for the skills namespace.
Example usage:
self.gui.send_event('event_name' {'param1': 12, 'param2': 'abc'})
The loose (conf > 0.5) Padatious match was previously occurring as Fallback
priority 99. The AIML fallback at priority 90 would consume lots of
utterances, interferring with many skills. Now Padatious runs at priority
89.
Additionally, added documentation of the intent and fallback system, including
guidelines for priorities.
Much of the code used "en-us" as the default value when not specified.
This limited the internationalization potential. Changing the default
to None and adds the ability to define the default lang code from other
locations in code. E.g.
```python
from mycroft.util.lang import set_default_lang
set_default_lang("en-us")
print("English date: "+nice_date(dt))
set_default_lang("de-de")
print("German date: "+nice_date(dt))
```
This allows easier localization of Skills by having the framework set the default without any changes necessary by the Skill writers.
Other minor changes:
* Changed the default return value of get_gender*() to None instead of False
* Split ADJUST into ADJUST and SET
* Use enum names instead of values.
* Add properties to CommonIoTSkill
* property -> attribute to avoid conflict with builtin
* Add increase and decrease actions
* Fix copy/paste error
Padatious was accepting fairly low confidence matches (0.5). This
would match a phrase such as "Where is the Empire State Building?" to the
intent "Where were you born?" (Which is a 0.54 match)
Now there are two passes mad on the Padatious fallback. The first is
looking for high confidence matches (> 0.8). Then other fallbacks -- such
as the CommonQuery fallbacks -- have an opportunity to consume the
utterance. If they don't consume it, Padatious is invoked again with
looser confidence before finally invoking the Unknown fallback.
Now the eyes go to a khaki color and "< < < LOADING < < <" displays
during the skill startup. When Padatious finishes the first training
we now emit a "mycroft.ready" on the messagebus.
A change to the skill_mark_1 to look for "mycroft.ready" instead of
"mycroft.skills.initialized" provides a good visual change to show that
it is ready for use.
The deprecation warning was firing off at the load of every skill when
the decorators are being iterated through by internal code (which
includes the self.config with an @property). Add check for these
special cases to not show a warning.
The scoring for CommonPlay would favor skills that responded with overly long
matches. E.g. "Huey Lewis and the News" would be considered a better fit than
just "News" for the request "Play the News"
* Deprecate self.config in skills
Skills should contain their own settings, the self.config concept is being
deprecated.
Also removed the defaults set for several old MycroftAI skills. The 19.02
version of these Skills initializes the default values using:
```python
self.settings["key"] = default
```
* Update padatious config to work with the config property.
* Add base common_iot_skill
* can_run takes IoTRequest
* Minor cleanup + documentation
* Fix pep8 issues
* Adjust scene and entity registration.
The controller skill is not guaranteed to be alive before
CommonIoTSkills, so we must call for values when it is alive, in
addition to accepting them at any time from other skills.
* Safer parsing
* Address PR feedback
* Add skill_id to register message
* Minor docstring edits
* Add remove page methods
* Implement SkillGUI.clear()
The method will now remove the namespace entirely from the gui.
This adds the message gui.clear.namespace
* Remove debug prints from SkillGUI
* Correcting docstring
* More docstring changes
* Remove whitespace added by Github webUI
* Attempt to create skill directory if not existing
* Handle missing priority skills
* Minor update of comments
* Handle skill load exception
Make sure an exception while trying to load/reload skill doesn't shutdown thread.
* Handle MsmException during SkillManager creation
If SkillManager can't be created due to an MsmException wait for network connection and retry.
* Update immediately if skill install file is missing
Missing skill install file indicates that this is a new venv and the requirements of the skills will need to be reinstalled.
* Add basic test for skill_manager
Basically only creating the skill_manager but it ensures that msm can be used on all supported python versions
* Update to pyee 5.0.0
- Update requirement
- Make the SkillSettings class hashable
* Update adapt to 0.3.2
* Upgrade websocket-client
* Remove special threading for common query
This sends a new system.update.check message when the device is booted and not paired
It is not sent every boot because the command could upgrade across major versions which the user could not want.
- Create a read_vocab_file() function that normal vocab loading and voc_match both uses. This function handles blank lines and comments
- Use a simpler regex instead of word logic to match
- Add a couple of test cases for the method
Previously the voc_match() method had several problems:
* Blank lines in the .voc file would cause it to match all strings
* Comments weren't ignored
* Substrings matched so "book" would match "ok", for instance
In the case where a network call during the initialization of the
settings poll fails the first time, it would never be tried again.
Now it will retry initialization once a minute.
The settings code worked, but was noisy and generally messy about
a few exceptional but common situations:
* When the .mycroft/skills/<SkillName> folder didn't already exist
* When network timeouts and such occcurred
I also slipped in a couple trivial code cleanups for an unused variable
and a log message.
To simplify the process of adding an idle page to a skill the decorator "resting_screen_handler" was added. In a skill class the decorator can be applied to a method to register it to handle idle.
@resting_page_handler("My Idle Page")
def handler(self, message):
...
The decorator will Register the method with the Mark-2 skill and perform all communications needed to make it work smoothly.
The wrong method was registered, instead of the wrapped function call
the original method was registered. This led to not being able to
unregister fallbacks.
* Add communication from GUI to skills
- "set" events from Qt will set/update a variable in the skills .gui member
- It's possible to add general event handlers using self.gui.register_handler()
- Moved registration of skill_id to just after skill init
* Ensure that simultaneously writes doesn't occur
Wrap WebSocketHandler.write_message() with a lock in an attempt to handle "buffererror: existing exports of data: object cannot be re-sized."
* Add better logging to help debug disconnect issue
* Allow overriding the idle page
SkillGUI.show_page() and SkillGUI.show_pages() now takes an optional
override_idle parameter. This is used as a hint by the mark-2 skill
and if possible the idle screen will not be shown.
* Improve debugging using Logger
* Raise exception when sending a non-existing gui page
* Restore running state to new connections
When a GUI is connected data and running namespaces are synchronised and
shown.
This refactors the code quite a bit moving the GUI state from the GUIConnection
object to the Enclosure.
The GUIConnection object does the handles the sync in the on_connection_open()
method.
* Add gui.page_interaction message
Currently triggered on page change on the display.
* Handle message when gui changes sessionData
* Check if socket exists on gui before sending data
* Increase port on each failure and retry
The audioservice can now jump forward and backward in the audio stream/file
The functionality is accessed via the audioservice class's seek_forward(),
seek_backward() and seek() methods
Several additions to the GUI protocol support
These changes allow switching between pages successfully with the current
mycroft-gui widget:
* Optimized commands to handle the active skill list
* MycroftSkill.gui.show_pages(list, idx) allows multiple-pages to be displayed
at a time starting with the given index visible.
* Merge SkillGUI.show_page with show_pages
This limits code duplication and makes things a bit more maintainable.
* Do not reload on changed .qmlc files
* Make EnclosureGeneric derive from Enclosure
* Update show function to match mycroft-gui-app
- adds internal representation of all loaded skills
- uses new commands to switch between pages and namespaces
* Add Extra debug output in enclosure
- Log if starting websocket fails
- Log the sending of page info in more detail
* Update GUI Debug client in CLI
- The CLI GUI now handles the new messages for switching pages
- Handle different data types better by using format instead of string concatenation
* Disable syncing code.
The sync code at startup outdated and needs to be reworked. Disabling it for now
to allow better interaction.
* Minor cleanups
- do not inherit from object
- use format instead of string concatenations
- remove duplicated self.loaded
- correct private member access
* Refactor GUIConnection.show()
Move the actions into separate methods for better overview of the logic
* Flipped "valid_file" to become "ignored_file"
Åke Forslund
Chris Veilleux
Kris Gesling
David Wagner
Chris Rogers
Steve Penrod
Aditya Mehra
Matthew D. Scholefield
Kris Gesling
Steve Penrod
Alistair Francis
ludwhe
jarbasal