Split skill_manager into three separate classes, SkillManager, SkillUpdater and SkillLoader splitting the responsibility into logical units
* Split the SkillManager.__init__ code to determine the download times into a new method
* Make docstrings consistent and PEP257 compliant. Also fixed a couple of spelling errors
* fixed two issues introduced in the previous refactoring
* removed unnecessary assignment of an instance attribute to a local variable
* updated the unit test to mock out code that reaches outside of core, like MSM and the configuration manager.
* add several unittests and refactored load_priority method.
* add a test for the _get_last_modified_date function.
* add "quick" argument to docstring
* removed unused import
* new class containing the logic to periodically update/install skills and send skill manifests to the backend.
* import MsmException from where it is defined, not from the skill manager.
* add some logging to the skill updater
* remove code now in SkillUpdater from SkillManager
* added imports to __init__.py to define the API into the message bus package
* new base class for unit tests and module for reusable mocks
* new skill loader class that will replace the _load_or_reload_skill() method in the SkillManager class.
* moved skill loading logic from core.py into skill_loader.py, resulting in some refactoring of skill loader and skill manager. change unit tests to match.
* added back some spacing that was inadvertently removed.
* change skill tester to use new SkillLoader class.
* Separate reload required check from performing reload to make logic easier
to follow
* Track skills that failed to load to handle infinite loop at first load
if skill fails to load
* Allow reloading skills that has failed to load
* Simplify first load of skills
- create activate, deactivate and unload methods for skill_loader
objects
- add sanity checks before activating and deactivating skills
- Update activation/deactivation test cases
Message bus config loading is now shared by service and client.
messagebus.client.ws file is still available in case skills are using it. It is a backport that inherits from the new MessageBusClient class. Adds depreciation warning.
* Run emitter using threadpool
This moves the thread pool from the websocket client into the eventemitter allowing each registered function to run in a separate thread and not just each event.
This speeds up cases where there is a one to many call such as the common play framework and the upcomming common query framework.
* Add unit tests for threaded event emitter
* Add standard header
* Add standard header
This ensures no attributes are None and simplifies some checks within the function
Otherwise, if, for instance, there were no 'data' field in a message, when deserializing, the caller would have to check for None
The settings code worked, but was noisy and generally messy about
a few exceptional but common situations:
* When the .mycroft/skills/<SkillName> folder didn't already exist
* When network timeouts and such occcurred
I also slipped in a couple trivial code cleanups for an unused variable
and a log message.
This moves the thread pool from the websocket client into the eventemitter allowing each registered function to run in a separate thread and not just each event.
This speeds up cases where there is a one to many call such as the common play framework and the upcomming common query framework.
Further fleshing out of the GUI mechanisms
* Support for data and page from Mycroft -> GUIConnection
* Add a 'reconnecting' event for the messagebus
* Add MycroftSkill.show_url()
* Plumb MycroftSkill.gui into the messagebus
* Implement MycroftSkill.gui dictionary
CLI extensions for the GUI:
* Can now act as a simple GUIConnection
* Minor revamp of messagebus connection, provides kinder handling when
messagebus isn't found or ready.
* BUGFIX: An empty filter would filter ALL messages
* BUGFIX: Input wider than the screen would cause a crash
* BUGFIX: "filter" or "find" with no param would filer "filter" or find "find"
This eliminates a lot of the noise in the log files. Later I'll add features in the CLI to
assist watching the messagebus messages rather than writing them all to logs.
Also corrected some language and formatting in settings.py docstrings.
To make reply work in the same manner as a newly created message data
shall be optional. This allows mistakes like L1141 of
mycroft/skills/core.py, where the data field isn't set. (And causes an
exception when the line is hit.)
This sends a ctrl+c signal to each process which will allow code to exit properly by handling KeyboardInterrupt
Other notable changes:
- create_daemon method used to clean up create daemon threads
- create_echo_function used to reduce code duplication with messagebus
echo functions
- wait_for_exit_signal used to wait for ctrl+c (SIGINT)
- reset_sigint_handler used to ensure SIGINT will raise KeyboardInterrupt
The emitted event "open" will in many cases call ws.emit, and this will
lock if the connected_event isn't set. This makes sure that the
connected_event is set before emitting the open event.
If the sentence is normalized the utterance_remainder() method fails if
the intent keyword expects normalization.
Example the joking skill has an intent "tell me joke" which is the
normalized equivalent of "tell me a joke". In this case the intent wouldn't be removed from the remainder and instead of an empty string the utterance remainder would contain "tell me joke"
* Add wait_for_response method to Websock client
The client handles the basic case when wanting to do a syncronous
request-response action.
The method sets up a handler waits for the response and handles timeout.
The expected format is that the reply message should have the same type as
the original message with ".response" appended.
An method in the Message class has been added to create a standard response for
a message.
* Let the :skills command use wait_for_response
* Minor docstring changes
Fixed typos and refined text
Add message.utterance_remainder() method
This helper will return the portion of an utterance not
consumed by the Adapt parser already. For example,
"turn on the kitchen light" would have a remainder of
"the kitchen" if there was an Intent with entities that
matched "turn on" and "light". The returned text is passed
through normalize().
Add Python 2/3 compatibility
==== Tech Notes ====
This allows the main bus, skills and cli to be run in both python 2.7 and
3.5+.
Mainly trivial changes
- syntax for exceptions
- logic for importing correct Queue module
- .iteritems -> future.utils.iteritems when accessing dicts key value
pairs
* Allow audio service to be run in python 3
* Make speech client work with python 3
* Importing of Queue version dependent
* Exception syntax corrected
* Creating sound buffer is version dependant
- Adapt context use range from builtins
- Use compatible next() instead of .next() when walking the skill
directory
* Make CLI Python 3 Compatible
- Use compatible BytesIO instead of StringsIO
- Open files as text instead of binary
- Make sure integer divisions are used
* Make messagebus send compatible
* Fix failing travis
Re-add future 0.16.0
* Make string checks compatible
* basestring doesn't exist in python 3 so it's imported from the "past"
* Fix latest compatibility issues in speech client
- handle urllib
- handle encoding before calling md5
* Make Api.build_json() python 2/3 compatible
==== Tech notes ====
The send script will throw an IOerror exception if the bus service isn't
started. This correctly catches it and emits a single warning.
==== Tech Notes ====
Previous approach did not work as websocket client was not running, this
uses the standalone send functionality recently added to send a single
message.
Significantly reworked the loading/updating of Skills. Unified
all management under a single SkillManager class. This class
runs as a thread that initially loads, upgrades (via MSM)
and reloads skills.
Removed the independent threads that were being run. The skill
updating still happens once an hour, but works in conjunction
with the scan to reload modified skills. Also added messagebus
notifications from MSM so mycroft-core can pause reloading
skills until the installation is complete.
Added a new mycroft.messagebus.send module to allow command
line interaction with the messagebus, e.g.:
python -m mycroft.messagebus.send mycroft.wifi.start
python -m mycroft.messagebus.send speak '{"utterance":"hello"}'
==== Fixed Issues ====
MSM installs that have PIP dependencies were failing, as the
load would occur after code was retrieved but before PIP install
completed. Restart was required to load new skills.
==== Tech Notes ====
TODO: Change the way we manage modules. The auto-load of the
remote configuration for the module is silly, slow and wasteful.
I made the WebsocketClient.build_url() method static in
anticipation of being able to do this more efficiently when the
submodule load doesn't hit the remove API automatically.
==== Localization Notes ====
Modified 'sorry I couldn't install default skills' message.
==== Protocol Notes ====
MSM now generates:
msm.updating
msm.installing
msm.install.succeeded { "skill" : name }
msm.install.failed { "skill" : name, "error" : code }
msm.installed
msm.updated
msm.removing
msm.remove.succeeded { "skill" : name }
msm.remove.failed { "skill" : name, "error" : code }
msm.removed
An update can now be forced by posting 'skillmanager.update' to the
messagebus.
This commit officially switches the mycroft-core repository from
GPLv3.0 licensing to Apache 2.0. All dependencies on GPL'ed code
have been removed and we have contacted all previous contributors
with still-existing code in the repository to agree to this change.
Going forward, all contributors will sign a Contributor License
Agreement (CLA) by visiting https://mycroft.ai/cla, then they will
be included in the Mycroft Project's overall Contributor list,
found at: https://github.com/MycroftAI/contributors. This cleanly
protects the project, the contributor and all who use the technology
to build upon.
Futher discussion can be found at this blog post:
https://mycroft.ai/blog/right-license/
This commit also removes all __author__="" from the code. These
lines are painful to maintain and the etiquette surrounding their
maintainence is unclear. Do you remove a name from the list if the
last line of code the wrote gets replaced? Etc. Now all
contributors are publicly acknowledged in the aforementioned repo,
and actual authorship is maintained by Github in a much more
effective and elegant way!
Finally, a few references to "Mycroft AI" were changed to the correct
legal entity name "Mycroft AI Inc."
==== Fixed Issues ====
#403 Update License.md and file headers to Apache 2.0
#400 Update LICENSE.md
==== Documentation Notes ====
Deprecated the ScheduledSkill and ScheduledCRUDSkill classes.
These capabilities have been superceded by the more flexible MycroftSkill
class methods schedule_event(), schedule_repeating_event(), update_event(),
and cancel_event().
Chris Veilleux
Åke Forslund
jarbasal
Matthew D. Scholefield
Steve Penrod
Steve Penrod
Matthew D. Scholefield
gaurav