Split skill_manager into three separate classes, SkillManager, SkillUpdater and SkillLoader splitting the responsibility into logical units
* Split the SkillManager.__init__ code to determine the download times into a new method
* Make docstrings consistent and PEP257 compliant. Also fixed a couple of spelling errors
* fixed two issues introduced in the previous refactoring
* removed unnecessary assignment of an instance attribute to a local variable
* updated the unit test to mock out code that reaches outside of core, like MSM and the configuration manager.
* add several unittests and refactored load_priority method.
* add a test for the _get_last_modified_date function.
* add "quick" argument to docstring
* removed unused import
* new class containing the logic to periodically update/install skills and send skill manifests to the backend.
* import MsmException from where it is defined, not from the skill manager.
* add some logging to the skill updater
* remove code now in SkillUpdater from SkillManager
* added imports to __init__.py to define the API into the message bus package
* new base class for unit tests and module for reusable mocks
* new skill loader class that will replace the _load_or_reload_skill() method in the SkillManager class.
* moved skill loading logic from core.py into skill_loader.py, resulting in some refactoring of skill loader and skill manager. change unit tests to match.
* added back some spacing that was inadvertently removed.
* change skill tester to use new SkillLoader class.
* Separate reload required check from performing reload to make logic easier
to follow
* Track skills that failed to load to handle infinite loop at first load
if skill fails to load
* Allow reloading skills that has failed to load
* Simplify first load of skills
- create activate, deactivate and unload methods for skill_loader
objects
- add sanity checks before activating and deactivating skills
- Update activation/deactivation test cases
- MycroftSkill and related core functions to mycroft_skill.py
- FallbackSkill to fallback_skill.py
- Add important classes decorators and functions to __init__.py
- move SkillGUI class to the enclosure along with the other enclosure
interfaces
core.py retains the same members as previously by means of imports to
retain backwards compatibility. When most of the available skills starts
using this new structure properly
The MycroftSkill.stop() method does not need to be implemented by the majority of skills, no reason to 'require' its implementation with an abstractmethod.
* Catch the new system.gui.user.interaction
This will in turn trigger the gui.page_interaction similar to the page flip but page_number will be None
* Fix issue when adding pages
Seems like appending would fail at times
* Add lock around namespace modifications
* Improve argument description for override_idle
Padatious is registered as a FallbackSkill and was pushing settings to
the backend.
This change allows Padatious to flag to the MycroftSkill class
that no settings should be created. This is done via the new optional
MycroftSkill parameter use_settings.
Previously Padatious intent matches were performed on non-normalized text, meaning that things like "what's the weather" wouldn't match a Padatious intent but
"what is the weather" would.
The "utterance" in Adapt intent data will still be non-normalized even if the intent match occurred on a normalized utterance. Retaining the existing behavior.
The "intent_failure" data. In there, "utterance" is always the raw version, "norm_utt" is the normalized one.
Also added better debugging info for intent matching to the log.
Also addresses a rare issue with the old code where the Adapt context could
have been updated even if the Adapt intent wasn't actually invoked due to
a higher Padatious intent match.
* Add the SkillGUI method send_event()
Sends raw mycroft.events.triggered messages to the gui for the skills namespace.
Example usage:
self.gui.send_event('event_name' {'param1': 12, 'param2': 'abc'})
The loose (conf > 0.5) Padatious match was previously occurring as Fallback
priority 99. The AIML fallback at priority 90 would consume lots of
utterances, interferring with many skills. Now Padatious runs at priority
89.
Additionally, added documentation of the intent and fallback system, including
guidelines for priorities.
Much of the code used "en-us" as the default value when not specified.
This limited the internationalization potential. Changing the default
to None and adds the ability to define the default lang code from other
locations in code. E.g.
```python
from mycroft.util.lang import set_default_lang
set_default_lang("en-us")
print("English date: "+nice_date(dt))
set_default_lang("de-de")
print("German date: "+nice_date(dt))
```
This allows easier localization of Skills by having the framework set the default without any changes necessary by the Skill writers.
Other minor changes:
* Changed the default return value of get_gender*() to None instead of False
The deprecation warning was firing off at the load of every skill when
the decorators are being iterated through by internal code (which
includes the self.config with an @property). Add check for these
special cases to not show a warning.
* Deprecate self.config in skills
Skills should contain their own settings, the self.config concept is being
deprecated.
Also removed the defaults set for several old MycroftAI skills. The 19.02
version of these Skills initializes the default values using:
```python
self.settings["key"] = default
```
* Update padatious config to work with the config property.
* Add remove page methods
* Implement SkillGUI.clear()
The method will now remove the namespace entirely from the gui.
This adds the message gui.clear.namespace
* Remove debug prints from SkillGUI
* Correcting docstring
* More docstring changes
* Remove whitespace added by Github webUI
- Create a read_vocab_file() function that normal vocab loading and voc_match both uses. This function handles blank lines and comments
- Use a simpler regex instead of word logic to match
- Add a couple of test cases for the method
Previously the voc_match() method had several problems:
* Blank lines in the .voc file would cause it to match all strings
* Comments weren't ignored
* Substrings matched so "book" would match "ok", for instance
To simplify the process of adding an idle page to a skill the decorator "resting_screen_handler" was added. In a skill class the decorator can be applied to a method to register it to handle idle.
@resting_page_handler("My Idle Page")
def handler(self, message):
...
The decorator will Register the method with the Mark-2 skill and perform all communications needed to make it work smoothly.
The wrong method was registered, instead of the wrapped function call
the original method was registered. This led to not being able to
unregister fallbacks.
* Add communication from GUI to skills
- "set" events from Qt will set/update a variable in the skills .gui member
- It's possible to add general event handlers using self.gui.register_handler()
- Moved registration of skill_id to just after skill init
* Ensure that simultaneously writes doesn't occur
Wrap WebSocketHandler.write_message() with a lock in an attempt to handle "buffererror: existing exports of data: object cannot be re-sized."
* Add better logging to help debug disconnect issue
* Allow overriding the idle page
SkillGUI.show_page() and SkillGUI.show_pages() now takes an optional
override_idle parameter. This is used as a hint by the mark-2 skill
and if possible the idle screen will not be shown.
* Improve debugging using Logger
* Raise exception when sending a non-existing gui page
* Restore running state to new connections
When a GUI is connected data and running namespaces are synchronised and
shown.
This refactors the code quite a bit moving the GUI state from the GUIConnection
object to the Enclosure.
The GUIConnection object does the handles the sync in the on_connection_open()
method.
* Add gui.page_interaction message
Currently triggered on page change on the display.
* Handle message when gui changes sessionData
* Check if socket exists on gui before sending data
* Increase port on each failure and retry
Several additions to the GUI protocol support
These changes allow switching between pages successfully with the current
mycroft-gui widget:
* Optimized commands to handle the active skill list
* MycroftSkill.gui.show_pages(list, idx) allows multiple-pages to be displayed
at a time starting with the given index visible.
* Merge SkillGUI.show_page with show_pages
This limits code duplication and makes things a bit more maintainable.
* Do not reload on changed .qmlc files
* Make EnclosureGeneric derive from Enclosure
* Update show function to match mycroft-gui-app
- adds internal representation of all loaded skills
- uses new commands to switch between pages and namespaces
* Add Extra debug output in enclosure
- Log if starting websocket fails
- Log the sending of page info in more detail
* Update GUI Debug client in CLI
- The CLI GUI now handles the new messages for switching pages
- Handle different data types better by using format instead of string concatenation
* Disable syncing code.
The sync code at startup outdated and needs to be reworked. Disabling it for now
to allow better interaction.
* Minor cleanups
- do not inherit from object
- use format instead of string concatenations
- remove duplicated self.loaded
- correct private member access
* Refactor GUIConnection.show()
Move the actions into separate methods for better overview of the logic
* Flipped "valid_file" to become "ignored_file"
Several small changes based on the code review feedback:
* Drop '_' from classes like Enclosure_Mark1
* Adopt Python 3 style for class definitions and don't explicitly list '(object)'
* Slightly better documentation
* Moved MycroftSkill.show_html() to SkillGUI, resulting in code like self.gui.show_page('Weather.qml')
* Renamed SkillGUI.__dict to SkillGUI.__session_data. This better reflects the
how values are accessed in the QML.
The QML files are typically not translated like other Mycroft resources,
they have internal translation tools. And at times there are other
resources that don't need to be translated all the time, for example
color strings like "AliceBlue" which might be used by non-English
speakers.
So now the translation mechanism looks for resource file X as follows:
1) Look for <res_dir>/<lang>/X
2) Look for <res_dir>/X
3) Look for anywhere under locale/<lang>/.../X
And now the show_page() method starts looking for resources under the skill/ui folder.
Further fleshing out of the GUI mechanisms
* Support for data and page from Mycroft -> GUIConnection
* Add a 'reconnecting' event for the messagebus
* Add MycroftSkill.show_url()
* Plumb MycroftSkill.gui into the messagebus
* Implement MycroftSkill.gui dictionary
CLI extensions for the GUI:
* Can now act as a simple GUIConnection
* Minor revamp of messagebus connection, provides kinder handling when
messagebus isn't found or ready.
* BUGFIX: An empty filter would filter ALL messages
* BUGFIX: Input wider than the screen would cause a crash
* BUGFIX: "filter" or "find" with no param would filer "filter" or find "find"
Still very much a work in progress.
For understand and testing, here is the sequence:
STEP 1: GUI announces itself
* Connect to the main Mycroft messagebus
* Send: "mycroft.gui.connected" with data { "gui_id": XXX } where XXX is a uniq ID (uuid)
STEP 2: Mycroft creates GUI socket
* Mycroft extracts the gui_id
* Mycroft prepares a socket and announces its availability on the Mycroft messagebus with:
self.bus.emit(Message("mycroft.gui.port",
{"port": self.GUIs[gui_id].port,
"gui_id": gui_id}))
STEP 3: GUI connects
In python, a very minimal test socket handler on the GUI side would look like this
from websocket import create_connection
port = 18181 (from the message above)
ws = create_connection("ws://0.0.0.0:"+port+"/gui")
ws.send("Hello, World")
print("Sent")
print("Receiving...")
result = ws.recv()
print("Received '%s'" % result)
ws.close()
Enclosures
* Create a mechanism to instantiate unique Enclosure classes, depending on the platform found in the SYSTEM mycroft.conf
* Implement a generic Enclosure, which support the new GUI protocol
* Implement a Mark 1 Enclosure (expects the serial connection to an Arduino)
* Implement the start of a Mark II enclosure
* Implement a generic enclosure (no screen)
* Implement the GUI announcement and protocol basics
MycroftSkill
* Implement the basis of the GUI-controlling interfaces. Namely:
- MycroftSkill.show_text()
- MycroftSkill.show_image()
- MycroftSkill.show_html()
- MycroftSkill.show_page()
- MycroftSkill.gui to set values for page displays.
Configuration
* Add "gui_websocket" to the mycroft.config.py
This makes a cross context call be treated as one level when calculating the probability. this makes previous contexes not be completely invalidated when a cross context call is sent.
Chris Veilleux
Åke Forslund
Steve Penrod
Chris Rogers
Aditya Mehra
Kris Gesling
Steve Penrod
ludwhe
Michael Nguyen