Major refactoring of the skills startup sequence
- Restructure to a less nested structure
- Remove usage of globals by wrapping a lot of state variables into a class this allows for things like caching a negative pairing status throughout the startup process
Message bus config loading is now shared by service and client.
messagebus.client.ws file is still available in case skills are using it. It is a backport that inherits from the new MessageBusClient class. Adds depreciation warning.
* Catch the new system.gui.user.interaction
This will in turn trigger the gui.page_interaction similar to the page flip but page_number will be None
* Fix issue when adding pages
Seems like appending would fail at times
* Add lock around namespace modifications
* Improve argument description for override_idle
* Add the SkillGUI method send_event()
Sends raw mycroft.events.triggered messages to the gui for the skills namespace.
Example usage:
self.gui.send_event('event_name' {'param1': 12, 'param2': 'abc'})
* Add remove page methods
* Implement SkillGUI.clear()
The method will now remove the namespace entirely from the gui.
This adds the message gui.clear.namespace
* Remove debug prints from SkillGUI
* Correcting docstring
* More docstring changes
* Remove whitespace added by Github webUI
* Add new api command to send visemes as single list. This allows more efficient use of the messagebus and gives implementors flexibility in how they handle the visualization.
* Switch mark1 to use viseme_list
* Add communication from GUI to skills
- "set" events from Qt will set/update a variable in the skills .gui member
- It's possible to add general event handlers using self.gui.register_handler()
- Moved registration of skill_id to just after skill init
* Ensure that simultaneously writes doesn't occur
Wrap WebSocketHandler.write_message() with a lock in an attempt to handle "buffererror: existing exports of data: object cannot be re-sized."
* Add better logging to help debug disconnect issue
* Allow overriding the idle page
SkillGUI.show_page() and SkillGUI.show_pages() now takes an optional
override_idle parameter. This is used as a hint by the mark-2 skill
and if possible the idle screen will not be shown.
* Improve debugging using Logger
* Raise exception when sending a non-existing gui page
* Restore running state to new connections
When a GUI is connected data and running namespaces are synchronised and
shown.
This refactors the code quite a bit moving the GUI state from the GUIConnection
object to the Enclosure.
The GUIConnection object does the handles the sync in the on_connection_open()
method.
* Add gui.page_interaction message
Currently triggered on page change on the display.
* Handle message when gui changes sessionData
* Check if socket exists on gui before sending data
* Increase port on each failure and retry
* Try to improve stability
- Remove sync_active
- Update the way variables are sent to the gui
* Do not show full path for pages
The cli now shows the basename of the current page to make it easier to
determine if the correct page is displayed.
* Make elements in loaded list named tuples
This makes the intent of the code a bit cleaner.
Several additions to the GUI protocol support
These changes allow switching between pages successfully with the current
mycroft-gui widget:
* Optimized commands to handle the active skill list
* MycroftSkill.gui.show_pages(list, idx) allows multiple-pages to be displayed
at a time starting with the given index visible.
* Merge SkillGUI.show_page with show_pages
This limits code duplication and makes things a bit more maintainable.
* Do not reload on changed .qmlc files
* Make EnclosureGeneric derive from Enclosure
* Update show function to match mycroft-gui-app
- adds internal representation of all loaded skills
- uses new commands to switch between pages and namespaces
* Add Extra debug output in enclosure
- Log if starting websocket fails
- Log the sending of page info in more detail
* Update GUI Debug client in CLI
- The CLI GUI now handles the new messages for switching pages
- Handle different data types better by using format instead of string concatenation
* Disable syncing code.
The sync code at startup outdated and needs to be reworked. Disabling it for now
to allow better interaction.
* Minor cleanups
- do not inherit from object
- use format instead of string concatenations
- remove duplicated self.loaded
- correct private member access
* Refactor GUIConnection.show()
Move the actions into separate methods for better overview of the logic
* Flipped "valid_file" to become "ignored_file"
Several small changes based on the code review feedback:
* Drop '_' from classes like Enclosure_Mark1
* Adopt Python 3 style for class definitions and don't explicitly list '(object)'
* Slightly better documentation
* Moved MycroftSkill.show_html() to SkillGUI, resulting in code like self.gui.show_page('Weather.qml')
* Renamed SkillGUI.__dict to SkillGUI.__session_data. This better reflects the
how values are accessed in the QML.
is kept in order, but old namespaces/skills are never culled.
* The active namespace list is synced on a GUIConnection level
* QML display requests now are sent as a list instead of a single entry, i.e.
with "gui_urls" instead of "gui_url". Currently a skill can only send a single
QML, however.
* Change CLI GUI client to handle "gui_urls" instead of "gui_url"
Further fleshing out of the GUI mechanisms
* Support for data and page from Mycroft -> GUIConnection
* Add a 'reconnecting' event for the messagebus
* Add MycroftSkill.show_url()
* Plumb MycroftSkill.gui into the messagebus
* Implement MycroftSkill.gui dictionary
CLI extensions for the GUI:
* Can now act as a simple GUIConnection
* Minor revamp of messagebus connection, provides kinder handling when
messagebus isn't found or ready.
* BUGFIX: An empty filter would filter ALL messages
* BUGFIX: Input wider than the screen would cause a crash
* BUGFIX: "filter" or "find" with no param would filer "filter" or find "find"
Still very much a work in progress.
For understand and testing, here is the sequence:
STEP 1: GUI announces itself
* Connect to the main Mycroft messagebus
* Send: "mycroft.gui.connected" with data { "gui_id": XXX } where XXX is a uniq ID (uuid)
STEP 2: Mycroft creates GUI socket
* Mycroft extracts the gui_id
* Mycroft prepares a socket and announces its availability on the Mycroft messagebus with:
self.bus.emit(Message("mycroft.gui.port",
{"port": self.GUIs[gui_id].port,
"gui_id": gui_id}))
STEP 3: GUI connects
In python, a very minimal test socket handler on the GUI side would look like this
from websocket import create_connection
port = 18181 (from the message above)
ws = create_connection("ws://0.0.0.0:"+port+"/gui")
ws.send("Hello, World")
print("Sent")
print("Receiving...")
result = ws.recv()
print("Received '%s'" % result)
ws.close()
Enclosures
* Create a mechanism to instantiate unique Enclosure classes, depending on the platform found in the SYSTEM mycroft.conf
* Implement a generic Enclosure, which support the new GUI protocol
* Implement a Mark 1 Enclosure (expects the serial connection to an Arduino)
* Implement the start of a Mark II enclosure
* Implement a generic enclosure (no screen)
* Implement the GUI announcement and protocol basics
MycroftSkill
* Implement the basis of the GUI-controlling interfaces. Namely:
- MycroftSkill.show_text()
- MycroftSkill.show_image()
- MycroftSkill.show_html()
- MycroftSkill.show_page()
- MycroftSkill.gui to set values for page displays.
Configuration
* Add "gui_websocket" to the mycroft.config.py
This is a step towards abstracting the idea of an Enclosure which ties Mycroft to the hardware that is running Mycroft.
- Move the enclosure API to mycroft.enclosure.api (previously was mycroft.client.enclosure.api)
- Move display_manager out of enclosure client to mycroft.enclosure.display_manager
- Merge EnclosureWeather into EnclosureMouth
- Wrap display manager in a class
The Mark 1 button press can now be "consumed" when a skill handles
the Stop command. When this happens, the button press will not
trigger listening mode. An additional press would be needed to
trigger listening.
This introduces the "mycroft.stop.handled" messagebus message. It
carries a data field called "by" which identifies who handled it.
Currently the values are "TTS" for when speaking ends or the name
of a skill which implements Stop and returns True from the call.
Also fixed a potential bug when the flag to clear queued visemes
was left set after a button press.
* Add interface to Mark 1 faceplace capabilities
This adds API interfaces for two Mark 1 faceplace capabilities to the
mycroft.client.enclosure.EnclosureAPI()
EnclosureAPI.setpixel(index, r,g,b)
- Set individual eye pixels to any color. The indices go from 0-23 with
the 0-12 corresponding to the right eye and 11-23 to the left.
EnclosureAPI.fill(percentage)
- Fill the eyes to a percentage, for use in meters or countdowns. The
right eye is 0-50%, the left eye also fills if going up to 100%.
* Made MycroftSkill.remove_event() return a bool, preventing unnecessary/misleading message from being posted by MycroftSkill.cancel_scheduled_event()
* More doc and several minor renames around intent processing
* Several minor typo and doc corrections
Withe the NTP checks in place, the sequence of visual and audio queues
was a little clunky. This refines it slightly for normal use and to
play better with the pairing process.
Raspberry Pi's don't have a built-in clock, so at boot-up the clock just picks up from when they were last running. Normally this is corrected very quickly by NTP from an internet server, but if there is no network connection that cannot happen.
When an out-of-the-box Mark 1 or Picroft is being setup, the clock is set to whenever the image was created. Upon completing the Wifi setup step the NTP service can finally sync with the internet, so time suddenly "jumps" to weeks later -- usually. In either case (when the date jumps or when the date is erronously months old), there is potential for havoc.
These changes deal with that situation. Upon network connection, an NTP synchonization is forced. If it is detected that a major time jump happened
(more than 1 hour), then the user is notified that the clock change requires
a reboot and the system restarts.
Other changes:
* use the new "system." message namespace
* add pause before the system.reboot during a WIPE, allowing reset to totally complete
*
The out-of-box spiel given by Mycroft was coming at the user pretty fast.
This adds a momentary pause in the spoken text.
Also cleaned up some ugly messagebus interaction to use the speak() method.
Add Python 2/3 compatibility
==== Tech Notes ====
This allows the main bus, skills and cli to be run in both python 2.7 and
3.5+.
Mainly trivial changes
- syntax for exceptions
- logic for importing correct Queue module
- .iteritems -> future.utils.iteritems when accessing dicts key value
pairs
* Allow audio service to be run in python 3
* Make speech client work with python 3
* Importing of Queue version dependent
* Exception syntax corrected
* Creating sound buffer is version dependant
- Adapt context use range from builtins
- Use compatible next() instead of .next() when walking the skill
directory
* Make CLI Python 3 Compatible
- Use compatible BytesIO instead of StringsIO
- Open files as text instead of binary
- Make sure integer divisions are used
* Make messagebus send compatible
* Fix failing travis
Re-add future 0.16.0
* Make string checks compatible
* basestring doesn't exist in python 3 so it's imported from the "past"
* Fix latest compatibility issues in speech client
- handle urllib
- handle encoding before calling md5
* Make Api.build_json() python 2/3 compatible
Creating several visual feedback points to let users know what is
occurring in the boot sequence. This also expects a new version
of the enclosure.
* On shutdown and reset, change eye color to a dark gray
* On initial startup the enclosure will have left the eyes at a
yellow color, indicating that it is on but not fully running yet.
When the enclosure comes up (and doesn't kick off the out-of-box
or the notice that they need to run wifi setup) it will begin to
scroll an "UPDATING" message.
This assumes that someone else will remove the "UPDATING" message.
That gets handled by the Mycroft-Mark-1 skill, which resets the
mouth and sets the eye color to default.
When the TTS engine provides visemes to the faceplate, the information
passed along consists of the mouth shape and the duration to display it.
When the system get backed-up for some reason (e.g. the CPU is briefly
overloaded), the code would attempt to catch up the animation but would
still send the 'expired' viseme across the serial port to the faceplate
with no wait-time. This results in a fast-moving mouth to catch up,
which isn't very pleasing.
Now the viseme is passed along with an expiration date, so if the time
to display it has already passed then the viseme code gets thrown away
instead of being sent across the (relatively slow) serial port. This
allows better catch-up.
Chris Veilleux
Åke
Steve Penrod
Dominik
Steve Penrod
Matthew D. Scholefield
jarbasai