* Add protection for naive skill authors
It is fairly common for new skill authors to attempt actions in the __init__()
method which are not legal yet, as the Skill has not been fully connected to
the Mycroft system. This adds an @property protection layer for the two most common
issues:
* Accessing MycroftSkill.bus
* Accessing MycroftSkill.enclosure
Now those are properties instead of variables and provide appropriate warnings
when used before they exist.
Also enhancing the handling of error logs in the CLI to highlight problems such
as this:
* Color "- ERROR -" log messages in red
* Retaining leading characters from log messages, improving readability in formatted messages
This eliminates a lot of the noise in the log files. Later I'll add features in the CLI to
assist watching the messagebus messages rather than writing them all to logs.
Also corrected some language and formatting in settings.py docstrings.
* Add "wait" option to MycroftSkill.speak() and speak_dialog()
The new "wait" option will cause the speak function to block until all
of the given dialog has been spoken by Mycroft. This means:
self.speak("Hello world", wait=True)
is now equivalent to:
self.speak("Hello world")
wait_while_speaking()
* Add ability to schedule event in seconds
The MycroftSkill.schedule_event() method now accepts an integer in addition to
a datetime for the 'when' parameter. The integer represents the number of
seconds in the future to fire off the event. E.g.
```python
self.schedule_event(some_handler, 7)
```
Will invoke some_handler() seven seconds from now.
Also unified language used in event docstrings.
Switch to an OrderedDict() for translate_namedvalues(), maintaining the
sequence of values defined in the original "list.value" file. This is
useful in circumstances where there are multiple values, but the order
of listing indicates some sort of preference.
This is used in the Alarm skill to allow synonyms like "weekdays"/"weekday",
"Mondays/Monday", but the first value is used when building the status string.
For example "You have an alarm for 8am on Mondays". Generically, this lets
translators consistently provide preferred names for values by adjusting the
order.
Adapt doesn't populate the entry from the one_of correctly from context. To work around the issue intent structure is scanned for empty keys and tries to populate them from entities in __tags__
* Add repeat option to audioservice
The audioservice.play() method now accepts a repeat parameter. If this
parameter is True the playlist passed to the audio service will be
repeated.
* Add repeat support to vlc
* Add the repeat parameter to all services
Not functional but playback will work at least. Hacktoberfest?
* Add cleanup if the skill loading fails
If a skill throws an exception in initialize the registered handlers
need to be removed. To cleanup this the skill shutdown procedure is
run.
* Don't try converse method on non-loaded skills
Checks if a valid skill instance is present before calling the converse method
to filter out skills that wasn't loaded.
* Mark unloaded skills as inactive in CLI
The recent changes to support the 'locale' directory were incorrectly
joining the os.walk() path and filename, resulting in double-directories,
like "./vocab/en-us/./vocab/en-us/Something.voc"
This is a step towards abstracting the idea of an Enclosure which ties Mycroft to the hardware that is running Mycroft.
- Move the enclosure API to mycroft.enclosure.api (previously was mycroft.client.enclosure.api)
- Move display_manager out of enclosure client to mycroft.enclosure.display_manager
- Merge EnclosureWeather into EnclosureMouth
- Wrap display manager in a class
* Add MycroftSkill.find_resource() that locates a localization resource file
from either under the old vocab/regex/dialog folder, or under any folder in
the new 'locale' folder. The locale folder unifies the three different
folders and allows arbitrary subfolders underneath it.
* MycroftSkill.speak_dialog() will now speak the entry name if no dialog file
is found. Periods are replaced with spaces, so
```self.speak_dialog("this.is.a.test.")``` would return "this is a test" if
no file named this.is.a.test.dialog is found.
* Remove MycroftSkills.vocab_dir value
* Minor edits to several docstrings
The method first tries to use the skill vocab directory and then tries
the mycroft/res/text... directory.
After the vocab is loaded from file once it's stored in a cache
dictionary and further uses of the method won't hit the disk.
Example:
if self.voc_match(utt, "Yes):
...
The initial implementation of web settings returned values as
strings all the time, even Boolean and numeric values. This
required unnecessarily complicated code, such as:
```python
if setting["show_time"] == "true":
# do whatever...
```
Now a value defined in metadata as a "checkbox" gets cast to a boolean,
and values defined in metadata as a "number" is typecast to int or
float, as appropriate. This allows cleaner code such as:
```python
if setting["show_time"]:
# do whatever...
```
NOTE: This can be a breaking change, verify you skill which uses 'checkbox'
webUI types handles this correctly for 18.08.
The list holding the names of scheduled events was being populated with decorated names,
but other code assumed it held the "friendly" name. Corrected this assumption.
Several locations also removed list entries while iterating on the same list, resulting
in entries being skipped and left un-deleted. This left behind phantom scheduled events
when skills were reloaded, goofing up many schedules after the reload.
Allow a Padatious intent to override Adapt when it is VERY
certain that the utterance is directed at it. (95% confidence
or greater.) Right now that only occurs if the intent match
for the given phrase is perfect.
This solves this kind of issue:
* Adapt: Matching on "Set" and "Alarm"
* Padatious: Handling "is an alarm set"
* Fix logic error for when no Padatious intent
While working on the Alarm skill I discovered several issues with the
event scheduler. This PR cleans up those findings and resolves several
other potential issues:
1) To avoid thread synchronization issues, the EventScheduler had several
queues which independently held objects to be added/deleted/updated. However, the order of the events was undefined and got mixed since they were all batched together. So, for instance, if skill code performed:
self.add_event("foo", self.handle_foo)
if SomeReason:
self.cancel_event("foo")
The actual order of queue handling would perform Remove first, then Add which resulted in "foo" not being found for delete, but then added and left as an active event.
Now the EventScheduler protects the list using a Lock and the queues have been removed. Modifications to the list happen immediately after obtaining the lock and are not batched up.
2) One-time events were triggered while the event was still in the EventScheduler list. Now the entry is removed before invoking the handler.
3) Within the MycroftSkill.add_event(name, handler) is a local 'wrapper' method that actually makes the callback. The MycroftSkill.remove_event(name) method attempted to find entries in the events list and the associated handler entries in the self.emitter to remove. However, the emitter actually held the wrapper(handler), not the handler itself. So the emitter handlers were left behind.
This was a quiet bug until the next time you scheduled an event of the same name. When that second event finally triggered, it would fire off both the new and the old handler -- which snowballed in the 'skill-alarm:Beep' case, doubling and redoubling with every beep.
Now this cancels all the emitter listeners by name. There is a very slim chance that someone has registered a listener with the same name, but since it is namespaced to "skill-name:Event" I think this is pretty safe.
Not technically related, but a failure that has been lurking for
some time and is a French unit test that doesn't work depending
on the time of day when the test is run.
At startup, Padatious was blocking in the fallback while the
training occurred. As a result, any attempts to use Mycroft
during that period would queue up rather than giving the
user feedback. Now it immediately returns False, allowing
user notification to occur elsewhere.
* Fix cases where the unmunge misses keywords
The unmunge would invariably miss keys due to the fact that the dict is modified while being iterated. This breaks out the keys into a list before iterating through them to ensure that all original keys are checked.
* Clean up the unmunge function slightly
This makes the dialog parameter dual-purpose. It can be used as a
literal spoken string, or a dialog resource. In the future the
announcement parameter can be retired.
New function allows simple yes/no queries to be asked, capturing common
affirmations (e.g. "yes", "yeah", "yep", "sure", ...) and rejections
("no", "nope", ...) that are consistent across all skills.
The method will return a normalized 'yes', 'no' or whatever else is spoken
for further parsing. None is also possible.
This also adds the MycroftSkill.is_match() method to assist in matching
translated synonyms within an utterance.
- Remove setup scripts for mycroft-skills-sdk since it's not used anymore
- Rename mycroft-base-setup.in/MANIFEST.in to setup.py and MANIFEST.in
- Remove skill-container, since it hasn't been used or kept up to date since 17.08 and the cli commands to remove and activate skills is easier to work with
==== Environment Notes ====
Small update of the packaging script is needed due to this change
When running on a platform for which there was not DEFAULT.platform
file in the mycroft-skills repo, the downloading of skills would
crash. Now an informational message is shown and the DEFAULT
skills alone are loaded.
The Mark 1 button press can now be "consumed" when a skill handles
the Stop command. When this happens, the button press will not
trigger listening mode. An additional press would be needed to
trigger listening.
This introduces the "mycroft.stop.handled" messagebus message. It
carries a data field called "by" which identifies who handled it.
Currently the values are "TTS" for when speaking ends or the name
of a skill which implements Stop and returns True from the call.
Also fixed a potential bug when the flag to clear queued visemes
was left set after a button press.
* Update format for skill listing
Now send the skills with id and active status
* Add commands to activate/deactivate skills
* Add "unload all except one" functionallity
* Update after rebasing
- fix identifying skills
* Unload skills if they're removed from disk
* Rename _shutdown to default_shutdown
The method is not intended to be non-public, and this should shut up
codacy bot.
* Handle keep command without argument
* Add new commands to help
- Split help into multiple pages as needed
* Support :activate all
This is necessary because in Python 3, hash(x) changes every single start of the application. Using the skill folder makes it consistent. In addition, the skill folder makes it easier to debug parts of the application in comparison to using something like an md5sum
This keeps track of the skills whose dependencies have already been installed. While it won't automatically register newly installed skills, it will attempt to reinstall dependencies the next boot only one time so it shouldn't be a big issue.
Checking against basestring was necessary in python two since unicode and string were separate classes. In python3 basestring was removed since different string classes have been removed.
Lots of minor fixes including, sorting dicts, making ints of strings,
MagicMock file spec and some other things
A couple of issues in the mycroft-core code base were identified and
fixed. Most notably the incorrect version check for python three when
adding basestring.
Update .travis.yml
When loading voc and regex files, lines starting with "#" are now
ignored, so developers and translators can use them to document their
decisions.
==== Fixed Issues ====
This sends a ctrl+c signal to each process which will allow code to exit properly by handling KeyboardInterrupt
Other notable changes:
- create_daemon method used to clean up create daemon threads
- create_echo_function used to reduce code duplication with messagebus
echo functions
- wait_for_exit_signal used to wait for ctrl+c (SIGINT)
- reset_sigint_handler used to ensure SIGINT will raise KeyboardInterrupt
Allow intents to be enabled or disabled from outside of the skill via
the messagebus.
Adds mycroft.skill.enable_intent and mycroft.skill.disable_intent
This defaults skill's configs to an empty dict instead of None,
which simplifies getting specific values in the config for skills.
It removes the check for None config.
==== Fixed Issues ====
==== Documentation Notes ====
Skill writers no longer need to check for None configs.
Modified the debugging message output after an utterance is complete, to
make it clear that metric data is only sent if opt_in is enabled.
This addresses #1494, which I filed in error, thinking that opt_in was
never checked (due to reading the log output and not knowing it was
checked elsewhere)
==== Fixed Issues ====
==== Localization Notes ====
Slight change to existing, unlocalized text
Previously if duplicates of a skill should be launched or a skill isn't properly
shutdown when reloaded multiple handlers could be registred. This commit disables multiples of repeating events.
An exception raised during the shutdown would previously cause the skill reload process to halt. This catches any exception and reports the error without halting the SkillManager
If super's shutdown was called before trying to cancel events an exception would be thrown since shutdown removed all registered events by setting self.events to None.
This replaces this with an empty list to allow skills to try to remove events/cancel events without incidents.
When a fresh image first updates the settings after pairing it is handled somewhat differently and this was missed in the original implementation of the callback handling.
This minor change includes this case as well.
If the scheduler was frozen for some time and the repeating event
scheduled time passes it would trigger the event constantly until the
next time is in the future again.
This will make the scheduler to disallow scheduling in the past and
instead schedule the next call one repeat period from now.
* Fix error message for enable_intent
The error was printed for each intent name mismatch instead of after all intents had been checked.
* Make sure intents aren't munged multiple times
Previously intents could be munged multiple times (This happened when enabling a disabled intent), resulting in an invalid name.
* Add test case for disable/enable intent
* Improve unmunging of messages
This make sure that only skill id's in the beginning of messages are removed and should speed up the process slightly
* Fix munging for register_vocab and register_regex
* Add testcases for register vocab and regex
When creating event handlers with add_event now by default there are no
messages about start and completion of the handler.
The handler info base name is now passed as an argument to the method
allowing for custom messages. skill intent handlers still report
skill.handler.start and skill.handler.complete but for example scheduled
events wont send these start/stop (but could instead trigger for example
skill.scheduled_event.start/complete)
* Update pyee to v5.0.0
The old version of pyee (1.0.1) could not remove events registered as "once" (instead of "on")
This fixes canceling scheduled events
* Restore MycroftSkill.remove_event() return value
The return statement in remove_event() was missing, probably lost while handling a conflict.
The data field of the message sent to the event handler may not always be a dictionary, (Example case Timer skill, which sets data to the timer name)
This validates the message type and the type of the data field before trying to unmunge.
* Skip skill update on startup if recent
Skills aren't updated on startup if the last update was less than 12
hours ago.
- msm adds a .msm file in the skills directory with a timestamp and a
list of the skills installed by default
- the UPDATING screen message is moved to the SkillManger when direct
update is scheduled.
- On internet connection the skill update time is scheduled
- Skills (other than priority skills) are not loaded until
- The timeout for when direct update is necessary is settable in the
config.
internet connection has been verified (message on messagebus)
All methods relating to loading vocabulary, dialog and regular
expressions has grown quite large. To make the core functionallity of
the skills more readable these are moved to the new module skill_data.
Additional method documentation has been addedi as well.
Convert keyword names to unique names by prepending the keyword with a
letter string derived from the unique skill id.
This commit modifies required keywords, optional keywords, one_of
keywords and regex matches.
This also munges the context keyword when that is sent to match the
intent correctly
* Made MycroftSkill.remove_event() return a bool, preventing unnecessary/misleading message from being posted by MycroftSkill.cancel_scheduled_event()
* More doc and several minor renames around intent processing
* Several minor typo and doc corrections
An error occured when testing skill settings when checking if the settings should be sent when trying to upload a field without a value. To guard against this there is a check if the field has a value before checking.
SkillManager now handles the skillmanager.list message and will reply with the
mycroft.skills.list message including a list of the loaded skills.
==== Protocol Notes ====
Added messages:
- skillmanager.list: skill manager send list of skills on messagebus
- mycroft.skills.list message with skill list
The bug was due to identifier collisions in the backend as well as settingsmeta schemas with a label field with no name attribute. This fix will make sure there are no more identifier collisions with already existing broken settingsmeta schemas in the database.
Withe the NTP checks in place, the sequence of visual and audio queues
was a little clunky. This refines it slightly for normal use and to
play better with the pairing process.
Raspberry Pi's don't have a built-in clock, so at boot-up the clock just picks up from when they were last running. Normally this is corrected very quickly by NTP from an internet server, but if there is no network connection that cannot happen.
When an out-of-the-box Mark 1 or Picroft is being setup, the clock is set to whenever the image was created. Upon completing the Wifi setup step the NTP service can finally sync with the internet, so time suddenly "jumps" to weeks later -- usually. In either case (when the date jumps or when the date is erronously months old), there is potential for havoc.
These changes deal with that situation. Upon network connection, an NTP synchonization is forced. If it is detected that a major time jump happened
(more than 1 hour), then the user is notified that the clock change requires
a reboot and the system restarts.
Other changes:
* use the new "system." message namespace
* add pause before the system.reboot during a WIPE, allowing reset to totally complete
*
- add_event() now accepts the parameter once, registring the event as a one shot event.
- remove_event for non-existing events is handled
- added a test for this
The speak method digs through the stack trying to find a Message object
and if found uses the context from that message when sending the data to
the speech subsystem.
==== Tech Notes ====
STT, intent handling, intent fallbacks, skill handlers are now timed and
tied together with a ident (consistent through the chain so the flow from
STT until completion of the skill handler can be follewed.
TTS execution time is also measured, right now this is not tied into the
ident due to the nature of the speech.
The report is always called "timing" and always contain the following
fields:
- id: Identifier grouping the metrics into interactions
- system: Which part (STT, intent service, skill handler, etc)
- start_time: timestamp for when the action started
- time: how long it took to execute the action
The different system adds their own specific information, for example
the intent_service adds the intent_type, i.e. which handler was matched.
==== Protocol Notes ====
mycroft.skills.loaded is sent togheter with skill id and skill name
whenever a skill is loaded. This is used in the intent_service to
convert from id to skill name when reporting
Add Python 2/3 compatibility
==== Tech Notes ====
This allows the main bus, skills and cli to be run in both python 2.7 and
3.5+.
Mainly trivial changes
- syntax for exceptions
- logic for importing correct Queue module
- .iteritems -> future.utils.iteritems when accessing dicts key value
pairs
* Allow audio service to be run in python 3
* Make speech client work with python 3
* Importing of Queue version dependent
* Exception syntax corrected
* Creating sound buffer is version dependant
- Adapt context use range from builtins
- Use compatible next() instead of .next() when walking the skill
directory
* Make CLI Python 3 Compatible
- Use compatible BytesIO instead of StringsIO
- Open files as text instead of binary
- Make sure integer divisions are used
* Make messagebus send compatible
* Fix failing travis
Re-add future 0.16.0
* Make string checks compatible
* basestring doesn't exist in python 3 so it's imported from the "past"
* Fix latest compatibility issues in speech client
- handle urllib
- handle encoding before calling md5
* Make Api.build_json() python 2/3 compatible
This method will load a translateable (and expandable) list of names
and values from the dialog/xx-xx/ folder of a skill. For example:
dialog/en-en/Colors.value
```
# List colors and their hex RGB values
alice blue, #F0F8FF
antique white, #FAEBD7
aqua, #00FFFF
```
When there is an error in settingsmeta.json, the load of the skill
would fail. Now it generates an error message but continues on.
Additionally the exception is caught and now displays information
about where the error in the JSON is (line and column).
* fixed identifier to be unique across all accounts
* now skills will load on http error and skills will upload once to web once identity2 exist
* extracted stuff out of __init__ so that skills work on load and skills get upload when identiy2 exists
* iniatiated class attribute to None
Several minor documentation changes, plus:
* 'cancel' now has to be an exact match
* Cancel events return None instead of the spoken cancelation string
* Reduced timeout to 10 seconds instead of 20
* Changed 'text' to 'announcement' and simplified logic
Since the garbage collection can be on a bit on the slow side it
shouldn't be done at a normal user's device.
A developer can still enable this by setting the configuration flag
"debug" to true.
The reference count sometimes reported that references were remaining
even if they actually weren't. By enforcing a garbage collector run
after shutting down a skill the false possitives are reduced if not
removed all together.
- Add support for unnamed intents.
- Add more debugging information for skill handler errors
- Clean up skill name for pronounciation
- Update docstring for initialize method
Corrected my refinement after a previous review.
Also added support for splitting the name of a skill before running it through
TTS, making "VolumeSkill" sound like "Volume Skill" and such. Plus a log
message before raising some errors in the skill wrapper.
This allows skill writers to ignore naming intents. Combined with a
forthcoming change to Adapt that creates a default of None for IntentBuilder()
This allows the current:
@intent_handler(IntentBuilder("CurrentWeatherIntent").require(
"Weather").optionally("Location").build())
def handle_current_weather(self, message):
...
To become:
@intent_handler(IntentBuilder().require("Weather").optionally("Location"))
def handle_current_weather(self, message):
...
Which will automatically name the Intent "handle_current_weather".
Also dropped the log message in the default initialize() method since it is
common to not override it now.
==== Tech Notes ====
queue command will start playback if no playback is running.
mpg123 and vlc services updated support queueing tracks while playing
==== Protocol Notes ====
mycroft.audio.service.queue message added
A Skill can now implement the get_intro_message() method, which can return a
string to be spoken the very first time a skill is run. This is intended to
be an announcement, such as further steps necessary to setup the skill.
Also stopped generation of the Error message when the expected StopIteration
occurs on a intent failure. This confused new developers and poluted the logs.
Finally, corrected some documentation typos.
==== Tech Notes ====
This allows you to remove message bus messages inside core.py. When canceling scheduling events, the message bus messages were not removed which could caused duplicate listeners and handlers for the same intent. Adding remove_event function removes the actual messages from the bus to prevent potential duplicates.
==== Tech Notes ====
Location was added as a default context keyword when the context manager
was added as an example of how the context feature could be used.
However in the current greedy implementation in can cause some confusion
with lingering context providing incorrect Location.
The feature can still be turned on in configuration if someone wants to
experiment with it.
The prompt during skill downloads was occurring even when the "speak" flag was
set to False. Now it is honored.
Also removed the "no network connection.dialog" which essentially was a copy of
the "not connected to the internet.dialog" file.
Steve Penrod
Steve Penrod
Åke
jarbasal
Michael Nguyen
Matthew D. Scholefield
Julien Kassar
JarbasAI
f-e-l-i-x
Matthew D. Scholefield
Sergio Oller
Greg Cowan
InconsolableCellist
Zachary T Welch
gaurav
Anu