Commit Graph

656 Commits (65fcfcfcff67248829230823f6af91c5b14ce413)

Author SHA1 Message Date
Åke 98ede52596
Merge pull request #1572 from MycroftAI/feature/py_msm
Cleanup skill loading and implement python msm
2018-05-09 23:25:40 +02:00
Matthew D. Scholefield 6ecf4d986f Reinstall pip dependencies if virtualenv is removed
This keeps track of the skills whose dependencies have already been installed. While it won't automatically register newly installed skills, it will attempt to reinstall dependencies the next boot only one time so it shouldn't be a big issue.
2018-05-09 16:07:04 -05:00
Matthew D. Scholefield e86e0546d7 Make settings meta hashes consistent
str(dict) in python in implementation defined. json.dumps(..., sort_keys=True) is not
2018-05-08 17:17:45 -05:00
Matthew D. Scholefield c301a3ca4d Change skill settings hash to use skill name
This prevents issues with the skill settings being dependent on skill folders. This is an issue because the new msm has a new skill naming scheme
2018-05-08 16:23:53 -05:00
Matthew D. Scholefield a20b5b565c Cleanup skill loading and implement python msm 2018-05-08 13:41:51 -05:00
Åke Forslund 61aeeb8c00 Fix issue using basestring
Checking against basestring was necessary in python two since unicode and string were separate classes. In python3 basestring was removed since different string classes have been removed.
2018-05-04 11:42:13 +02:00
Åke Forslund 8c5911ecf0 Fix code standard issues
- dev_setup.sh - replace `` with $()
- mycroft/skills/core.py remove remove redundant imports
- mycroft/tts/__init__.py remove remove redundant imports
2018-05-03 12:32:21 +02:00
Åke Forslund 5d50d22061 Fix listing skills
python3 dict_keys can't be serialized to json, and needs to be
explicitly converted to list
2018-05-02 08:35:17 +02:00
Matthew D. Scholefield 82a61c76c7 Fix infinite skill reloading by disabling folder checks
Python 3 modifies the __pycache__ folder which makes the parent folders update their timestamps causing an infinite skill reload
2018-04-27 11:44:07 -05:00
Matthew D. Scholefield 0abc3c78b1 Fix counting function arguments in Python 3 2018-04-27 11:42:54 -05:00
Åke Forslund 5447983999 Use sorted json to perform hash of settings 2018-04-27 08:57:48 -05:00
Åke Forslund a2993e4ba6 Remove backwards compatibility with python 2.7 2018-04-27 08:51:47 -05:00
Åke Forslund b200d51d39 Fix test cases under python 3
Lots of minor fixes including, sorting dicts, making ints of strings,
     MagicMock file spec and some other things

A couple of issues in the mycroft-core code base were identified and
fixed. Most notably the incorrect version check for python three when
adding basestring.

Update .travis.yml
2018-04-27 08:50:46 -05:00
Åke Forslund 8840a43886 Fix hashes in settings for python3
convert strings to bytes before hashing
2018-04-27 08:50:46 -05:00
Sergio Oller 5e8d8eb1a6 Feature: Allow comments in voc and regex files
When loading voc and regex files, lines starting with "#" are now
ignored, so developers and translators can use them to document their
decisions.

==== Fixed Issues ====
2018-04-14 21:53:14 +02:00
Åke aa4261252b
Merge pull request #1519 from MycroftAI/feature/stop-sigint
Shutdown improvements
2018-04-12 12:07:02 +02:00
Åke Forslund d61e3c2dd0 Extra safety around skill specific shutdown
Catch error in shutdown method of skill and continue with general shutdown
2018-04-12 11:11:04 +02:00
Matthew D. Scholefield fa11ac7dbe Rename MycroftSkill.shutdown to MycroftSkill._shutdown
This removes the need for skills to call Super(MySkill, self).shutdown()
2018-04-11 21:29:14 -05:00
Matthew D. Scholefield 10bd9a1cf3 Change initial kill to SIGINT
This sends a ctrl+c signal to each process which will allow code to exit properly by handling KeyboardInterrupt
Other notable changes:
 - create_daemon method used to clean up create daemon threads
 - create_echo_function used to reduce code duplication with messagebus
 echo functions
 - wait_for_exit_signal used to wait for ctrl+c (SIGINT)
 - reset_sigint_handler used to ensure SIGINT will raise KeyboardInterrupt
2018-04-11 21:29:10 -05:00
Matthew D. Scholefield 2e005a99e8 Support triggering skill settings update via a message 2018-04-11 15:24:07 -05:00
Matthew D. Scholefield 8cd8eb65b1 Decrease polling to 60 seconds
Now that the server uses ETags, it should be fine with polling every minute
2018-04-11 15:24:07 -05:00
Michael Nguyen 3e2f008c12
Merge pull request #1523 from forslund/bugfix/shutdown-sheduled-events
Fix cancel_all_repeating_events()
2018-04-10 11:15:11 -05:00
Åke 6d339039de
Merge pull request #1524 from forslund/feature/Jarbas-messagebus-intent-control
Add enable/disable intent using the messagebus.
2018-04-06 13:37:24 +02:00
Åke Forslund 58325a83fd Fix cancel_all_repeating_events
Unnamed repeating events caused an exception in
cancel_all_repeating_events, this makes sure the event name is stored
correctly and fixes the issue.
2018-04-06 11:34:36 +02:00
Jarbas 9f1c5e4cbf External enable/disable intent
Allow intents to be enabled or disabled from outside of the skill via
the messagebus.

Adds mycroft.skill.enable_intent and mycroft.skill.disable_intent
2018-04-05 23:29:44 +02:00
Greg Cowan dfa9a65e96 Default skill config to empty dict (#1521)
This defaults skill's configs to an empty dict instead of None,
which simplifies getting specific values in the config for skills.
It removes the check for None config.

==== Fixed Issues ====

====  Documentation Notes ====
Skill writers no longer need to check for None configs.
2018-04-04 23:12:55 -05:00
Åke bc74c36713
Merge pull request #1508 from MycroftAI/feature/notify-backend-down
Add proper messages when the backend replies incorrectly
2018-03-29 12:53:34 -05:00
Matthew D. Scholefield eb100706df Add proper messages when the backend replies incorrectly 2018-03-28 11:48:05 -05:00
Michael Nguyen 2caf77bc9d
Merge pull request #1481 from forslund/feature/repeating-event-safety
Feature/repeating event safety
2018-03-27 19:15:32 -05:00
Åke eac0d81e94
Merge pull request #1502 from JarbasAl/bug_fix_audio_service_old_class_style
AudioService - no old style class
2018-03-23 13:17:04 -05:00
jarbasai 8cf915b83e no old style class 2018-03-23 15:07:21 +00:00
InconsolableCellist 7447dbd5ea Simple log output change to prevent confusion (#1497)
Modified the debugging message output after an utterance is complete, to
make it clear that metric data is only sent if opt_in is enabled.

This addresses #1494, which I filed in error, thinking that opt_in was
never checked (due to reading the log output and not knowing it was
checked elsewhere)

==== Fixed Issues ====

==== Localization Notes ====
Slight change to existing, unlocalized text
2018-03-22 13:18:11 -05:00
Michael Nguyen d6b4037258 remove unused function in skill settings 2018-03-21 14:49:31 -05:00
Åke Forslund 764d200d1b Only allow one repeating even of each type
Previously if duplicates of a skill should be launched or a skill isn't properly
shutdown when reloaded multiple handlers could be registred. This commit disables multiples of repeating events.
2018-03-15 12:39:25 +01:00
Åke Forslund 6dae1bb328 Do not reschedule an active repeating event
- Hinder multiple events with the same name to be registered
- Automatically cancel repeating events when skill is shutdown
2018-03-15 11:42:08 +01:00
Steve Penrod 8ef691ffb7
Slowing the skill setting poll to once every 5 min
Experiment to see if this is overloading the backend.
2018-03-14 03:37:57 -05:00
Michael Nguyen b884378fbb
Merge pull request #1466 from forslund/feature/quieter-skill-fetching-error
Reduce verbosity when skill settings fetch fails.
2018-03-11 12:34:07 -05:00
Åke Forslund 65cba626fd Reduce verbosity when skill settings fetch fails.
Replace the stack trace with a shorter message, briefly printing the cause.
2018-03-11 17:05:52 +01:00
Åke 2c151fe594
Merge pull request #1458 from MycroftAI/refactor/handler-args
Refactor event handler argument calling
2018-03-08 08:29:40 +01:00
Åke Forslund b4ed9369fd Handle exception in skill shutdown
An exception raised during the shutdown would previously cause the skill reload process to halt. This catches any exception and reports the error without halting the SkillManager
2018-03-07 13:23:45 -06:00
Åke Forslund 11bae2aea3 Fix issue in skill shutdown.
If super's shutdown was called before trying to cancel events an exception would be thrown since shutdown removed all registered events by setting self.events to None.

This replaces this with an empty list to allow skills to try to remove events/cancel events without incidents.
2018-03-07 13:23:45 -06:00
Matthew D. Scholefield 0962a3b6af Fix potential crash in add_event 2018-03-07 10:13:20 -06:00
Matthew D. Scholefield 4131730089 Refactor handler argument calling 2018-03-07 10:12:13 -06:00
Åke 2719f96950
Merge pull request #1454 from MycroftAI/bugfix/intent-mixups
Use function attributes for intent decorators
2018-03-01 08:37:45 +01:00
Åke 5a68908e80 Call callback when remote settings are inited (#1452)
When a fresh image first updates the settings after pairing it is handled somewhat differently and this was missed in the original implementation of the callback handling.

This minor change includes this case as well.
2018-03-01 01:07:55 -06:00
Matthew D. Scholefield 881687012e Train Padatious on mycroft.skills.initialized (#1455)
Fix sporadic issues from training prematurely.
2018-02-28 21:55:14 -06:00
Matthew D. Scholefield 07ce2d98d0 Use function attributes for intent decorators
This prevents needing to use a shared list which misbehaves when multiple skills initialize at once
2018-02-28 19:13:33 -06:00
Matthew D. Scholefield cdb1416252 Check for pairing before performing routine polling (#1451)
This gets rid of error messages during the pairing process
2018-02-28 15:41:23 -06:00
Zachary T Welch a7044f9967 only NTP sync on certain enclosures 2018-02-27 18:00:33 -06:00
Åke 743082734c Fix multiple triggering of repeating events (#1435)
If the scheduler was frozen for some time and the repeating event
scheduled time passes it would trigger the event constantly until the
next time is in the future again.

This will make the scheduler to disallow scheduling in the past and
instead schedule the next call one repeat period from now.
2018-02-27 16:37:29 -06:00
Matthew D. Scholefield ce908b6098 Add message when skills are initialized (#1449)
Posts ```mycroft.skills.initialized``` once the initial skill load completes.
2018-02-27 16:22:40 -06:00
Matthew D. Scholefield efc1c1d71e Wait while speaking before running __get_response (#1432)
This fixes issues with long TTS responses
2018-02-23 01:49:06 -06:00
Åke b58a533e39 Bugfix/munging related issues (#1434)
* Fix error message for enable_intent

The error was printed for each intent name mismatch instead of after all intents had been checked.

* Make sure intents aren't munged multiple times

Previously intents could be munged multiple times (This happened when enabling a disabled intent), resulting in an invalid name.

* Add test case for disable/enable intent

* Improve unmunging of messages

This make sure that only skill id's in the beginning of messages are removed and should speed up the process slightly

* Fix munging for register_vocab and register_regex

* Add testcases for register vocab and regex
2018-02-23 00:51:55 -06:00
Matthew D. Scholefield b57165477c Fix self parameter in decorator functions with single argument 2018-02-15 17:33:03 -06:00
Åke Forslund 7970f5bc9d optional start/completed messages for add_event
When creating event handlers with add_event now by default there are no
messages about start and completion of the handler.

The handler info base name is now passed as an argument to the method
allowing for custom messages. skill intent handlers still report
skill.handler.start and skill.handler.complete but for example scheduled
events wont send these start/stop (but could instead trigger for example
        skill.scheduled_event.start/complete)
2018-02-15 15:31:33 -06:00
Åke ea7c7efee1 Fix cancel_scheduled_event for non-repeating events by updateing pyee to v5.0.0 (#1425)
* Update pyee to v5.0.0

The old version of pyee (1.0.1) could not remove events registered as "once" (instead of "on")
This fixes canceling scheduled events

* Restore MycroftSkill.remove_event() return value

The return statement in remove_event() was missing, probably lost while handling a conflict.
2018-02-15 14:52:18 -06:00
Åke b4c6f63352 Validate message before trying to demunge (#1427)
The data field of the message sent to the event handler may not always be a dictionary, (Example case Timer skill, which sets data to the timer name)

This validates the message type and the type of the data field before trying to unmunge.
2018-02-15 14:11:21 -06:00
penrods f54eacbd53 Add default when for schedule_repeating_event()
MycroftSkill.schedule_repeating_event(handler, when, frequency) now
will compute the 'when' as now+frequency seconds if None is passed.
2018-02-15 05:40:38 -06:00
Åke 7fd63f9d5a
Merge pull request #1421 from MycroftAI/feature/doc_cleanup
Internal documentation, typos, remove misleading message
2018-02-15 10:51:33 +01:00
Åke 3271ff3e24
Merge pull request #1420 from MycroftAI/feature/cli_declutter
Reduce CLI log clutter from settings, etc
2018-02-15 10:31:51 +01:00
Åke ed6ab224c9
Merge pull request #1422 from forslund/feature/munge-keywords
Make keywords per skill to fix overlapping keyword names
2018-02-15 09:57:03 +01:00
Steve Penrod 8139727714
Merge branch 'dev' into feature/doc_cleanup 2018-02-15 02:48:10 -06:00
Åke bda8a0cc94 Skip skill update on startup if recent (#1392)
* Skip skill update on startup if recent

Skills aren't updated on startup if the last update was less than 12
hours ago.

- msm adds a .msm file in the skills directory with a timestamp and a
    list of the skills installed by default
- the UPDATING screen message is moved to the SkillManger when direct
    update is scheduled.
- On internet connection the skill update time is scheduled
- Skills (other than priority skills) are not loaded until
- The timeout for when direct update is necessary is settable in the
    config.
internet connection has been verified (message on messagebus)
2018-02-15 02:43:06 -06:00
Steve Penrod 23302b60d9 Add copyright notice and minor docstring changes 2018-02-15 09:29:04 +01:00
Åke Forslund 378d353572 Move data loading helpers to separate file.
All methods relating to loading vocabulary, dialog and regular
expressions has grown quite large. To make the core functionallity of
the skills more readable these are moved to the new module skill_data.

Additional method documentation has been addedi as well.
2018-02-15 09:28:27 +01:00
Åke Forslund 51ed5cd810 Munge keywords for intents
Convert keyword names to unique names by prepending the keyword with a
letter string derived from the unique skill id.

This commit modifies required keywords, optional keywords, one_of
keywords and regex matches.

This also munges the context keyword when that is sent to match the
intent correctly
2018-02-15 09:26:35 +01:00
penrods 2af217bed0 Wrote and cleaned up more docstrings
Made docstrings more consistent with the preferred 'Google Style Docstring'
standard:
http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
2018-02-14 17:25:57 -06:00
Michael Nguyen 50c7ab6c6e
Merge pull request #1364 from forslund/bugfix/scheduled-event
Bugfix/scheduled event
2018-02-13 22:34:51 -06:00
penrods fa589fead4 Quiet PEP8 2018-02-13 03:05:43 -06:00
penrods c68ad44b1c Internal documentation, typos, remove misleading message
* Made MycroftSkill.remove_event() return a bool, preventing unnecessary/misleading message from being posted by MycroftSkill.cancel_scheduled_event()
* More doc and several minor renames around intent processing
* Several minor typo and doc corrections
2018-02-13 02:54:12 -06:00
penrods 920845c407 Reduce CLI log clutter from settings, etc
* Drop settings INFO messages to DEBUG
* Add DEBUG filter in the CLI by default
2018-02-13 02:40:44 -06:00
Åke Forslund b632377e0e Check that value exists before checking for change
An error occured when testing skill settings when checking if the settings should be sent when trying to upload a field without a value. To guard against this there is a check if the field has a value before checking.
2018-02-02 15:03:41 +01:00
Michael Nguyen 9c80112fd9 added fix for new schema from backend 2018-02-01 18:44:21 -06:00
Åke 0e3acbada4 add :skills command to list skills in cli (#1395)
SkillManager now handles the skillmanager.list message and will reply with the
mycroft.skills.list message including a list of the loaded skills.

==== Protocol Notes ====
Added messages:
- skillmanager.list: skill manager send list of skills on messagebus
- mycroft.skills.list message with skill list
2018-02-01 02:14:38 -06:00
Åke 2309c9cd33
Merge pull request #1397 from MycroftAI/bugfix/skill-settings-disappearing
The bug was due to identifier collisions in the backend as well as settingsmeta schemas with a label field with no name attribute. This fix will make sure there are no more identifier collisions with already existing broken settingsmeta schemas in the database.
2018-02-01 08:56:21 +01:00
Åke b8893a7643 Ignore .files when checking for skill modification (#1378)
This ignores all files starting with . when checking for modifications
of skills
2018-02-01 01:53:57 -06:00
Michael Nguyen 9af4b95650 modified identifier, add .mypy_cache to git ignore 2018-01-31 09:42:44 -06:00
Åke Forslund acdeb0853e Minor fix for code coherency 2018-01-31 15:26:24 +01:00
jarbasai 98960012bb add skill load fail message 2018-01-31 14:58:04 +01:00
jarbasai 93860da4a0 loaded skills bus messages 2018-01-31 14:57:50 +01:00
Åke Forslund aab426a140 Add settable callback on change from backend
If settings are updated a callback can be set to notify the skill that a
change has occured.
2018-01-23 17:12:09 +01:00
penrods 26444c9c5f Tweak UI for NTP checking
Withe the NTP checks in place, the sequence of visual and audio queues
was a little clunky.  This refines it slightly for normal use and to
play better with the pairing process.
2018-01-18 14:44:53 -06:00
penrods a6bfed268b Fix Codacy gripe and add new .dialog 2018-01-17 21:16:39 -06:00
penrods 6c1cdb47b3 Fix startup time-warp issue
Raspberry Pi's don't have a built-in clock, so at boot-up the clock just picks up from when they were last running.  Normally this is corrected very quickly by NTP from an internet server, but if there is no network connection that cannot happen.

When an out-of-the-box Mark 1 or Picroft is being setup, the clock is set to whenever the image was created.  Upon completing the Wifi setup step the NTP service can finally sync with the internet, so time suddenly "jumps" to weeks later -- usually.  In either case (when the date jumps or when the date is erronously months old), there is potential for havoc.

These changes deal with that situation.  Upon network connection, an NTP synchonization is forced.  If it is detected that a major time jump happened
(more than 1 hour), then the user is notified that the clock change requires
a reboot and the system restarts.

Other changes:
* use the new "system." message namespace
* add pause before the system.reboot during a WIPE, allowing reset to totally complete
*
2018-01-17 21:05:51 -06:00
Åke Forslund 6c4f485323 Make handlers for single scheduled events one shot
- add_event() now accepts the parameter once, registring the event as a one shot event.
- remove_event for non-existing events is handled
- added a test for this
2018-01-16 17:39:51 +01:00
Åke Forslund f88acdb60f Fix converse handling
Fallbacks were triggered even if utterance had been handled by converse.
This checks if converse handled the utterance as well.
2018-01-08 18:01:03 -06:00
Åke Forslund 936bd54180 Add a function to build the standard timing report 2018-01-08 18:01:03 -06:00
Åke Forslund fe3b85c755 Add tts into the flow by digging throug the stack
The speak method digs through the stack trying to find a Message object
and if found uses the context from that message when sending the data to
the speech subsystem.
2018-01-08 18:01:03 -06:00
Åke Forslund baa15b98aa Send timing metrics for various subsystems
====  Tech Notes ====
STT, intent handling, intent fallbacks, skill handlers are now timed and
tied together with a ident (consistent through the chain so the flow from
STT until completion of the skill handler can be follewed.

TTS execution time is also measured, right now this is not tied into the
ident due to the nature of the speech.

The report is always called "timing" and always contain the following
fields:

- id: Identifier grouping the metrics into interactions
- system: Which part (STT, intent service, skill handler, etc)
- start_time: timestamp for when the action started
- time: how long it took to execute the action

The different system adds their own specific information, for example
the intent_service adds the intent_type, i.e. which handler was matched.

==== Protocol Notes ====
mycroft.skills.loaded is sent togheter with skill id and skill name
whenever a skill is loaded. This is used in the intent_service to
convert from id to skill name when reporting
2018-01-08 18:01:03 -06:00
Michael Nguyen 5d842fd369 removed log line 2018-01-08 17:02:49 -06:00
Michael Nguyen 884b65f0e4 fixed settings disappearing on load 2018-01-08 17:02:49 -06:00
Åke Forslund b19e73025f Minor cleanup 2018-01-03 10:09:34 +01:00
jarbasai 6bba0f0ae4 Optional auto update 2018-01-03 10:09:25 +01:00
Michael Nguyen 8e2c97484c fixed skill settings migration for web ui 2017-12-20 15:59:09 -06:00
Åke 742558046d Python 2/3 compatibility (#1259)
Add Python 2/3 compatibility

====  Tech Notes ====
This allows the main bus, skills and cli to be run in both python 2.7 and
3.5+.

Mainly trivial changes
- syntax for exceptions
- logic for importing correct Queue module
- .iteritems -> future.utils.iteritems when accessing dicts key value
pairs

* Allow audio service to be run in python 3
* Make speech client work with python 3
* Importing of Queue version dependent
* Exception syntax corrected
* Creating sound buffer is version dependant
- Adapt context use range from builtins
- Use compatible next() instead of .next() when walking the skill
directory

* Make CLI Python 3 Compatible
- Use compatible BytesIO instead of StringsIO
- Open files as text instead of binary
- Make sure integer divisions are used

* Make messagebus send compatible
* Fix failing travis

Re-add future 0.16.0

* Make string checks compatible
* basestring doesn't exist in python 3 so it's imported from the "past"
* Fix latest compatibility issues in speech client
- handle urllib
- handle encoding before calling md5

* Make Api.build_json() python 2/3 compatible
2017-12-18 17:24:21 -06:00
Åke Forslund 9fa4b4e138 Empty except->except Exception:
This fixes pep8-speak issue
2017-12-15 12:40:41 +01:00
penrods de328a43d9 Fix PEP8 2017-12-15 05:19:11 -06:00
penrods 0385f46fcc Add MycroftSkill.translate_namedvalues()
This method will load a translateable (and expandable) list of names
and values from the dialog/xx-xx/ folder of a skill.  For example:

dialog/en-en/Colors.value
```
# List colors and their hex RGB values
alice blue, #F0F8FF
antique white, #FAEBD7
aqua, #00FFFF
```
2017-12-15 05:07:16 -06:00
Michael Nguyen 4b7d6e5526 Feature/overwritten settings (#1304)
prevent settings initalization from skills __init__ to override settings.json values
2017-12-13 23:25:33 +01:00
Michael Nguyen 0bbb05ddd2
Merge pull request #1300 from MycroftAI/feature/meta-crashes
Protect against bad settingsmeta.json
2017-12-13 10:50:54 -06:00