Commit Graph

92 Commits (mark-ii/log-rotation)

Author SHA1 Message Date
Matthew D. Scholefield 14ceb1f1dd Remove all references to /opt/mycroft/skills
The only thing that should reference it is the config
2018-05-23 08:22:14 +02:00
CarstenAgerskov a353c43748 Fixes for Python3 migration 2018-05-21 18:45:47 -05:00
Åke Forslund b9cb58be79 Fix single_test.py
Update metaclass declaration to python3 syntax
2018-05-21 15:42:16 +02:00
Åke Forslund 580c053e22 Use skill id similar to the normal loading
Fixes issues with skills using regexes.
2018-05-21 15:21:38 +02:00
Matthew D. Scholefield 7ef551ddc6 Fix error with skills test descriptor 2018-05-14 14:27:37 -05:00
Matthew D. Scholefield 0850f5073a Fix tests for Python 3
- Change print statements
 - Change queue import
2018-05-14 14:27:37 -05:00
Åke Forslund 40fd0487d0 Clean up output slightly
The converse request / response messages are now hidden.
2018-05-14 14:27:37 -05:00
Åke Forslund 83a6be7fcb Override get_response to handle conversations
Adds the possibility to add a responses list in the test case. The test
will respond with each of the entries in the list to get_response
requests.

Ex:
{
  [...]
  "responses": ["yes", "Miami"]
}
2018-05-14 14:27:37 -05:00
Åke Forslund ce410c2f0c Fix expected_data
expected_data would fail in combination with Intent tests, this
differentiates it from the other rules
Add message type as "__type__" in the data for processing using
expected_data.
2018-05-14 14:27:37 -05:00
Åke Forslund c8b58f3f37 Inject message type as __type__ in the test data 2018-05-14 14:27:37 -05:00
Åke Forslund dc58402f39 Fix message_tester.py
Make skill optional and disable expected_dialog if missing.
2018-05-14 14:27:37 -05:00
Åke Forslund e6017ec496 Fix issues detected by codacy 2018-05-14 14:27:37 -05:00
Åke Forslund 4a7cf45ada Add possibility to add a custom test runner
The custom test runner can be used for mocking third-party applications
or services.

To use create a __init__.py in the SKILL_DIR/test

The base of the file should look something like:

```python
from test.integrationtests.skills.skill_tester import SkillTest

def skill_runner(skill, example, emitter, loader, m1, m2):
    return SkillTest(skill, example, emitter).run(loader)
```

Then the skill_runner can be decorated by `mock.patch` to mock out resources.
2018-05-14 14:27:37 -05:00
Åke Forslund cab556450e Add expected_data option
Allows the intent test to check expected data content.

Example:

{
  "utterance": "set a weekend alarm at  9 am",
  "expected_data": {
      "ampm": "am",
      "time": "9",
      "daytype": "weekend"
  },
  "expected_response": "Okay. Setting a .* alarm"
}
2018-05-14 14:27:37 -05:00
Åke Forslund 9c4ac178b9 Give better feedback when skill isn't loaded
If a skill required for a test case isn't loaded an exception will be
raised.
2018-05-14 14:27:37 -05:00
Åke Forslund 3917726bb5 Use queue to get skill completion
This allows the skill test to exit directly after skill handler
completes.
2018-05-14 14:27:37 -05:00
Åke Forslund e7ead7ca6e Reduce timeout when skill has been handled 2018-05-14 14:27:37 -05:00
Matthew D. Scholefield 79bf94e48e Speed up intent tests by responding to converse requests 2018-05-14 14:27:37 -05:00
Åke Forslund 02565efd59 Use pytest to discover test cases 2018-05-14 14:27:37 -05:00
Matthew D. Scholefield 154ccd59d7 Expose single threaded training option from Padatious Service
This is necessary to prevent a deadlock when training during intent tests using pytest
2018-05-14 14:27:37 -05:00
Åke Forslund 921bc5f23d Add support for "expected dialog"
Loads dialogs and checks if any skills responds with one of the possible
dialogs.
2018-05-14 14:27:37 -05:00
Åke Forslund d10ba29f70 Fix pep8 issues 2018-05-14 14:27:37 -05:00
Åke Forslund 3fcb40c2bb Add new rule 'exists'
Rule is satisfied if the entry exists in the data field not taking into
account it's value at all:

Example:
{
    "utterance": "set an alarm on july 4th 2016 at 3pm",
    "evaluation_timeout": 10,
    "assert": "[['and', ['exists', 'ampm'], ['exists', 'time'] ]]"
}
2018-05-14 14:27:37 -05:00
Åke Forslund 90e719c29e Add padatious and fallbacks 2018-05-14 14:27:37 -05:00
Åke Forslund 3a9242c1c7 Allow utterance without intent_type and intents 2018-05-14 14:27:37 -05:00
Åke Forslund a206807606 Add simple script for running a single skill
Based on the skill_developers_testrunner.py, takes the path to a skill
as argument and runs any tests for that skill.
2018-05-14 14:27:37 -05:00
CarstenAgerskov 8ab30500d7 Cleaner and better format for examples in message_tester 2018-05-14 14:27:37 -05:00
CarstenAgerskov 1cbedaea3c Added test_all_skills. More doc. 2018-05-14 14:27:37 -05:00
CarstenAgerskov 6c226ea4d9 Added message tester utility. Reduced line length to <80 chars in general 2018-05-14 14:27:37 -05:00
CarstenAgerskov a16c2a0ecc Added support for context, mycroft response and more. 2018-05-14 14:27:37 -05:00
CarstenAgerskov 7e9a05f4e0 First cut at skill tester
Minimum viable solution for running existing testcases (json files).
Currently considered additions:
(16, 3) # TODO: Make template for testing one skill only, for the skill
developer to use
(138, 11) # TODO: Pass something to intent, that tells that this is a
test run. The skill intent can then avoid side effects
(144, 11) # TODO: add optional timeout parameter to test_case
(155, 11) # TODO: Check that all intents are checked (what about
context)
(169, 3) # TODO: Add command line utility to test an event against a
test_case, allow for debugging tests
(173, 11) # TODO: Add support for expected response, and others
2018-05-14 14:27:37 -05:00
Matthew D. Scholefield fa11ac7dbe Rename MycroftSkill.shutdown to MycroftSkill._shutdown
This removes the need for skills to call Super(MySkill, self).shutdown()
2018-04-11 21:29:14 -05:00
penrods 8f2e5d9498 Change to Apache 2.0 license from GPLv3.0
This commit officially switches the mycroft-core repository from
GPLv3.0 licensing to Apache 2.0.  All dependencies on GPL'ed code
have been removed and we have contacted all previous contributors
with still-existing code in the repository to agree to this change.

Going forward, all contributors will sign a Contributor License
Agreement (CLA) by visiting https://mycroft.ai/cla, then they will
be included in the Mycroft Project's overall Contributor list,
found at: https://github.com/MycroftAI/contributors.  This cleanly
protects the project, the contributor and all who use the technology
to build upon.

Futher discussion can be found at this blog post:
https://mycroft.ai/blog/right-license/

This commit also removes all __author__="" from the code.  These
lines are painful to maintain and the etiquette surrounding their
maintainence is unclear.  Do you remove a name from the list if the
last line of code the wrote gets replaced?  Etc.  Now all
contributors are publicly acknowledged in the aforementioned repo,
and actual authorship is maintained by Github in a much more
effective and elegant way!

Finally, a few references to "Mycroft AI" were changed to the correct
legal entity name "Mycroft AI Inc."

==== Fixed Issues ====
#403 Update License.md and file headers to Apache 2.0
#400 Update LICENSE.md

====  Documentation Notes ====
Deprecated the ScheduledSkill and ScheduledCRUDSkill classes.
These capabilities have been superceded by the more flexible MycroftSkill
class methods schedule_event(), schedule_repeating_event(), update_event(),
and cancel_event().
2017-10-04 01:28:44 -05:00
Matthew D. Scholefield 5e392f34aa Optimize imports
Remove unused imports and group local vs external alphabetically
2017-09-18 16:07:50 -05:00
Åke Forslund 12bf78cd5f Move funcs only used in skill_tester from core.py
====  Tech Notes ====
Some functions have been orphaned in core.py and are only used in the
tests. To clean up these have been moved to where they're used.
2017-08-21 15:59:48 +02:00
harika 173bee9f95 Fixed pep8 format 2017-08-17 02:05:02 -05:00
harika 4c3dd3aa7c Updating skill_tester to automate skills testing(regression) 2017-08-17 02:05:02 -05:00
harika 632df2a498 Update tests 2017-08-17 02:05:02 -05:00
harika e2811ccde2 added some stuffs --arron says it's awesome! 2017-08-17 02:05:02 -05:00
Åke Forslund c8dd1a9b46 Make discover test directory settable
The skill directory can be specified when calling the discovery_test

`PYTHONPATH=./ python test/integrationtests/skills/discovery_test.py
 SKILL_DIR`
2017-08-17 02:05:02 -05:00
Åke Forslund f0a7a71e01 Fix shutdown error.
None type can be in skill list
2017-08-17 02:05:02 -05:00
Åke Forslund 39d0451143 Separate tests into unittests and integrationtests 2017-08-17 02:05:02 -05:00