Commit Graph

20 Commits (mark-ii/qa)

Author SHA1 Message Date
Åke Forslund e2a7fe9b2a Load test environment with importlib instead of imp 2020-08-21 08:00:57 +02:00
Åke Forslund 8a77892821 Fix test_setup issue in discover_test.py 2018-09-26 09:58:52 +02:00
Åke Forslund 788090ad6a Add support for a test_setup() function
The test setup function will be run after the skill is loaded but before the testing starts.

This provides a place to setup things that is standard for all test cases.
2018-09-07 18:51:30 +02:00
Steve Penrod dd807b2b2e
Improve the standalone skill tester (#1683)
* Improve the standalone skill tester

Using the skill tester was difficult -- the skill author had to copy a file
locally, and figuring out what was wrong with a test wasn't obvious.  This
change improves it in several ways:
* The tester can now be run as a module, allowing any skill to be tested
  by entering the skill folder and running:
  ```
  python -m test.integrationtests.skills.runner
  ```
  Optionally you can pass along a path to the skill.
* The runner will display help with a '--help' parameter, pointing to
  documentation on creating the tests.
* Information on where the tests are expected is printed during execution
* The *.intent.json was reduced to simply *.json since this is under an
  ```test/intent/``` folder already.  (This is backwards compatible for
  existing intent tests.)
* The failing rule is now displayed at the bottom of the run report, making
  it easier to figure out where issues exist during test creation
* Headers and terminal colors are used in the output, making it easier to
  visually parse the output from the execution of tests.

====  Documentation Notes ====
Update the skill documentation to reflect using
  python -m test.integrationtests.skills.runner
instead of copying the skill_developers_testrunner.py.

NOTE: This does have to be performed within a developer venv in order to
access the test suite.  Consider moving this into a mycroft-core-dev package
in the future for Mark 1 / Picroft users.

* Quieting warning from Codacy

* Replace single_test with runner

Adds support for "test_env" to the runner script

* Update the discover_test to match the runner

- catch intent tests from *.json files
- add failure msg to assert

* Turn off color using MST_NO_COLOR env variable
2018-07-19 02:23:22 -05:00
Matthew D. Scholefield a5b3b6f9e4 Add support for SKILL_DIR variable to execute just one skill test 2018-05-31 19:18:15 -05:00
Matthew D. Scholefield cc582ce38c Fix starting skillstest via start-mycroft.sh
This also removes the ability to pass the skills dir as the first argument. Instead this was moved to an environment variable called SKILLS_DIR to prevent conflict with pytest arguments
2018-05-31 19:17:26 -05:00
Åke Forslund e7a6a20967 Sort test cases so order is enforced 2018-05-28 15:32:56 +02:00
Åke Forslund d99377dc78 Fix discover_test get_skills_dir()
The discover_test didn't handle configurable data path correctly.
2018-05-28 09:10:35 +02:00
Matthew D. Scholefield 14ceb1f1dd Remove all references to /opt/mycroft/skills
The only thing that should reference it is the config
2018-05-23 08:22:14 +02:00
Åke Forslund 4a7cf45ada Add possibility to add a custom test runner
The custom test runner can be used for mocking third-party applications
or services.

To use create a __init__.py in the SKILL_DIR/test

The base of the file should look something like:

```python
from test.integrationtests.skills.skill_tester import SkillTest

def skill_runner(skill, example, emitter, loader, m1, m2):
    return SkillTest(skill, example, emitter).run(loader)
```

Then the skill_runner can be decorated by `mock.patch` to mock out resources.
2018-05-14 14:27:37 -05:00
Åke Forslund 02565efd59 Use pytest to discover test cases 2018-05-14 14:27:37 -05:00
CarstenAgerskov 1cbedaea3c Added test_all_skills. More doc. 2018-05-14 14:27:37 -05:00
penrods 8f2e5d9498 Change to Apache 2.0 license from GPLv3.0
This commit officially switches the mycroft-core repository from
GPLv3.0 licensing to Apache 2.0.  All dependencies on GPL'ed code
have been removed and we have contacted all previous contributors
with still-existing code in the repository to agree to this change.

Going forward, all contributors will sign a Contributor License
Agreement (CLA) by visiting https://mycroft.ai/cla, then they will
be included in the Mycroft Project's overall Contributor list,
found at: https://github.com/MycroftAI/contributors.  This cleanly
protects the project, the contributor and all who use the technology
to build upon.

Futher discussion can be found at this blog post:
https://mycroft.ai/blog/right-license/

This commit also removes all __author__="" from the code.  These
lines are painful to maintain and the etiquette surrounding their
maintainence is unclear.  Do you remove a name from the list if the
last line of code the wrote gets replaced?  Etc.  Now all
contributors are publicly acknowledged in the aforementioned repo,
and actual authorship is maintained by Github in a much more
effective and elegant way!

Finally, a few references to "Mycroft AI" were changed to the correct
legal entity name "Mycroft AI Inc."

==== Fixed Issues ====
#403 Update License.md and file headers to Apache 2.0
#400 Update LICENSE.md

====  Documentation Notes ====
Deprecated the ScheduledSkill and ScheduledCRUDSkill classes.
These capabilities have been superceded by the more flexible MycroftSkill
class methods schedule_event(), schedule_repeating_event(), update_event(),
and cancel_event().
2017-10-04 01:28:44 -05:00
Matthew D. Scholefield 5e392f34aa Optimize imports
Remove unused imports and group local vs external alphabetically
2017-09-18 16:07:50 -05:00
harika 173bee9f95 Fixed pep8 format 2017-08-17 02:05:02 -05:00
harika 4c3dd3aa7c Updating skill_tester to automate skills testing(regression) 2017-08-17 02:05:02 -05:00
harika 632df2a498 Update tests 2017-08-17 02:05:02 -05:00
harika e2811ccde2 added some stuffs --arron says it's awesome! 2017-08-17 02:05:02 -05:00
Åke Forslund c8dd1a9b46 Make discover test directory settable
The skill directory can be specified when calling the discovery_test

`PYTHONPATH=./ python test/integrationtests/skills/discovery_test.py
 SKILL_DIR`
2017-08-17 02:05:02 -05:00
Åke Forslund 39d0451143 Separate tests into unittests and integrationtests 2017-08-17 02:05:02 -05:00