Commit Graph

3068 Commits (edafce059d6a26367b6d379f314a86e2fb8a2b14)

Author SHA1 Message Date
Alistair Francis edafce059d skills: Open all skill data files as utf8 encoded
Signed-off-by: Alistair Francis <alistair.francis@wdc.com> and Åke Forslund <ake.forslund@gmail.com>
2019-02-13 21:05:18 +01:00
Åke 65a7197519
Merge pull request #1977 from ChristopherRogers1991/feature/issue-1959
Add extract_duration() method
2019-02-13 16:08:49 +01:00
Steve Penrod 065273bf4f New formatters: nice_duration() and join_list() (#1988)
* New formatters:  nice_duration() and join_list()

Adding two new formatting functions:
* nice_duration(duration, lang="en-us", speech=True)
  Accept seconds or duration and produce a nice sounding duration.
  Example:  nice_duration(61) == "one minute one second"
            nice_duration(61, speech=False) == "1:01"
* join_list(items, connector, sep=None, lang="en-us")
  Example:  join_list(["a", "b", "c"], "and") == "a, b and c"

This includes a translation helper that uses text files in the
mycroft/res/text/LANG/ directory, such as "second.word".
2019-02-13 12:52:35 +01:00
Chris Rogers 113352339d Fix pep8 issues.
This is in support of issues-1959.
2019-02-12 17:29:21 -05:00
Chris Rogers 4bca969316 Log unsupported languages.
This extracts the logging logic that was being used in a few places to
indicate certain parcing functions are not supported in particular
languages, and adds the logging to extract_duration.
2019-02-12 17:01:16 -05:00
Chris Rogers acbe46aede Fix documentation
Fix typo and add an explanation for
_extract_number_with_text_en_helper.
2019-02-12 16:36:11 -05:00
Chris Rogers cc0d3da62c Change _Token to a namedtuple.
This is in support of issues-1959.
2019-02-12 16:23:53 -05:00
Åke 7e15366fc2
Merge pull request #1980 from MycroftAI/feature/doc-save-utt
Replace "record_utterances" config option with "save_utterances" option and improve flag documentation.
2019-02-11 21:25:02 +01:00
Ale 307910e53e update format_it.py test_format_it.py (#1984)
* Update format_it.py and tests
2019-02-11 18:07:14 +01:00
Åke Forslund bcae0ac4bd Update default config
- Move "save_utterenaces" comments to correct block
- Correct information, records utterance not wakeword
- replace "record_utterances" with the new "save_utterances"
2019-02-11 15:52:48 +01:00
Steve Penrod 2424fd1c9b Change to "save_utterances" instead of "record_utterances"
Change to match the documented and more intuitive name "save_utterances",
but add backwards compatibility code to support the original
"record_utterances".
2019-02-11 01:53:28 -06:00
Steve Penrod 28f18b745d
Document "save_utterance" option
The Listener has an option to save the sound that was recognized as a wake-word.   This isn't new functionality, but was previously undocumented.
2019-02-04 15:34:37 -06:00
Chris Rogers 1bb74f5c79 Use isinstance instead of type.
This is in support of issues-1959.
2019-02-03 12:58:31 -05:00
Chris Rogers cdf7dc3756 Use datetime.timedelta for extract_duration_en.
This is in support of issues-1959.
2019-02-03 12:16:36 -05:00
Åke 98c0376966
Merge pull request #1978 from forslund/bugfix/sudo-check
Move sudo check to beginning of dev_setup.sh script
2019-02-03 09:18:35 +01:00
Åke Forslund 811e7e37e8 Move sudo check to beginning of script
The sudo command check was performed only when packages were installed and not before the interactve portion leaving $SUDO unset when trying to create the /opt/mycroft folder.
2019-02-02 23:15:50 +01:00
Chris Rogers d50cb00de8 Fix pep8 issues.
This is in support of issues-1959.
2019-02-02 14:21:37 -05:00
Chris Rogers 9e193c74e0 Make constant and functions private.
Making things weak private, to limit surface area of support. As things
become increasingly stable/tested/useful, it may make sense to open them
up, but for now, keeping them private will limit risk.

This is in support of issue-1959.
2019-02-02 14:07:48 -05:00
Chris Rogers 351381bca2 Fix pep8 issues.
This is in support of issues-1959.
2019-02-02 14:03:19 -05:00
Chris Rogers 7049e65cbe Minor shuffling + update docs.
After many changes, things had gotten a little disorganized, and the
docs were a little out of date. This brings them up to date.

This is in support of issues-1959.
2019-02-02 13:45:22 -05:00
Chris Rogers 534ca2aff9 All regressions in number parsing fixed.
This is in support of issues-1959.
2019-02-02 13:12:25 -05:00
Chris Rogers 6da1ec5c6e Fix regression in number parsing.
Fix regression that caused "X and one half" to parse as just X.

This is in support of issues-1959.
2019-02-01 23:24:45 -05:00
Chris Rogers f4eee8726a Refactor many methods in parse_en.
This improves the utility of the _ReplaceableNumber class, and updates
most of the number parsing functions to take tokens rather than text.
This simplifies the interactions between many of the functions, as there
is no need to convert back and forth between text and tokens.

This also adds some tests. Note that there are a few regressions that
will be fixed in a subsequent commit.
2019-02-01 23:04:54 -05:00
Chris Rogers 95aca10294 Fix _extract_decimal
Actually use the short_scale and ordinals values.
2019-02-01 18:57:28 -05:00
Chris Rogers 2ce632389f Fix and simplfy extract_numbers_en
This was calling convert_words_to_numbers and parsing out the resuling
numbers, which was a simple way of getting the numebrs in order, but it
choked on anything that didn't match the regex being used to parse
numbers, in particular numbers of the form '6e18'. The better solution
is to directly use extract_numbers_with_text (which now sorts by
start_index) and get the values from there directly.

This is in support of issues-1959.
2019-02-01 18:51:03 -05:00
Chris Rogers 6645ab6bfe Add short_scale and ordinal args to helpers.
This is in support of issues-1959.
2019-02-01 18:28:40 -05:00
Chris Rogers cbebf957e4 Add tests for extract_duration_en.
This is in support of issues-1959.
2019-02-01 18:22:46 -05:00
Chris Rogers 5c74789c2d Lower text before parsing.
This is in support of issues-1959.
2019-02-01 18:21:13 -05:00
Chris Rogers 23edb9eb00 Fix decimal/fraction parsing with leading numbers.
"Five hours seven and a half minutes" was parsing as 5.5. This is
resolved. Multiple fractions/decimals still cause problems, e.g.

convert_words_to_numbers("seven and a half and nine and a half")
Out[5]: '7 and a 0.5 and 9 and a 0.5'

This is in support of issues-1959.
2019-02-01 17:39:01 -05:00
devs-mycroft b8beadfbf2 Version bump from 18.8.11 to 18.8.12 2019-01-31 14:03:20 +00:00
Åke 31656d6294 Add tests for commented dialogs and DialogLoader (#1853)
* Add tests for DialogLoader

* Handle Path/PosixPath

LOG messages when files/directories were missing would fail when a PosixPath/Path object was sent to as argument. This uses format to get the correct string representation.

* Add test for dialog.get()
2019-01-31 02:34:44 -06:00
Åke 8ff2ccc4fa
Merge pull request #1963 from forslund/feature/audioservice-prefer-local
Play on local audiobackends before trying remotes
2019-01-31 09:09:32 +01:00
Åke 2c858fba48 Add python version option to dev_setup.sh (#1952)
-p arg or --python arg allows the user to select which installed python version is used in the mycroft virtualenv.
2019-01-31 01:53:35 -06:00
Steve Penrod 01ea0afdeb
Minor edit of comments 2019-01-31 01:46:35 -06:00
Åke a94019a8b8 Use language set in mycroft configuration (#1925)
Slight refactoring to accommodate for this in a nice way. Created
function connect_to_mycroft() handling fetching the config and
connecting to the mycroft messagebus since these are related and the
order is important to maintain for it to work.
2019-01-31 01:38:47 -06:00
Åke 6b09770957
Merge pull request #1971 from forslund/feature/audio-test-basic-info
Add some additional info to audiotest
2019-01-31 08:36:46 +01:00
Åke 0ae66c3f93 Bugfix/speech stop (#1961)
* Refactor mimic2 to use the shared tts architecture

* Make sure the queue is cleared

- Add a convenience method grouping clear_queue and clear_visemes
- The start time is now set before the lock to allow multiple speech requests queued before the stop signal to also be cancelled
- Make sure the any pending TTS generation is cleared from the queue by calling tts.clear() when breaking from the chunking loop.
2019-01-31 01:30:53 -06:00
Åke 0a686be7d1 Add a decorator for simply adding idle pages (#1970)
To simplify the process of adding an idle page to a skill the decorator "resting_screen_handler" was added. In a skill class the decorator can be applied to a method to register it to handle idle.

    @resting_page_handler("My Idle Page")
    def handler(self, message):
        ...

The decorator will Register the method with the Mark-2 skill and perform all communications needed to make it work smoothly.
2019-01-31 01:22:33 -06:00
Steve Penrod df4f6ca6dd Fix typo and add --list option
The -l / --list option will now display the name and index of all available input devices.
2019-01-31 08:19:21 +01:00
Åke 267c7bf571 Single viseme message (#1973)
* Add new api command to send visemes as single list.  This allows more efficient use of the messagebus and gives implementors flexibility in how they handle the visualization.

* Switch mark1 to use viseme_list
2019-01-31 00:37:55 -06:00
Åke 60ab1aab69 Fix registering fallback (#1974)
The wrong method was registered, instead of the wrapped function call
the original method was registered. This led to not being able to
unregister fallbacks.
2019-01-31 00:21:15 -06:00
Chris Rogers a3e94bcbc6 Add numbers, e.g. '20', '30' to sums
This is in support of issues-1959.
2019-01-30 22:53:22 -05:00
Chris Rogers 4732feab41 Fix indicies and substitution logic.
Placeholders are inserted into the text to maintain accurate
indicie relative to the original string.

This is in support of issues-1959.
2019-01-30 22:06:02 -05:00
Chris Rogers 49274493d9 Update convert_words_to_numbers logic.
The logic has been updated for start/end indexs.

This is in support of issues-1959.
2019-01-30 21:18:48 -05:00
Chris Rogers 03a445991c Fix multiplies and extract_numbers_with_text
A small bug caused things like "two hundred twenty" to return only the
"hundred tenty" for the text. This has been fixed.

extract_numbers_with_text was updated to deal with the new return types
of the functions it depends on. Specifically, it accounts for the start
and end index values.

This is in support of issues-1959.
2019-01-30 18:24:18 -05:00
Chris Rogers 71836b61ec Fix decimal and fraction parsing.
This updates the _extract_fraction and _extract_decimal functions to
handle the new token format.
2019-01-30 18:03:25 -05:00
Chris Rogers 48214ca66a Introduce tokens for number parsing.
Replace use of tuples with a dedicated class. This improves clarity by
giving named accessors.

This is in support of issues-1959.
2019-01-30 16:48:59 -05:00
Chris Rogers 9db9b6107b Change approach to number/text replacment.
Previously it was assumed that the orgiginal text would be enough to
determine where in a string a number should go, however, in some
scenarios, that does not work, and results in the wrong values being
parsed.

A different, and smarter approach is being taken now, in which the
original string is initially split into a list of tuples of
(index, word) where index is the index of the word within the string.
All subsequent processing is done on these tuples, meaning we always
know exactly where the words were in the orginal string. This should
make text replacement perfect, as we can always sub out the exact,
correct words, based on their indicies.

extract_number_with_text_en now returns the number parsed, the text that
represents the number, the start index, and the end index.

Things are not yet working perfectly. Here is roughly the current state
of the world:

from mycroft.util.lang.parse_en import *
extract_number_with_text_en("this is some two hundred thousand twenty
two hours")
Out[3]: (200022, 'hundred thousand twenty two', 4, 7)
extract_number_with_text_en("this is some twenty two hours")
Out[4]: (22, 'twenty two', 3, 4)
extract_number_with_text_en("this is some twenty hours")
Out[5]: (20, 'twenty', 3, 3)
extract_number_with_text_en("this is some two and a half hours")
Out[6]: (2, 'two', 3, 3)
extract_number_with_text_en("this is some two point five hours")
Out[7]: (2, 'two', 3, 3)

The list of tuples is a bit of a hassle to deal with. In a future
commite the will be replaced with dictionaries, or even better, Token
objects, that contain the word and it's index. This would make the
code easier to reason about (removing lots things like words[0][1]
which has no meaning without deep understanding of the code).

This is in support of issues-1959.
2019-01-29 22:33:23 -05:00
Chris Rogers 8a5bf49651 Handle lists of summation numbers.
Phrases like "twenty thirty forty" would return
(40, "twenty thrity forty"). This changes that so
(40 "forty") is returned.
2019-01-29 21:25:06 -05:00
Chris Rogers 690df0b1d3 Update entract_numbers_en to use the new functions.
This is in support of issues-1959.
2019-01-29 20:57:16 -05:00