Commit Graph

58 Commits (mark-ii/latest)

Author SHA1 Message Date
Kris Gesling 6327ac5dbb
Upgrade Lingua Franca to v0.4.1 (#2890)
**Update Lingua Franca to v0.4.1**

The update from Lingua Franca v0.2.x to v0.4.x includes few
breaking changes.
- Some API methods have been updated.
- Mycroft-core tests have been updated to reflect improvements in
  Lingua Franca's formatting and parsing.
- add LF default lang setting method to config.locale 
  including warning that this method will change in the future
- Add TODO's for 21.08 - moving more methods to LF
- simplify loading and setting default of languages in Skills service
- Remove unneeded wrappers for Lingua Franca functions
- Fix documentation of format and parse utils
- Fix test warnings
2021-05-08 08:34:02 +09:30
Åke Forslund 9245f551f4 Remove backwards compat. imports from lang tools
This removes the whole mycroft.util.lang submodule replacing it with a
minimum needed set of functions

extract_datetime: when a date was missing en-us would return the current date
to not mess up existing skills. This is no longer the case, if an
utterance doesn't contain a date the function will return None as
indicated by the docstring
2020-02-12 09:27:50 +01:00
Åke Forslund 9c8c8e0144 Send time reference for "now" to Lingua-franca
This will provide timezone info as well as now reference to the
extracted sentences.
2020-01-29 15:33:28 +01:00
Chris Rogers 7338387d4d Add failing unit test for 100% 2019-12-06 15:18:14 -08:00
Åke Forslund 5e863515da Fix broken test case on Python 3.8
- Simplify broken test and make it work on python 3.8
- Add python 3.8 to mandatory test cases
2019-12-01 12:56:01 +01:00
maxbachmann 7a48297132 cleanup python2 unicode 2019-11-26 23:09:32 +01:00
ChanceNCounter 16d6bf4b22 fix bug, redundancy in parse_en; improve coverage
- Remove unreachable conditions from parse_en:1122-1133
  - input string passed through clean_string() on line 763
  - articles stripped from input before line 1122
  - removed conditions relied on presence of "the" in input
- Improved test coverage on parse_en.py by approx 65 statements
  - Directly test certain helper functions which are difficult to
invoke indirectly
  - Add tests for certain missed conditions
  - Approx. 20 uncovered statements remain
2019-10-03 14:41:21 -07:00
Åke Forslund d41fe97fd6 Add test case for the previously failing string 2019-09-30 11:02:29 +02:00
Kris Gesling efa69e0526 'next {day}' to be 3-9 days in future (#2184)
* 'next {day}' to be 2-9 days in future

* restore module level docstring

* remove lang specific definition of 'next'

* restore module level docstring
2019-07-13 11:53:38 +02:00
Kris Gesling 946b7bf3de fix requests with non-month 'may' 2019-07-08 15:36:14 +02:00
jarbasal 64a58f3290 tonight remainder 2019-03-27 11:15:24 +00:00
Ruthvicp b28011681e handling tonight in date & time extraction (#2066)
* handling tonight in date & time extraction
2019-03-27 01:41:50 -05:00
Steve Penrod a5dcc8d3d3 Fix unit test errors 2019-03-14 10:57:31 +01:00
Ruthvicp ec7ed25ed5 date and time - for tonight, weekdays (#2023)
* date and time - for tonight, weekdays

* Updating the previous commit test_extractdatetime_en

* Editing comments for extract_date_time_en

* Generalized as a marker

Expanded this from just handling "weekends" to any day or plural of the day, "weekend", "weekday" or "weekdays".
2019-02-28 01:15:55 -06:00
Chris Rogers cdf7dc3756 Use datetime.timedelta for extract_duration_en.
This is in support of issues-1959.
2019-02-03 12:16:36 -05:00
Chris Rogers 351381bca2 Fix pep8 issues.
This is in support of issues-1959.
2019-02-02 14:03:19 -05:00
Chris Rogers 534ca2aff9 All regressions in number parsing fixed.
This is in support of issues-1959.
2019-02-02 13:12:25 -05:00
Chris Rogers 6da1ec5c6e Fix regression in number parsing.
Fix regression that caused "X and one half" to parse as just X.

This is in support of issues-1959.
2019-02-01 23:24:45 -05:00
Chris Rogers f4eee8726a Refactor many methods in parse_en.
This improves the utility of the _ReplaceableNumber class, and updates
most of the number parsing functions to take tokens rather than text.
This simplifies the interactions between many of the functions, as there
is no need to convert back and forth between text and tokens.

This also adds some tests. Note that there are a few regressions that
will be fixed in a subsequent commit.
2019-02-01 23:04:54 -05:00
Chris Rogers cbebf957e4 Add tests for extract_duration_en.
This is in support of issues-1959.
2019-02-01 18:22:46 -05:00
Chris Rogers 31760e6c9c Fix existing extract_duration tests.
This is in support of issues-1959.
2019-01-29 20:31:31 -05:00
Chris Rogers 9ff7fd5452 Fix issues with extractnumber_en_with_text
Issues fixed:
Lists, e.g. "some words one two three" would return (3, "one two three")
Negaitve words were not included in output, e.g. "negative five" would
return (-5, "five").

This is in support of issues-1959.
2019-01-29 19:53:30 -05:00
Chris Rogers 12e5fd603a Implement methods useful for extract_duration
Methods implemented include:

extract_number_with_text
extract_numbers_with_text
convert_words_to_numbers
extract_duration

This is in support of issues-1959. This continues the work of
returning the relevant text that corresponds to a number
parsed from a string.
2019-01-28 17:04:12 -05:00
Chris Rogers 3a9f11c67e Add extract_duration skeleton 2019-01-23 18:25:22 -05:00
Chris Rogers 12a55c4cb8 Issues-1962 - Fix/add tests for normalize 2019-01-21 18:44:20 -05:00
Åke Forslund 77c76416d3 Update "third one" test to match behaviour 2018-11-26 09:26:43 +01:00
Kris Gesling 32a3860bce Issue-1877 - fix ordinal followed by one
"Third one" will now return the expected 3 instead of 1.
2018-11-26 09:25:44 +01:00
JarbasAI 60d2905678 feature/extract multiple numbers from text (#1867)
Add extract_numbers() function

The function extracts all numbers from the input string and returns them as a list.
2018-11-19 12:39:59 +01:00
jarbasal fa2b3007a9 improve datetime parsing with long times and a couple
add support for decades, centuries, millemniums
add support for "within the hour", "in a second/minute",
add support for "a couple time_unit" and "a couple of time_unit"
2018-11-17 09:03:41 +01:00
jarbasal d9a905c8b1 extract date time improvements 2018-11-07 04:14:21 +00:00
Niels Mündler 1b16b4dbaf Include problematic input from #1718 as test
Fix trillion being saved with wrong number (10e10 instead of 10e12)

==== Fixed Issues ====
1718

====  Tech Notes ====
NONE - explain new algorithms in detail, tool changes, etc.

====  Documentation Notes ====
NONE - description of a new feature or notes on behavior changes

==== Localization Notes ====
NONE - point to new strings, language specific functions, etc.

==== Environment Notes ====
NONE - new package requirements, new files being written to disk, etc.

==== Protocol Notes ====
NONE - message types added or changed, new signals, APIs, etc.
2018-10-09 13:18:13 +02:00
Åke 25ccc3bed7 Feature/extract datetime update (#1804)
* Fix ambiguous time handling
  In certain cases the ambiguous time handling skipped a day forward. This updates the logic to handle a bit better.
* Adds a test for the ambiguous time
* Remove references to timeStr
   timeStr was never set and the logic that used it would never activate.
* Remove rename of currentDate
* Add extract_datetime parameter default_time
   If a time is not found in the input string the time will be set from the
   datetime/time object passed in as the default_time argument. If None the
   time will be Midnight as previously.
2018-09-24 03:08:39 -05:00
Åke Forslund b7e1cb835c Fix broken test
- update test_parse*
- correct the reference in mycroft.util
2018-08-21 11:57:12 +02:00
Michael Nguyen 7b54149bcd
Merge pull request #1653 from JarbasAl/feature/pronounce_scientific
Feature/pronounce scientific
2018-07-31 13:04:14 -05:00
Steve Penrod 3624680698 Enhance extract_datetime(), add time utilities
Many cases that were missed in the unittests for extract_datetime()
from the original source.  Restored those tests and made code
adjustments to support them all.

Also adding the mycroft.util.time module.  This supports:
* mycroft.util.time.default_timezone()
  Returns the user-configured timezone based on location
* mycroft.util.time.now_utc()
  Returns the time in UTC
* mycroft.util.time.now_local()
  Returns the time in the user's timezone
* mycroft.util.time.to_utc()
  Converts to UTC
* mycroft.util.time.to_local()
  Converts to user's timezone

NOTE: Several skills should be updated to use these now.

==== Fixed Issues ====
Several issues for skills regarding parsing of "today"

====  Documentation Notes ====
Note the new module:  mycroft.util.time

==== Localization Notes ====
Localized versions of extract_datetime() likely need to be
updated, as most were based on the original English implementation
2018-07-18 13:46:27 +02:00
JarbasAI 6cf2ed814c feature/allow to pronounce ordinals and very small fractions (#1663)
* allow to pronounce ordinals

* cleanup

* long scale / short scale very small fractions
2018-07-10 02:54:04 -05:00
Åke 1093383443 Fix extraction of 0 in extract_number() (#1673)
0 was not detected as a valid number when checking if the function should proceed to check for fractions.
2018-07-10 02:26:25 -05:00
Åke 622748d7d3 Bugfix/parse corrections (#1670)
* Restore extractdatetime to return False

- Restore extractdatetime to return False if no time was found
- Add tests to make sure this is true
- Add a extract_datetime function to keep coherency with the rest of the functions. (the old extractdatetime still exists for compatibility)
- Update documentation to match

* Minor corrections to docstrings.
2018-07-02 02:44:28 -05:00
JarbasAI fa4173a2d3 fix extract date (#1651)
* Fix errors when spaces are missing "3pm" "5seconds"
* Fix relative times "in 15 minutes" is now from current time not from midnight

Resolves #1650
2018-06-26 11:45:39 +02:00
jarbasal 2f05b0b820 fix extra zero 2018-06-22 01:01:59 +01:00
jarbasal 7e1fb80cb4 improve extract_number_en
- Support higher numbers, almost unreasonably high!
- Support long scale and short scale
2018-06-18 15:45:14 +02:00
Åke Forslund b200d51d39 Fix test cases under python 3
Lots of minor fixes including, sorting dicts, making ints of strings,
     MagicMock file spec and some other things

A couple of issues in the mycroft-core code base were identified and
fixed. Most notably the incorrect version check for python three when
adding basestring.

Update .travis.yml
2018-04-27 08:50:46 -05:00
Åke b2af245554 Add match_one fuzzymatching function (#1372)
match_one finds the closest match for a query in a list or dictionary of
possibilities.
2018-02-01 01:52:14 -06:00
Åke Forslund 35a5528a48 Split the test_parse.py by language
- switch over to utf-8 encoding for all the files
2018-01-10 16:21:28 +01:00
Åke Forslund 0114ce473e Split format.py and parse.py per language
Move the language specific functions and constants into separate files.
This will avoid many unnecessary conflicts due to involuntary encoding
changes.
2018-01-08 17:08:31 -06:00
Augusto Monteiro e3243e2a62
Merge pull request #1307 from forslund/bugfix/extract_datetime
Minor fix of extract datetime
2017-12-19 15:15:22 -05:00
Åke Forslund b7f5bd2dc4 Fix failing test (changed limit) 2017-12-15 13:03:57 +01:00
penrods 5a12fc471c Add new parse function fuzzy_match (take 2)
A fuzzy_match() produces a basic ratio match of two strings.
The result is between 0.0 and 1.0, where 1.0 is a perfect
match.
2017-12-15 05:54:14 -06:00
Åke Forslund 7b091e9b46 Add failing test cases for extract_datetime
- "tonight" isn't handled
- sentences containing nothing but a date throws an exception ("december 3")
2017-12-12 08:32:54 +01:00
jarbasai aa4d3a47cf pep8 strikes again 2017-12-12 03:22:11 +00:00