Commit Graph

67 Commits (sj201r5)

Author SHA1 Message Date
ChanceNCounter a211441acc stop passing lang=None to Lingua Franca
LF's only breaking change over the past two versions has been the
deprecation of `lang=None` as a valid parameter. This is because the new
language loading paradigm wants to load certain functions on the fly,
which it cannot do when it is explicitly told to look for a null lang.

I've addressed this by passing `lingua_franca.get_default_lang()` where
the `lang=None` call remained.

Bonus: Gets rid of over 200 DeprecationWarnings in unit tests!
2021-06-23 21:56:19 -07:00
Kris Gesling e75a05557b docstring cleanup 2021-05-11 15:10:32 +09:30
Kris Gesling 6327ac5dbb
Upgrade Lingua Franca to v0.4.1 (#2890)
**Update Lingua Franca to v0.4.1**

The update from Lingua Franca v0.2.x to v0.4.x includes few
breaking changes.
- Some API methods have been updated.
- Mycroft-core tests have been updated to reflect improvements in
  Lingua Franca's formatting and parsing.
- add LF default lang setting method to config.locale 
  including warning that this method will change in the future
- Add TODO's for 21.08 - moving more methods to LF
- simplify loading and setting default of languages in Skills service
- Remove unneeded wrappers for Lingua Franca functions
- Fix documentation of format and parse utils
- Fix test warnings
2021-05-08 08:34:02 +09:30
Åke Forslund 2dde23467c Let lingua franca check extract_duration lang
Mycroft doesn't need to check for supported languages for
extract_duration, lingua_franca handles that internally.
2020-08-07 07:47:11 +02:00
Åke Forslund 9245f551f4 Remove backwards compat. imports from lang tools
This removes the whole mycroft.util.lang submodule replacing it with a
minimum needed set of functions

extract_datetime: when a date was missing en-us would return the current date
to not mess up existing skills. This is no longer the case, if an
utterance doesn't contain a date the function will return None as
indicated by the docstring
2020-02-12 09:27:50 +01:00
Åke Forslund 9c8c8e0144 Send time reference for "now" to Lingua-franca
This will provide timezone info as well as now reference to the
extracted sentences.
2020-01-29 15:33:28 +01:00
Åke Forslund 37caf7837d Remove logic from format.py and parse.py
The functions are still there providing the docstrings but calling the
lingua_franca functions.

Note, there is a INTENTIONAL addition of an inconsistence. Making the
extract_datetime return todays date instead of None as is documented.
This needs to be removed as soon as the mycroft default skills have been
added.
2020-01-18 08:58:23 +01:00
Åke Forslund 1b5b4f3d84 Add wrapper for extract_datetime 2020-01-18 08:58:23 +01:00
Åke Forslund 2cc90f3f9f Remove functions from format.py and parse.py 2020-01-18 08:58:23 +01:00
Åke Forslund 1b88db4fa1 Replace format and parse calls with lingua-franca 2020-01-18 08:58:23 +01:00
Åke e6385720b3
Merge pull request #2396 from maxbachmann/python2cleanup
cleanup python2 unicode
2019-11-27 08:46:20 +01:00
maxbachmann 7a48297132 cleanup python2 unicode 2019-11-26 23:09:32 +01:00
Mike Woudenberg cc5631c45f Adds rudimentary Dutch parsing support 2019-11-14 08:47:36 +01:00
ChanceNCounter 1107f5d6cc Fix extractnumber_es, add extract_numbers_es
- Fix bug causing extractnumber_es to return a sum instead of a list
- Add Spanish parser to extract_numbers and extract_number

==== Fixed Issues ====
Closes #2310

====  Tech Notes ====
Further obscures #2056: short_scale and ordinals parameters added to the
Spanish parsers, but they don't do anything. Present for compat only.
There is a TODO for this.

==== Localization Notes ====
It's all Spanish stuff!

This will require a native Spanish speaker to analyze the relationship
between extractnumber and isFractional, and determine why certain
fractions do not parse correctly. There is a TODO for this.
2019-10-03 17:11:39 -07:00
Åke Forslund 25026e9f4c Add docs for the mycroft.util.format module
- Adds a module docstring to both format and parse module
- Update toc with a short description for these members
- Fix slight formatting issues
2019-06-28 15:30:13 +02:00
Steve Penrod 44f60ec6f3 Change default lang to None, not English
Much of the code used "en-us" as the default value when not specified.
This limited the internationalization potential.  Changing the default
to None and adds the ability to define the default lang code from other
locations in code.  E.g.

```python

from mycroft.util.lang import set_default_lang

set_default_lang("en-us")
print("English date: "+nice_date(dt))

set_default_lang("de-de")
print("German date: "+nice_date(dt))
```

This allows easier localization of Skills by having the framework set the default without any changes necessary by the Skill writers.

Other minor changes:
* Changed the default return value of get_gender*() to None instead of False
2019-03-14 10:57:31 +01:00
Andreas Lorensen 6c0cd8d427 Add danish 2 - initial work to get danish to core (#2033)
* Danish formatting and parsing functions
2019-03-04 21:40:12 +01:00
Ale df7dfaa006 Update parse_it.py to 18.8.12 (#1990)
* update functions in parse_it.py
* update tests
* translate docstrings to english
2019-02-27 08:00:50 +01:00
Steve Penrod 684363ffaf Correct extract_duration doc and failure value
The documentation indicated an array instead of a tuple, plus
the default return value didn't match the documented return type.
2019-02-25 01:54:58 -06:00
Chris Rogers 4bca969316 Log unsupported languages.
This extracts the logging logic that was being used in a few places to
indicate certain parcing functions are not supported in particular
languages, and adds the logging to extract_duration.
2019-02-12 17:01:16 -05:00
Chris Rogers 3a9f11c67e Add extract_duration skeleton 2019-01-23 18:25:22 -05:00
JarbasAI 60d2905678 feature/extract multiple numbers from text (#1867)
Add extract_numbers() function

The function extracts all numbers from the input string and returns them as a list.
2018-11-19 12:39:59 +01:00
Åke Forslund 3682391960 Fix final pep8 issues 2018-10-09 16:36:15 +02:00
Silvia O'Dwyer 45eadaab49 Warnings for unsupported languages are now logged. 2018-10-09 16:15:57 +02:00
Steve Penrod 3c94435f0e Extract_datetime cleanup
Cleaned up a few things noticed during review of Spanish extract_datetime_es() implementation.
2018-09-24 15:59:28 -05:00
Angel Docampo f2c033f1f7 added functions for the spanish parser (#1571)
Added functions for the spanish parser

* Added spanish function calls in parser.py for extractnumber_es
and extract_datetime_es
* Added spanish functions in util/paser_es.py for extractnumber_es
extract_datetime_es and added some missing numbers

Merged in changes from #1804
2018-09-24 05:06:29 -05:00
Åke 25ccc3bed7 Feature/extract datetime update (#1804)
* Fix ambiguous time handling
  In certain cases the ambiguous time handling skipped a day forward. This updates the logic to handle a bit better.
* Adds a test for the ambiguous time
* Remove references to timeStr
   timeStr was never set and the logic that used it would never activate.
* Remove rename of currentDate
* Add extract_datetime parameter default_time
   If a time is not found in the input string the time will be set from the
   datetime/time object passed in as the default_time argument. If None the
   time will be Midnight as previously.
2018-09-24 03:08:39 -05:00
Steve Penrod 51b60398a3 Remove deprecated extractnumber
Removing the deprecated mycroft.util.parse.extractnumber(), use extract_number() instead.
2018-08-21 04:06:34 -05:00
Steve Penrod 718703c94b Silence Codacy complaints
More import cleanup
2018-07-18 13:46:27 +02:00
Steve Penrod 1029881a78 Minor code cleanup
* Add support for "o'clock" variations
* Clean up imports
* Simplified some overly-complex code
* Normalize line endings
2018-07-18 13:46:27 +02:00
Steve Penrod 3624680698 Enhance extract_datetime(), add time utilities
Many cases that were missed in the unittests for extract_datetime()
from the original source.  Restored those tests and made code
adjustments to support them all.

Also adding the mycroft.util.time module.  This supports:
* mycroft.util.time.default_timezone()
  Returns the user-configured timezone based on location
* mycroft.util.time.now_utc()
  Returns the time in UTC
* mycroft.util.time.now_local()
  Returns the time in the user's timezone
* mycroft.util.time.to_utc()
  Converts to UTC
* mycroft.util.time.to_local()
  Converts to user's timezone

NOTE: Several skills should be updated to use these now.

==== Fixed Issues ====
Several issues for skills regarding parsing of "today"

====  Documentation Notes ====
Note the new module:  mycroft.util.time

==== Localization Notes ====
Localized versions of extract_datetime() likely need to be
updated, as most were based on the original English implementation
2018-07-18 13:46:27 +02:00
JarbasAI 6cf2ed814c feature/allow to pronounce ordinals and very small fractions (#1663)
* allow to pronounce ordinals

* cleanup

* long scale / short scale very small fractions
2018-07-10 02:54:04 -05:00
Åke 622748d7d3 Bugfix/parse corrections (#1670)
* Restore extractdatetime to return False

- Restore extractdatetime to return False if no time was found
- Add tests to make sure this is true
- Add a extract_datetime function to keep coherency with the rest of the functions. (the old extractdatetime still exists for compatibility)
- Update documentation to match

* Minor corrections to docstrings.
2018-07-02 02:44:28 -05:00
Åke Forslund 39c4e0aaf1 Add wiki link for long/short scale 2018-06-18 15:46:20 +02:00
jarbasal 7e1fb80cb4 improve extract_number_en
- Support higher numbers, almost unreasonably high!
- Support long scale and short scale
2018-06-18 15:45:14 +02:00
f-e-l-i-x b239d3dc2f German language support (#1634)
* Add parse and format functions for german
* Add german dialog files
2018-06-12 08:55:21 +02:00
Åke Forslund b200d51d39 Fix test cases under python 3
Lots of minor fixes including, sorting dicts, making ints of strings,
     MagicMock file spec and some other things

A couple of issues in the mycroft-core code base were identified and
fixed. Most notably the incorrect version check for python three when
adding basestring.

Update .travis.yml
2018-04-27 08:50:46 -05:00
Kévin C 692877fa07
Fix PEP8 issue 2018-02-15 12:53:51 +01:00
Kévin C f0869dcf31
Merge branch 'dev' into dev 2018-02-15 12:51:45 +01:00
Cakeh c1918da9cb Add fr support for ordinal numbers and time qualifiers 2018-02-12 12:02:54 +01:00
Cakeh 3a11f39d7d Issue-1375 - Fix and complete french translation 2018-02-08 21:46:24 +01:00
Åke Forslund a11b093e4d Add format and parsing methods for swedish 2018-02-01 15:32:08 +01:00
Åke b2af245554 Add match_one fuzzymatching function (#1372)
match_one finds the closest match for a query in a list or dictionary of
possibilities.
2018-02-01 01:52:14 -06:00
Ale c56b293d3d Update italian parsing methods
Add extract_datetime_it
Add tests for italian methods
2018-01-29 09:47:10 +01:00
Ale 158b25170b translate italian functions for format.py and parse.py based on pt_pt job (#1363)
Add basic functions for Italian language

- nice_number()
- extract_number()
- normalize()
2018-01-25 15:58:32 +01:00
Åke Forslund 0114ce473e Split format.py and parse.py per language
Move the language specific functions and constants into separate files.
This will avoid many unnecessary conflicts due to involuntary encoding
changes.
2018-01-08 17:08:31 -06:00
Augusto Monteiro e3243e2a62
Merge pull request #1307 from forslund/bugfix/extract_datetime
Minor fix of extract datetime
2017-12-19 15:15:22 -05:00
penrods 5a12fc471c Add new parse function fuzzy_match (take 2)
A fuzzy_match() produces a basic ratio match of two strings.
The result is between 0.0 and 1.0, where 1.0 is a perfect
match.
2017-12-15 05:54:14 -06:00
Åke Forslund 01b37f06eb Handle "tonight" and text containing only a date
- "tonight" is re-interpreted as PM
- check is performed to check if previous word exist before accessing it
to handle sentences containing only a simple date
2017-12-13 15:00:57 +01:00
jarbasai d7926cde07 pt unicode bug fix in normalization 2017-12-12 03:10:00 +00:00