Commit Graph

41 Commits (dev)

Author SHA1 Message Date
Marc Mueller 5ed7d32749
Remove unnecessary asyncio EventLoopPolicy init_watcher backport (#129628) 2024-11-01 13:44:49 +01:00
Marc Mueller 5cd8e4ab7e
Update mypy-dev to 1.12.0a3 (#124939)
* Update mypy-dev to 1.12.0a3

* Fix
2024-08-31 19:34:41 +02:00
Erik Montnemery 6c01e4b99c
Use BaseEventLoop._thread_id instead of a custom attribute (#124054)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-08-17 10:59:39 +02:00
J. Nick Koston 72e4aee155
Ensure name of task is logged for unhandled loop exceptions (#118822) 2024-06-04 12:48:29 -04:00
Sid b456d97e65
Replace pylint protected-access with Ruff SLF001 (#115735) 2024-05-06 20:33:26 +02:00
J. Nick Koston 53a179088f
Add debug mode to catch unsafe thread operations using core helpers (#115390)
* adjust

* adjust

* fixes

* one more

* test

* debug

* move to config

* cover

* Update homeassistant/core.py

* set debug from RuntimeConfig

* reduce

* fix message

* raise

* Update homeassistant/core.py

* Update homeassistant/core.py

* no flood check for raise

* cover
2024-04-24 03:36:05 +02:00
Marc Mueller 19ab3d6daf
Add empty line after module docstring [helpers + other] (#112707) 2024-03-08 10:36:11 -05:00
Erik Montnemery 97cc05d0b4
Make it possible to restart core in safe mode (#102606) 2023-10-24 14:47:58 +02:00
Erik Montnemery c481fdb7d0
Rename safe mode to recovery mode (#102580) 2023-10-23 20:33:08 +02:00
Erik Montnemery 09ad1a9a36
Remove unnecessary block use of pylint disable in components p-z (#100192) 2023-09-12 20:47:48 +02:00
J. Nick Koston 790c1bc251
Decrease event loop latency by binding time.monotonic to loop.time directly (#98288)
* Decrease event loop latency by binding time.monotonic to loop.time directly

This is a small improvment to decrease event loop latency. While the goal is
is to reduce Bluetooth connection time latency, everything using asyncio
is a bit more responsive as a result.

* relo per comments

* fix too fast by adding resolution, ensure monotonic time is patchable by freezegun

* fix test that freezes time too late and has a race loop
2023-08-13 20:37:45 -04:00
Marc Mueller 447479d0a0
Add packaging as default requirement (#97712) 2023-08-04 12:29:18 +02:00
J. Nick Koston 99def97ed9
Add cancel messages to core task cancelation (#96972) 2023-07-20 18:03:36 -05:00
rlippmann 3a72054f93
Make dataclasses in HA core slotted (#91208) 2023-04-11 07:58:28 -10:00
Marc Mueller 386533a16f
Update mypy to 1.1.1 (#89268)
* Update mypy to 1.1.1
* Update pydantic to 1.10.6
2023-03-08 22:57:54 +01:00
Marc Mueller bfb89fd8f2
Update pylint to 2.17.0 (#89377)
* Update pylint to 2.17.0

* Remove unused pylint disable comments
2023-03-08 10:54:19 -05:00
J. Nick Koston 03eea7bd3f
Avoid subprocess memory copy when c library supports posix_spawn (#87958)
* use posix spawn on alpine

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy all the memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
1b736838e6/Lib/subprocess.py (L745)

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided and subprocess creation does not
get discernibly slow the larger the Home Assistant
python process grows.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
1b736838e6/Lib/subprocess.py (L745)

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* missed some

* adjust more tests

* coverage
2023-02-13 09:02:51 -05:00
J. Nick Koston d1e1734fc7
Use PidfdChildWatcher by default when available (#87951)
This is a backport from cpython 3.12

https://docs.python.org/3/library/asyncio-policy.html
> PidfdChildWatcher is a “Goldilocks” child watcher implementation. It doesn’t require signals or threads, doesn’t interfere with any processes launched outside the event loop, and scales linearly with the number of subprocesses launched by the event loop. The main disadvantage is that pidfds are specific to Linux, and only work on recent (5.3+) kernels.

https://github.com/python/cpython/pull/98024
There are some additional fixes in cpython 3.12
in https://github.com/python/cpython/pull/94184
when there is no event loop running in the main thread
but this is not a problem we have
2023-02-12 15:41:55 -05:00
Franck Nijhof 79b52a2b41
Stricter pylint message control (#86154) 2023-01-20 13:47:55 +01:00
Franck Nijhof 64c2340fab
Core code styling improvements (#85963) 2023-01-15 23:00:51 +01:00
puddly 8c8994352d
Allow only specific packages to be skipped during startup dependency installation (#82758) 2022-11-30 08:38:52 +01:00
Marc Mueller 93cbb331e5
Update Mypy to 0.950 (#70948)
Co-authored-by: J. Nick Koston <nick@koston.org>
2022-04-27 20:49:54 -05:00
Marc Mueller 8d2fb72cc3
Add type ignore error codes [core] (#66773) 2022-02-17 23:09:22 -08:00
Ville Skyttä 872bc456a9
Clean up no longer needed Python 3.8 support code (#65231)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-01-30 15:07:07 -06:00
Marc Mueller 240c9979c7
Move `disallow-any-generics` to mypy.ini (#63917) 2022-01-11 13:33:25 -08:00
Ruslan Sayfutdinov 55f4962c06
Fix pylint plugin which checks relative imports (#62693) 2021-12-23 11:14:47 -08:00
J. Nick Koston 6576225c48
Log unhandled loop exception traces when asyncio debug is on (#57602) 2021-10-18 20:07:51 -07:00
Marc Mueller 238b488642
Use assignment expressions 03 (#57710) 2021-10-17 20:08:11 +02:00
J. Nick Koston 9831ff0487
Avoid deadlock on shutdown when a task is shielded from cancelation (#56499) 2021-09-21 14:35:14 -07:00
J. Nick Koston 1ba5c1c9fb
Fix deadlock at shutdown with python 3.9 (#52613) 2021-07-07 09:23:24 +02:00
J. Nick Koston 04a0ca14e0
Ensure shutdown does not deadlock (#49282) 2021-04-18 08:55:51 -10:00
Marc Mueller e55702d635
Update typing 01 (#48013) 2021-03-17 17:34:55 +01:00
Ville Skyttä 94dbcc9d2b
Generics and other type hint improvements (#45250) 2021-01-18 22:23:25 +01:00
Ville Skyttä 43474762b2
Drop remaining Python < 3.8 support (#44743)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
Co-authored-by: Franck Nijhof <git@frenck.dev>
2021-01-04 11:47:29 +01:00
Paulus Schoutsen 819dd27925
Automatically clean up executor as part of closing loop (#43284) 2020-11-16 15:43:48 +01:00
Ville Skyttä 1bf2c4d976
Upgrade pylint to 2.6.0 (#39363) 2020-08-29 07:59:24 +02:00
Franck Nijhof 1c2ebdf307
Upgrade black to 20.8b1 (#39287) 2020-08-27 13:56:20 +02:00
J. Nick Koston a6cec21c43
Make executor max_workers consistent between python versions (#38821)
The default on python 3.8 is for max_workers is significantly
lower than the default on python 3.7 which means we can get starved
for workers.

To determine a reasonable maximum, the maximum was increased to large
number on 5 production instances.

The number of worker threads created during startup that were
needed to avoid waiting for a thread:

  HOU 1 - 71
  HOU 2 - 48
  OGG 1 - 60
  OGG 2 - 68
  OGG 3 - 64

This lead to a selection of 64 as it was reliable in all cases
and did not have a significant memory impact
2020-08-13 10:12:18 +02:00
Ville Skyttä aa1c5fc43d
Various type hint improvements (#37952) 2020-07-22 08:06:37 -07:00
Paulus Schoutsen c47fd90eb3 Fix loop exception handler 2020-07-07 00:00:46 +00:00
Paulus Schoutsen f49ce5d1b4
Protect loop set default executor (#37438)
Co-authored-by: J. Nick Koston <nick@koston.org>
2020-07-06 15:58:53 -07:00