When the define LPTICKER_DELAY_TICKS is set deep sleep can be randomly
disallowed when using the low power ticker. This is because a Timer
object, which locks deep sleep, is used to protect from back-to-back
writes to lp tickers which can't support that. This causes tests which
assert that deep sleep is allowed to intermittently fail.
To fix this intermittent failure this patch adds the function
sleep_manager_can_deep_sleep_test_check() which checks if deep sleep
is allowed over a duration. It updates all the tests to use
sleep_manager_can_deep_sleep_test_check() rather
than sleep_manager_can_deep_sleep() so the tests work even if deep
sleep is spuriously blocked.
I'm submitting a test plan for Mbed OS sockets, so that we have
reference that describes wanted behaviour, required test environment,
and priorities of test cases.
This servers as a both documentation and specification. It needs to live
with source code, as opposed to Handbook, because it will be extend, modified
and fixed for faster cycle than what should be done for Handbook.
Part of this document (How to run -section) will be submitted to Handbook
with links to this.
Test is giving problems in some networks. Could be that DNS server
is regarding repeated DNS enquiries on the test to be caused by
malfunctioning device.
This is solution to issue #7724.
Ticker free tests have been removed since ticker free function has been implemented only for CI boards.
Implementation for the remaining platforms will be done on feature branch feature-hal-ticker-free.
RTX waits, such as sem.wait(N), delays for between N-1 and N cycles.
As lp_ticker is not so accurate as us_ticker,
sometines, wait is expiring just before the lp_timeout.
In `ticker speed` test case execution time of ticker API functions is measured using Timer object. Test replaces original ticker handler for testing purposes, so test should not relay on higher level ticker based features(like Timer).
Use low level ticker API for time measuring.
DEVICE_EMAC is defined only for boards with default Ethernet emac.
It is not defined for example for Realtek RTL8195AM board that
does not have Ethernet. Removed the check for device emac from emac
greentea tests.
Made to prevent timeout if a single test case fails. The goal is that
each test case might wait only half of the remaining time reserved for
running TCP test cases.
In mbed-os-tests-mbed_hal-common_tickers/Microsecond ticker overflow test, some targets
would fail to catch specified ticker value near overflow in time and so fail. This commit
alleviates the issue by checking ticker value range rather than one exact ticker value near
overflow.
Serial buffer must be flushed before entering deep sleep mode. In the test this is done by the additional delay which is implemented on the busy loop which decrements given value down to 0 (`void wait_cycles(volatile unsigned int cycles)`). This solution is not appropriate since it is very target specific and the cycles value has been already increased few times. Additionally very big number of loop cycles which is suitable for fast targets may take much longer on slower boards and results in test timeout.
It has been verified that 20ms is sufficient delay for the green-tea transmission. In this test we cannot simply use `wait_ms(20)` since this potentially may put board to sleep and wake up using lp ticker. The test re-initialzies the lp ticker(disables ticker interrupt) and this operation may break the schedule and time tracing by the upper layer. But we can use us ticker which is not affected by this test. The solution is to add a delay routine based on busy loop and us ticker only. This way are able to wait exactly 20 ms.
The test sometimes fails on NRF51_DK (test case: "Test: LowPowerTimer - time measurement 1 ms.") in morph tests.
The test verifies if LowPowerTimer class correctly counts elapsed time. Sometimes we got measured ~1600 us for delay 1000 us (delta 550 us).
The delay is performed using `wait_us()` function which for delays greater than or equal to 1 ms (our case) calls `Thread::wait((uint32_t)ms);`. This causes rescheduling and potentially can put board into sleep (deep sleep mode is disabled by `wait_us()`). For our test purposes we don't need rescheduling/sleep since this actions takes extra time and have influence on the time measurement accuracy.
The solution is to implement function for delay which is based on busy loop and uses us ticker. It has been verified that this solves the problem. With this fix when measurement of 1 ms is repeated 1000 times we got usually measured time equal to ~1080 us, and sometimes ~1300us (checked that this is caused by systick interrupt handling). Since this is test for drivers layer and the results are acceptable I decided to not disabling systick in the test).
As we do not include rtx_lib header file anymore, these symbols are not available.
Use core util for if ISR is active. And for OS tick, pull in os_tick header file.
Mark mbed-os-tests-mbed_hal-common_tickers/time_cpu_cycles not in-lined.
Otherwise, time_cpu_cycles may be in-lined somewhere and takes different
CPU cycles on invocation.
Increase timeout for netsocket TCP and UDP tests. Old timeout
(240 seconds) was not enough for slower devices to complete the
tests.
Accept error codes NSAPI_ERROR_CONNECTION_TIMEOUT and
NSAPI_ERROR_NO_CONNECTION when running test wifi_connect_secure_fail
The new configuration make Error history tracking switched off by default and enabled by using the config flag MBED_CONF_PLATFORM_ERROR_HIST_ENABLED.
Config flag MBED_CONF_PLATFORM_ERROR_ALL_THREADS_INFO enables printing info of all threads. This will be turned off by default.
Increases tolerance value for sleep_usticker_test to cover extra time needed
for cpu stats computation (for more details see MBED_CPU_STATS_ENABLED).
Prevent scheduling interrupt during ticker initialization (in lp_ticker_init)
while test execution.
Make multiticker test more reliable when scheduling very early interrupts.
When very early interrupt is scheduled (e.g now + 2[ticks]) then it's likely
that it won't be fired in some circumstances and as a result there is overdue
event in queue. That overdue event can be mistakenly scheduled again by
`Ticker::detach` (detach calls schedule if head was removed). That's why we
should check interrupts counter immediately after wait period and before detach loop
Updated EMAC memory manager to use libservice nsdynmemlib
for EMAC memory buffers. Located the nsdynmemlib buffer heap
to DMA safe memory bank on LPCxx boards. Optimized placement of
static variables on EMAC test environment for LPCxx boards to
maximize available memory.
In case when base tick count is different than next tick count check first if the difference is equal to 1 tick (this is what we are looking for) if not then decrease the tick count.
Repeat counting process few times before incrementing the number of cycles in case when base tick count is equal to next tick count.
RTC reset test was failing when board has been just powered and RTC reset test was executed for the first time (issue has been detected on CI). In such case RTC initialization takes more time than in futher rtc_init calls. This has impact on green-tea communication. Commands send by host immediately after init command (write, read) were not handled on the device side and no response to the host was provided.
To fix this problem test communication flow has been modified. Device sends ack to the host after RTC init is done. Host sends further RTC commands when ack from the device has been received.
Edit:
There are still some communication issues on the CI. Add ack from the device after each executed command. Increase test timeout and wait after reset.
Since some platforms use RTC for low power timer RTC may be already enabled.
Because of that we will only verify if rtc_isenabled() returns 1 in case when init is done and RTC time is set.
Test inserts event into the TimerEvent object which is scheduled 50 ms in the future. Then test thread hangs on the semaphore for 51 ms and after that time we expect that event handler has been executed. This test fails sometimes on NRF51_DK, NRF52_DK since different clocks are used for event handling and delay. TimerEvent class uses fast us ticker and semaphores are based on lp ticker (NRF51_DK) or System Tick Timer (NRF52_DK).
We assume that ticker measurement error is 10% and our 1 [ms] extra corresponds to 2% error. I suggest to increase delay to 2 [ms] (4%). This should be enough for all boards at this moment.
It is possible that the difference between base and next tick count on some platforms is greater than 1, in this case we need to repeat counting with the reduced number of cycles (for slower boards).
If number of cycles has been reduced than base tick count needs to be redefined. This operation is missing and is added by this patch.
It has been noticed on NRF51_DK board that occurrence of system tick have impact on test execution.
NRF51_DK is a slow board with fast us ticker and handling of system tick interrupt takes about 250 us ticker ticks which have huge influence on interrupt tests where interrupt is set to <current count> + 100, 200, 500 ticks.
This test fails sometimes while testing 1 ms timeout on NRF51_DK board.
This board is very slow and lp ticker is inaccurate.
I noticed that measured results are very close to the tolerance value. For 1ms delay we measure in failure case 1560 us and tolerance is set to 550.
Since measurement method is also significantly flawed (extra time for setting up ticker and execution of a ticker irq handler) I recommend to increase tolerance by least +100 us for now.
Since now us ticker is disabled in deep-sleep mode and tests-mbed_drivers-timerevent verifies TimerEvent class which uses us ticker then deep-sleep mode needs to be disabled during the test execution.
On some boards without SYSTICK support lp ticker is used to trace system ticks.
Event queue uses system ticker to count delay when callback is to be fired.
New ticker implementation for NRF51_DK provides fast and accurate counter for us ticker which shows inaccuracy of the system ticker(which is based on slow and inaccurate lp ticker). This is why measured error grows linearly and percentage form needs to be used.
Currently we use constant tolerance value equal to: 5 [ms].
This patch modifies tolerance to the following form: 5% of measured time [ms].
In this test the lowest delay is equal to 100 [ms], so 5% corresponds to 5 [ms] tolerance - original value. This means that min tolerance is 5 [ms] so this change should not have negative impact on other targets.
For NR51_DK US_TICKER_OV_LIMIT needs to be increased since if test is run few times in row sometimes fails. This is because NR51_DK is a slow board (16 MHz) with fast and short us ticker counter 1 MHz/16 bits.
Since this change concerns new ticker driver for NRF51_DK it goas on ticker feature branch.
On this branch us ticker is based on 1 MHz higher precision counter (on master is based on 32kHz RTC). As a result of using higher precision counter measurement error increased:
:485::FAIL: Expected 1.060000 Was 1.070478
For delay 1060 ms we measured ~1070 ms so we have about ~1% measurement error which is not bed for this board.
To make it work with 1MHz us ticker increase test tolerance. Define tolerance as follows:
tolerance = 500us + 0.02 * delay.
Since this change concerns new ticker driver for NRF51_DK it goas on ticker feature branch.
On master NRF51_DK Ticker driver uses 32kHz RTC for lp and us ticker. Test uses us ticker to measure lp ticker interrupt scheduling (LowPowerTicker.attach), but since the same hardware is used for both tickers the measurement error is constant. On this branch us ticker uses 1 MHz higher precision clock and the results are different - measurement error grows linearly and shows inaccuracy of the RTC.
Test implements constant delta for measured time equal to 2000 us.
Change delta so it depends on lp ticker tested timeout value:
500us for measurement inaccuracy + 5% tolerance for LowPowerTicker
Update the ticker common test to clean up after each case by restoring
the ticker IRQ handler. This allows tickers to function normally after
each case has completed.
- count_ticks: fix counter overflow handling,
- count_ticks: use reg_cycles variable in while loop,
- increment test: reduce number of cycles for slow cores if measure process needs to be repeated (difference is greater than 1).
On slow targets with fast high frequency tickers like NRF51_DK (16 Mhz CPU/1MHz ticker) time window for test case execution without overflow needs to be increased. Add parameter to `overflow_protect` function to be able to set different time window without overflow for us ticker and lp ticker.
Increment test proves that ticker counter is incremented by one.
This is done indirectly for high frequency counters where it is impossible to read 'value' and 'value + 1' in two successive ticker reads.
This check is done indirectly by counting ticker ticks elapsed during execution of N cycles of empty while loop. Unfortunately on slow boards with fast tickers like NRF51_DK(16 MHz CPU/1MHz hf ticker) it is possible that for the same N cycles measured number of elapsed ticks in two successive calls is greater than 1. This patch provides fix for such case - measure operation is repeated with the same number of cycles.
Since target specific ticker drivers are not ready also features which uses ticker in upper layers may not work correctly and tests for these features.
We need to disable also failing higher level ticker related tests (by adding check if DEVICE_USTICKER symbol is available and raise #error [NOT_SUPPORTED] if not).
Note:
tests-mbed_drivers-rtc is new and uses us ticker to perform a delay.
count_ticks() function counts ticker ticks elapsed during execution of N cycles of empty while loop.
In current version N value (cycles) is given as volatile paramater in order to disable possible compiler optimalisation.
There was a problem with measured time on different compilers and additionally results on ARM compiler were unexpected (the difference beetween measured elapsed ticks for the same number of cycles was to large). This might be caused by the memory access in order to store updated variable in memory. To fix this issue given numer of cycles has been stored into register and register is decremented (no memory access).
With this fix count_ticks(NUM_OF_CYCLES, 1) call returns 2500 +/-1 for us ticker ticks using each compiler (K64F).
On some platforms (e.g. K64F) different counters are used for time measurement and interrupt generation.
Because of that we should relax interrupt test case and give additional time before checking if interrupt handler has been executed.
Since according to the ticker requirements min acceptable counter size is 12 bits (low power timer) for which max count is 4095, then all test cases must be executed in this time window.
HAL ticker layer handles overflow and it is not handled in the target ticker drivers.
The intention was to use ticker_overflow_delta equal to 0 for low power ticker tests and ticker_overflow_delta equal to 50 for high frequency ticker tests. Current implementation is invalid since for devices which provide LOW_POWER_TIMER feature delta is equal to 0 and for devices without this feature delta is equal 50.
Since target specific ticker drivers are not ready also features which uses ticker in upper layers may not work correctly and tests for these features.
We need to disable also failing higher level ticker related tests (by adding check if DEVICE_USTICKER symbol is available and raise #error [NOT_SUPPORTED] if not).
These boards will be re-enabled when sleep driver for them is ready.
Note:
This operation is done by removing "SLEEP" feature from target's "device_has" list (in targets.json config file).
For NRF52_DK removing of "SLEEP" feature causes some timing issues which have influence on tests. In order to successfully disable this board we need to disable also related features like "USTICKER", "LOWPOERTIMER" and slightly modify ticker tests, so they will not be executed if usticker support is not available (by default all targets support us ticker).
From lowercase to uppercase and drops 'test_'-prefix. Test case
descriptions do also match to TC names now.
Additionally all the TCs are not behing MBED_EXTENDED_TESTS
anymore.
Starts splitting Greentea test suites to TCP and UDP suites and adds
more test cases like
tcpsocket_echotest_burst
tcpsocket_echotest_burst_nonblock
tcpsocket_endpoint_close
tcpsocket_recv_100k
tcpsocket_recv_100k
tcpsocket_recv_timeout
tcpsocket_send_timeout
tcpsocket_thread_per_socket_safety
udpsocket_echotest_burst
udpsocket_echotest_burst_nonblock
udpsocket_recv_timeout
udpsocket_sendto_timeout
With small wait time at the start, chances are that device does not
enter sleep and idle time is zero. Increasing wait time to make sure
device goes to sleep.
If user has initiated a delayed event (either with call_in or call_every),
user might need to know how much time is left until the event is
due to be dispatched.
Added time_left() function can be used to get the remaining time.
CPUID base register is available for Cortex-M processors only.
Cortex-A devices have Main ID Register, which can be used in future to
get processor info.
Russ Butler
Cruz Monrreal
David Saada
Seppo Takalo
Nir Sonnenschein
Mika Leppänen
Maciej Bocianski
Przemyslaw Stekiel
jeromecoutant
kegilbert
Martin Kojtal
Veijo Pesonen
ccli8
Jimmy Brisson
Deepika
Juha Ylinen
Senthil Ramakrishnan
Anna Bridge
Jarno Lamsa
Cruz Monrreal
Wilfried Chauveau
Marc Emmers
Kimmo Vaisanen
Bartek Szatkowski
Russ Butler
Kevin Bracey
Asif Rizwan
Michael Zhang
Amanda Butler