It is possible that the difference between base and next tick count on some platforms is greater than 1, in this case we need to repeat counting with the reduced number of cycles (for slower boards).
If number of cycles has been reduced than base tick count needs to be redefined. This operation is missing and is added by this patch.
It has been noticed on NRF51_DK board that occurrence of system tick have impact on test execution.
NRF51_DK is a slow board with fast us ticker and handling of system tick interrupt takes about 250 us ticker ticks which have huge influence on interrupt tests where interrupt is set to <current count> + 100, 200, 500 ticks.
This test fails sometimes while testing 1 ms timeout on NRF51_DK board.
This board is very slow and lp ticker is inaccurate.
I noticed that measured results are very close to the tolerance value. For 1ms delay we measure in failure case 1560 us and tolerance is set to 550.
Since measurement method is also significantly flawed (extra time for setting up ticker and execution of a ticker irq handler) I recommend to increase tolerance by least +100 us for now.
Since now us ticker is disabled in deep-sleep mode and tests-mbed_drivers-timerevent verifies TimerEvent class which uses us ticker then deep-sleep mode needs to be disabled during the test execution.
On some boards without SYSTICK support lp ticker is used to trace system ticks.
Event queue uses system ticker to count delay when callback is to be fired.
New ticker implementation for NRF51_DK provides fast and accurate counter for us ticker which shows inaccuracy of the system ticker(which is based on slow and inaccurate lp ticker). This is why measured error grows linearly and percentage form needs to be used.
Currently we use constant tolerance value equal to: 5 [ms].
This patch modifies tolerance to the following form: 5% of measured time [ms].
In this test the lowest delay is equal to 100 [ms], so 5% corresponds to 5 [ms] tolerance - original value. This means that min tolerance is 5 [ms] so this change should not have negative impact on other targets.
For NR51_DK US_TICKER_OV_LIMIT needs to be increased since if test is run few times in row sometimes fails. This is because NR51_DK is a slow board (16 MHz) with fast and short us ticker counter 1 MHz/16 bits.
Since this change concerns new ticker driver for NRF51_DK it goas on ticker feature branch.
On this branch us ticker is based on 1 MHz higher precision counter (on master is based on 32kHz RTC). As a result of using higher precision counter measurement error increased:
:485::FAIL: Expected 1.060000 Was 1.070478
For delay 1060 ms we measured ~1070 ms so we have about ~1% measurement error which is not bed for this board.
To make it work with 1MHz us ticker increase test tolerance. Define tolerance as follows:
tolerance = 500us + 0.02 * delay.
Since this change concerns new ticker driver for NRF51_DK it goas on ticker feature branch.
On master NRF51_DK Ticker driver uses 32kHz RTC for lp and us ticker. Test uses us ticker to measure lp ticker interrupt scheduling (LowPowerTicker.attach), but since the same hardware is used for both tickers the measurement error is constant. On this branch us ticker uses 1 MHz higher precision clock and the results are different - measurement error grows linearly and shows inaccuracy of the RTC.
Test implements constant delta for measured time equal to 2000 us.
Change delta so it depends on lp ticker tested timeout value:
500us for measurement inaccuracy + 5% tolerance for LowPowerTicker
Update the ticker common test to clean up after each case by restoring
the ticker IRQ handler. This allows tickers to function normally after
each case has completed.
- count_ticks: fix counter overflow handling,
- count_ticks: use reg_cycles variable in while loop,
- increment test: reduce number of cycles for slow cores if measure process needs to be repeated (difference is greater than 1).
On slow targets with fast high frequency tickers like NRF51_DK (16 Mhz CPU/1MHz ticker) time window for test case execution without overflow needs to be increased. Add parameter to `overflow_protect` function to be able to set different time window without overflow for us ticker and lp ticker.
Increment test proves that ticker counter is incremented by one.
This is done indirectly for high frequency counters where it is impossible to read 'value' and 'value + 1' in two successive ticker reads.
This check is done indirectly by counting ticker ticks elapsed during execution of N cycles of empty while loop. Unfortunately on slow boards with fast tickers like NRF51_DK(16 MHz CPU/1MHz hf ticker) it is possible that for the same N cycles measured number of elapsed ticks in two successive calls is greater than 1. This patch provides fix for such case - measure operation is repeated with the same number of cycles.
Since target specific ticker drivers are not ready also features which uses ticker in upper layers may not work correctly and tests for these features.
We need to disable also failing higher level ticker related tests (by adding check if DEVICE_USTICKER symbol is available and raise #error [NOT_SUPPORTED] if not).
Note:
tests-mbed_drivers-rtc is new and uses us ticker to perform a delay.
count_ticks() function counts ticker ticks elapsed during execution of N cycles of empty while loop.
In current version N value (cycles) is given as volatile paramater in order to disable possible compiler optimalisation.
There was a problem with measured time on different compilers and additionally results on ARM compiler were unexpected (the difference beetween measured elapsed ticks for the same number of cycles was to large). This might be caused by the memory access in order to store updated variable in memory. To fix this issue given numer of cycles has been stored into register and register is decremented (no memory access).
With this fix count_ticks(NUM_OF_CYCLES, 1) call returns 2500 +/-1 for us ticker ticks using each compiler (K64F).
On some platforms (e.g. K64F) different counters are used for time measurement and interrupt generation.
Because of that we should relax interrupt test case and give additional time before checking if interrupt handler has been executed.
Since according to the ticker requirements min acceptable counter size is 12 bits (low power timer) for which max count is 4095, then all test cases must be executed in this time window.
HAL ticker layer handles overflow and it is not handled in the target ticker drivers.
The intention was to use ticker_overflow_delta equal to 0 for low power ticker tests and ticker_overflow_delta equal to 50 for high frequency ticker tests. Current implementation is invalid since for devices which provide LOW_POWER_TIMER feature delta is equal to 0 and for devices without this feature delta is equal 50.
Since target specific ticker drivers are not ready also features which uses ticker in upper layers may not work correctly and tests for these features.
We need to disable also failing higher level ticker related tests (by adding check if DEVICE_USTICKER symbol is available and raise #error [NOT_SUPPORTED] if not).
These boards will be re-enabled when sleep driver for them is ready.
Note:
This operation is done by removing "SLEEP" feature from target's "device_has" list (in targets.json config file).
For NRF52_DK removing of "SLEEP" feature causes some timing issues which have influence on tests. In order to successfully disable this board we need to disable also related features like "USTICKER", "LOWPOERTIMER" and slightly modify ticker tests, so they will not be executed if usticker support is not available (by default all targets support us ticker).
From lowercase to uppercase and drops 'test_'-prefix. Test case
descriptions do also match to TC names now.
Additionally all the TCs are not behing MBED_EXTENDED_TESTS
anymore.
Starts splitting Greentea test suites to TCP and UDP suites and adds
more test cases like
tcpsocket_echotest_burst
tcpsocket_echotest_burst_nonblock
tcpsocket_endpoint_close
tcpsocket_recv_100k
tcpsocket_recv_100k
tcpsocket_recv_timeout
tcpsocket_send_timeout
tcpsocket_thread_per_socket_safety
udpsocket_echotest_burst
udpsocket_echotest_burst_nonblock
udpsocket_recv_timeout
udpsocket_sendto_timeout
With small wait time at the start, chances are that device does not
enter sleep and idle time is zero. Increasing wait time to make sure
device goes to sleep.
If user has initiated a delayed event (either with call_in or call_every),
user might need to know how much time is left until the event is
due to be dispatched.
Added time_left() function can be used to get the remaining time.
CPUID base register is available for Cortex-M processors only.
Cortex-A devices have Main ID Register, which can be used in future to
get processor info.
Currently test assumes that 1 sec is long enough to set RTC time and read same time which has been set.
In some cases extra time for synchronisation between clock domains is needed and after setting/reading operations the read value might be different than one which has been set (+1 sec).
Additionally in some cases when lp ticker is based on RTC, the RTC implementation may use mechanism to trace elapsed seconds without modifying RTC registers. In such case it is possible that second will change immediately after setting time.
Add 1 sec tolerance (min possible) for such checks.
The unified NRF51 target and feature BLE directories have been
reorganized to follow the naming and directory structure of the
NRF52 implementation.
This reorganization does not include TARGET_MCU_NRF51822 and
derived targets.
The flash clock test is disabled for the NRF52 series. This change
re-enables the test but with a higher tolerance to accommodate the
high jitter on the current ticker implementation.
Previously, the echo test followed a flow like the following:
-STEP- -HOST PC- -DEVICE-
0 send _sync
1 echo back _sync
2 send echo_count
3 echo back echo_count
4 send first echo packet
5 echo back echo packet
(repeat echo steps)
However, as noted by issue #6659, this test would somtimes fail between
steps 4 and 5. To ensure each KV pair makes to the correct destination,
we usually write the KV back. Step 4 does not wait for this to happen
and starts sending echo packets. So the device is acting as the "echo
server".
This change makes the host PC the "echo server". The idea being that the
device will be slower and the host pc should always be able to keep up
with it, not the other way around.
Update the SysTimer test to match the updated API. Changes are:
- increment_tick() renamed to _increment_tick() and explicitly
synchronized
-update_tick() replaced with resume() and a call to suspend() was added
before this
sleep_manager_racecondition test fix for devices with low CPU clock
This RP contains fix for sleep_manager_racecondition test
for very slow devices (like NRF51). It fixes the test itself
as well as side effects of fix introduced in
#5046 (us ticker: fix fire interrupt handling)
The idea of the test was to test race condition between main thread
and interrupt handler calling the same function.
To efficiently test this, each handler call should interrupt
main thread to make race more likely.
On very slow devices (like NRF51) when we set very low ticker period
(e.g less then 1000us for NRF51) there is no much time for thread scheduling.
On such slow devices, setting period to 500 us cause that
main thread is scheduled very rarely and only handler is
constantly called making test unreliable.
Fix introduced in #5046 (us ticker: fix fire interrupt handling)
changed fire_interrupt function implementation causing more
interrupt tailing thus even less time for main thread scheduling.
After introduction of #5046 (us ticker: fix fire interrupt handling)
when running sleep_manager_racecondition test on NRF51
(with ticker1.attach_us(&sleep_manager_locking_irq_test, 500);)
test is failing with timeout due to the fact that interrupt
handler is constantly called and main thread is never scheduled.
In the 'Testing accuracy of equeue semaphore' test case result is printed out in each loop iteration.
Since debug prints should not exist in the final test version I suggest to print information only in case of failure.
Additionally time needed to print single info is equal to ~25 ms (K64F/GCC_ARM). The while loop is designed to execute until 20000 ms elapses, so this print has also impact on number of times the loop is executed (number of semaphore accuracy checks).
The features/mbedtls/targets/TARGET_STM/* files include constant needed
for the error codes returned from the MD functions.
The features/mbedtls/targets/hash_wrappers.c provides thin redirection
layer for the hardware accelerated MD implementations that rely on the
old API.
The TESTS/mbedtls/multi/main.cpp has been changed to use the new API
as its build environment does not rely on the translation unit
containing the necessary wrappers.
A call to
`TCPSocket::recv(void *data, nsapi_size_t size)`
returns, following the mbed documentation, the number of received bytes on
success, and a negative error code on failure.
So in case of success, the return value depends on both the value of parameter
`size` but also on the amount of data already available. This means, that the
value returned can be lower than or equal to the `size` of the `data` buffer
passed as argument to the call.
Therefore, in the cases of `test_tcp_hello_world()` & `find_substring()`
(i.e. test `socket_sigio`), the calls to `TCPSocket::recv()` might return from
one byte up to `sizeof(buffer) - 1` (i.e. 511) bytes for each single call,
while the tests expect to receive the whole response string with a single call.
This commit applies a fix to this situation by implementing a receive loop
which exits once there is no data anymore available to be read from the socket.
test_case_2x_callbacks test was redesigned to eliminate ticker rescheduling and improve time mesure accuracy.
Constant ticker rescheduling (detach()/attach_us() calls)
was causing the gap between consecutive callback calls was not exact 1ms
but 1ms + time needed to call the callback and attach new one.
New design just uses two tickers to update counter alternatively every 1ms without rescheduling them
This commit fixes ticker cross-schedule bug in test_case_2x_callbacks subtest
In effect of this bug:
ticker_callback_1_switch_to_2 was called only once
ticker2 was never been fired because it was repeatedly detached just before fire and attached again
The current 0.1% clock tolerance is too small for certain platforms
which natural variance is larger than this. This commit increases the
tolerance to 0.5% instead.
Sample output from time_cpu_cycles called repeatedly in init,
before running any flash tests:
[1515706585.63][CONN][RXD] diff: 49316
[1515706585.69][CONN][RXD] diff: 49256
[1515706585.75][CONN][RXD] diff: 49286
[1515706585.81][CONN][RXD] diff: 49256
[1515706585.87][CONN][RXD] diff: 49225
[1515706585.94][CONN][RXD] diff: 49286
[1515706585.99][CONN][RXD] diff: 49317
[1515706586.06][CONN][RXD] diff: 49255
[1515706586.12][CONN][RXD] diff: 49286
[1515706586.18][CONN][RXD] diff: 49285
[1515706586.24][CONN][RXD] diff: 49286
[1515706586.31][CONN][RXD] diff: 49347
[1515706586.36][CONN][RXD] diff: 49347
[1515706586.43][CONN][RXD] diff: 49286
[1515706586.49][CONN][RXD] diff: 49286
[1515706586.55][CONN][RXD] diff: 49256
[1515706586.61][CONN][RXD] diff: 49286
[1515706586.68][CONN][RXD] diff: 49346
[1515706586.74][CONN][RXD] diff: 49347
[1515706586.80][CONN][RXD] diff: 49256
Notice the outliers will cause intermittent CI failures.
For each of the following years test example time of the first and last day of each month:
- first - 1970
- example not leap year (not divisible by 4)
- example leap year (divisible by 4 and by 100 and by 400)
- example leap year (divisible by 4 and not by 100)
- example not leap year (divisible by 4 and by 100)
- last fully supported - 2105
Test execution time on K64F is now ~39 sec.
In pararell threads of "thread" test is a simple thread, but it seems that there is a difference in the stack used between Cortex-A and Cortex-M.
As a result of check, in Cortex-A, program aborts because of the lack of stack when PARALLEL_THREAD_STACK_SIZE is 384, workes properly when PARALLEL_THREAD_STACK_SIZE is 512 at least.
"malloc" test is used a simple thread that executes only malloc and free, but it seems that there is a difference in the stack used between Cortex-A and Cortex-M.
As a result of check, in Cortex-A, program aborts because of the lack of stack when THREAD_STACK_SIZE is 256, workes properly when THREAD_STACK_SIZE is 512 at least. Since it seems that the definition processing of "DEFAULT_STACK_SIZE" was gone from Mbed OS by updating CMSIS5/RTX5, I setted 512 directly for Cortex-A.
Most of our IP stacks don't allow removal of interfaces so
interface destructor can not reliably clean up. Therefore we
cannot rely its behaviours in test case.
Instead run interface->disconnect() in case interface was already
created.
HTTP_SERVER_NAME and HTTP_SERVER_FILE_PATH are replaced by
MBED_CONF_APP_HTTP_SERVER_NAME and MBED_CONF_APP_HTTP_SERVER_FILE_PATH
to allow user to make local tests
Test on HTTP connect added as test was always OK even with no connection...
HTTP_SERVER_NAME and HTTP_SERVER_FILE_PATH are replaced by
MBED_CONF_APP_HTTP_SERVER_NAME and MBED_CONF_APP_HTTP_SERVER_FILE_PATH
to allow user to make local tests
UUID lines are removed as they were not used
default case:
If MBED_CONF_APP_ECHO_SERVER_ADDR and MBED_CONF_APP_ECHO_SERVER_PORT are not defined
test is using Greentea to get server information (code before OS 5.6.1 version)
TCP_ECHO_PREFIX is no more a mandatory step
UUID lines are removed as they were not used
default case:
If MBED_CONF_APP_ECHO_SERVER_ADDR and MBED_CONF_APP_ECHO_SERVER_PORT are not defined
test is using Greentea to get server information (code before OS 5.6.1 version)
The test for construction of local static objects doesn't check
to see if the object returned has been fully initialized. Because
of this, an error with ARMCC was not detected.
This patch adds an assert to the race_test to ensure that the
object has been properly initialized.
LowPoterTimer test gives sometimes failed result while testing measured time accumulation. The check which verifies if total number of elapsed milliseconds is valid fails. Test assumes that delta value equal to 1 ms is sufficient for all test cases, which is not true since in case where time measurement is performed few times in sequence the measurement error also accumulates and 1 ms might be not enough. To solve this problem delta value for milliseconds tests must be updated.
Przemyslaw Stekiel
Bartek Szatkowski
jeromecoutant
Russ Butler
Russ Butler
Veijo Pesonen
Juha Ylinen
Cruz Monrreal
Kevin Bracey
deepikabhavnani
Senthil Ramakrishnan
Asif Rizwan
Mika Leppänen
Michael Zhang
Amanda Butler
Martin Kojtal
Kimmo Vaisanen
David Saada
Marcus Chang
Andrzej Kurek
Brian Daniels
Filip Jagodzinski
Cruz Monrreal
Martin Kojtal
Shrikant Tudavekar
Maciej Bocianski
Brendan McDonnell
Krzysztof Stachowiak
Wolfgang Betz
Jimmy Brisson
Anna Bridge
TomoYamanaka
Seppo Takalo
Sarah Marsh
Przemyslaw Stekiel