changed linker files to put application at 32kB offset
added VECT_TAB_OFFSET define in targets.py so application runs properly after jump from bootloader
Test suite should report IOERR_COPY when shell copy command fails (fixed).
2. Host test shell plugin update
How you can specify with option -c value 'shell' and copy plugin will detect
for you your OS and choose between 'cp' (for 'posix') and 'copy' (for
Windows).
Description
===========
Added thread model for function SingleTest::execute where for each target we
have separate thread.
Added new experimental feature. Currently we are only building in parallel
mbed SDK and libraries with dependencies.
Each MUT is flashed separately, but with this new feature we are able to
flash multiple boards at the same time and get results in parallel.
Test execution time is reduced by n where n is number of MUTs (MUT: mbed under
test).
using --parallel means that:
* separate thread will be spawned to build mbed SDK and dependencies (libraries)
for each unique target with all declared toolchains.
E.g:
Thread(n==1): LPC1768: ARM, uARM, GCC_ARM
Thread(n==2): K64F: ARM, GCC_ARM
Thread(n==3): NUCLEO_X: uARM
.
.
.
Thread(4): for toolchain in NUCLEO_Z[toolchains]
-> build mbed SDK
-> build libs
-> build project
-> run MUT testrunner
return test results for NUCLEO_Z[toolchain]
Done:
1. Added option --parallel.
2. Decoupled execute function so it can be run in parallel with other execute
functions.
3. Thread join via Queue, not Thread::Join() to avoid deadlocks or waits for
particular thread to finish.
4. All builds are in parallel, but because each target and library for each
toolchain have different directory we do not worry about building in
parallel and compiler collisions.
Missing:
1. No sync for 'print' (TODO).
2. No sync on test result structures - not needed because we only append to
them (?).
Experimental --parallel bugfix: self.test_suite_properties_ext dict was not
populated with target name.
1. Removed globaly initialized data inside class test() function
2. Removed global variables initialization dependency. Any cause some Python
implementations and configurations to fail in runtime
3. Added info about Echo port #7 rationale.
4. Testsed with K64F and network tests:
Test summary:
+--------+--------+---------+----------------------------+--------------------+
| Result | Target | Test ID | Test Description | Elapsed Time (sec) |
+--------+--------+---------+----------------------------+--------------------+
| OK | K64F | NET_1 | TCP client hello world | 3.26 |
| OK | K64F | NET_13 | TCP client echo loop | 2.05 |
| OK | K64F | NET_2 | NIST Internet Time Service | 3.43 |
| OK | K64F | NET_3 | TCP echo server | 1.54 |
| OK | K64F | NET_4 | TCP echo client | 1.54 |
| OK | K64F | NET_5 | UDP echo server | 1.46 |
| OK | K64F | NET_6 | UDP echo client | 1.6 |
| OK | K64F | NET_7 | HTTP client hello world | 3.4 |
| OK | K64F | NET_8 | NTP client | 2.39 |
+--------+--------+---------+----------------------------+--------------------+
Result: 9 OK
Completed in 122.18 sec
Adde missing check for optional auto_detect parameter in CLI options.
Test procedure:
* Check with mbed-ls installed:
$ singletest.py --auto -j 8
MBEDLS: Detecting connected mbed-enabled devices...
MBEDLS: Detected K64F, port: COM61, mounted: E:
Building library CMSIS (K64F, ARM)
Copy: startup_MK64F12.o
Copy: sys.o
Copy: cmsis_nvic.o
Copy: system_MK64F12.o
* Uninstall mbed-ls iools:
$ pip uninstall mbed-ls
Uninstalling mbed-ls:
c:\python27\lib\site-packages\mbed_ls-0.1.4-py2.7.egg
c:\python27\scripts\mbedls-script.py
c:\python27\scripts\mbedls.exe
c:\python27\scripts\mbedls.exe.manifest
Proceed (y/n)? y
Successfully uninstalled mbed-ls
$ mbedls
'mbedls' is not recognized as an internal or external command,
operable program or batch file.
$ python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mbed_lstools
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named mbed_lstools
* Check singletest.py work flow without mbed-ls:
$ singletest.py -i test_spec.json -M muts_all.json
Building library CMSIS (K64F, ARM)
Building library MBED (K64F, ARM)
Building project DETECT (K64F, ARM)
TargetTest::K64F::ARM::DTCT_1::Simple detect test [OK] in 0.50 of 10 sec
Building project DEV_NULL (K64F, ARM)
TargetTest::K64F::ARM::EXAMPLE_1::/dev/null [OK] in 3.49 of 20 sec
Building project HELLO (K64F, ARM)
TargetTest::K64F::ARM::MBED_10::Hello World [OK] in 0.38 of 5 sec
Building project TICKER (K64F, ARM)
TargetTest::K64F::ARM::MBED_11::Ticker Int [OK] in 11.35 of 15 sec
Boolean value used to calclate single RTC test PASS / FAIL was not used to
calculate overall result.
==== ISSUE ====
The test suite treats the RTC test as successful, but test itself reports
failures. Which one is correct – does it fail or does it pass? See below:
HOST: Run test...
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 4.02 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.03 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
{{success}}
{{end}}
Test::Output::Finish
TargetTest::ARM_MPS2_M4::ARM::MBED_16::RTC [OK] in 30.37 of 40 sec
==== FIX ====
HOST: Run test...
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 0.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
MBED: [0] [1970-01-01 00:00:00 AM]
HOST: [0] [1970-01-01 00:00:00 AM] received time +0 sec after 1.04 sec... FAIL
{{failure}}
{{end}}
Mike Fiore
Martin Kojtal
bcostm
ohagendorf
Przemek Wirkus
explora26
Mihail Stoyanov
GustavWi
Masao Hamanaka