83d7444331
Merge pull request #6886 from cmonr/additional-py3-fixes
...
Corrected iteritems py2/3 compatability in test_api.py
2018-05-15 10:08:28 -05:00
daa0f9bc35
Corrected iteritems py2/3 compatability in test_api.py
2018-05-11 13:17:48 -05:00
badd753466
mbed test: add argument `--ignore` to allow passing in extra mbedignore args
...
In particular this allows ignoring a project main.cpp file when running unit tests
2018-05-09 10:02:02 +10:00
53d3c4344f
Merge pull request #6781 from theotherjimmy/refactor-notify
...
tools: Refactor notification API
2018-05-07 10:48:48 -05:00
c186c3ce86
Use mocked notifier for individual tests
...
That way we separate the collection of notifications from everything else
2018-05-03 10:06:47 -05:00
0f7e900417
Pass app config to get config
...
This corrects a bug which would manifest as --app-config not having
much of an affect on test builds (not the first libary bulid)
2018-05-02 09:38:44 -05:00
dc006d6aa3
Remove silent from test invocation
...
apparently I forgot earlier
2018-04-30 14:44:02 -05:00
de91c93108
Use new notifier API in test infra
2018-04-26 09:32:43 -05:00
b286838ebf
Updated python tools to be cimpliant with PEP 3113
2018-04-10 13:37:47 -05:00
1dd39fbe80
Python2+3: mbed test --compile
2018-02-05 11:07:37 -06:00
cca4425af6
Python2+3: iteritems -> items
2018-02-05 11:04:36 -06:00
45bdd98a85
Python2+3: tests import correctly
2018-02-05 11:04:36 -06:00
d82d9888d4
Added COMMON folder for tests
...
A COMMON folder allows code reuse across different test cases. This
avoids code duplication or code enterying the application space.
The COMMON folder is uppercase to match naming conventions in Mbed OS.
2018-01-09 16:11:14 -06:00
b3e8dec8ea
Updated get_test_config function returns comment
2017-11-13 15:01:11 -06:00
da78647c1d
Merge pull request #5077 from dhwalters423/dhwalters423-add-detect-info
...
Added interface version information to mbed detect command.
2017-10-05 11:11:33 -05:00
6a44ceb094
Add copyright headers and use mbed error to skip tests without network config files
2017-09-20 18:31:06 -05:00
4161e5c48d
Move test config keys out of targets.json
...
Change TestConfig class methods to module methods
2017-09-20 18:30:08 -05:00
ba6eb98b5c
Use only one flag for test configuration file
...
Tools will figure out if user passed in a custom path (to indicate configration file for module) or whether they used a keyword (to indicate they'd like to use an mbed OS configuration)
2017-09-20 18:29:37 -05:00
e982eed4ad
Add option to add a configuration file for modules
2017-09-20 18:27:59 -05:00
8c0bc781d9
Allow tools to use networkinterface configs in tests
2017-09-20 18:27:59 -05:00
e4b55dd6a0
Removed get_mounted_details_txt from test_api.py
...
Fixed minor typos in test_api.py
Added get_interface_version to detect_targets
2017-09-15 10:15:24 +01:00
d43a847667
Added additional DAPLink version info for `mbed detect` command`
2017-09-11 16:54:32 +01:00
5bcd97a2bc
Merge pull request #4984 from theotherjimmy/unify-unit-tests
...
Unify mbed OS tools testing style with what's used in mbed-ls, htrun, greentea
2017-09-06 14:26:14 -05:00
b99e6b5f7a
Pass stats depth to singletest correctly
...
Fixes #4930
2017-08-31 10:41:20 -05:00
a2bcae0b7a
Rework test_api testing and fix a bug
...
Bug was found by the new tests
2017-08-28 15:40:35 -05:00
7053ef9412
memap enhancements with depth level configurable
...
Removed heap/stack from report
Add --stats-depth option to mbed compile
Fix minor issues identified in review
Fix bug when parsing armcc libs
Fix code style
Fix minor issues on existing tests
Fix memap for automated TESTS
Fix stats-depth parameter for tests
2017-08-06 10:30:04 +01:00
b2901098cd
Update test_api call to prepare_toolchain
...
An earlier patch in this series changed the API for
`build_api.prepare_toolchain`. This commit updates the `find_test`
function to call `prepare_toolchain` correctly.
2017-03-06 18:37:46 -06:00
83e843bd8c
Fix access before variable defined bug in test_api
...
The worker_result variable was not guaranteed to have a `'result'` or
`'reason'` key and accessing them before testing for them could result
in an error when they are not provided. This patch changes the checks to
prevent check for their existence before accessing them.
2017-02-08 13:56:43 -06:00
3baeb8a8a7
Preventing app_config from clobbering CLI macros
2017-01-03 13:27:12 -05:00
57e693de86
Passing toolchain path info to subprocesses.
...
Since the toolchain path info lives as a global variable, its not
propigating to the subprocesses that are created to compile the tests in
parallel. This change manually passes these global variables and then
reassigns them.
2016-11-17 12:20:34 -08:00
9d8ec61df5
Merge pull request #3258 from sarahmarshy/build_test_err
...
Prevent process pool inside of process pool when building tests
2016-11-14 21:54:24 -06:00
9f39132d3b
Prevent process pool inside of process pool when building tests
2016-11-10 14:41:19 -07:00
e3ff8da6f0
Test spec baud_rate now set based on mbed config
2016-11-10 15:21:18 -06:00
0e4b841cb3
Removing need for memap to printed through toolchains
2016-10-11 18:24:01 -05:00
655c37715c
Handling output of parallelized test building.
...
This makes use of the reports generated by the building of tests to
prevent output from interleaving when the build is parallelized. This
required some changes to memap to return a generated string from
the 'generate_output' function. I also had an option to stop the prints
from memap to prevent text from interleaving
2016-10-11 15:34:20 -05:00
87103abd56
Making building and linking tests fully parallel.
...
This uses similar code that is used withing the toolchains to parallelize
the linking process of all the tests accross all the available CPUs. It
also respects the `-j` parameter if you wish to limit the number of cores
used.
2016-10-11 13:24:50 -05:00
e0b0ad05cb
Adding build profiles configuration to build_release.py and singletest.py
2016-09-30 16:54:34 -05:00
b3522fe9d4
Merge pull request #2855 from bridadan/handle-exceptions-better
...
[Tools] Handling Exceptions better when building tests
2016-09-30 15:23:08 -05:00
d88852d150
Replace -o with profiles
2016-09-29 10:20:05 -05:00
cadcc80371
Handling Exceptions better when building tests
...
Previously, when building tests with test.py, if an exception occurred, the
error message would be masked by the function 'build_tests'. This commit
handles NotSupportedExceptions and ToolExceptions, but lets all other
Exceptions propigate up to the caller function. In most cases, this is the
CLI scripts, which will print a traceback. This will allow us to better
debug the python tools if errors occur.
2016-09-28 14:13:23 -05:00
7c72a22508
Add test command switch for app config file
2016-09-10 15:29:19 +01:00
da5a19f149
Merge pull request #2630 from sarahmarshy/patch-4
...
Test names not dependent on disk location of root
2016-09-10 07:25:04 -05:00
ac11d94724
Test names not dependent on disk location of root
...
#2613 should be merged first
Using test_api, I found that the test names were dependent on where mbed-os (if that is the root) is stored on disk if you provide anything other than '.' as the root directory.
This would change names like:
```
repos-mbed-os-example-blinky-mbed-os-features-storage-feature_storage-tests-cfstore-example3
```
to
```
features-storage-feature_storage-tests-cfstore-example3
```
2016-09-06 10:23:32 -05:00
8b9a6dc19d
Correctly providing directories to build_apis
...
The shared `prepare_toolchain` and `scan_resources` functions in build_api
expect a list, not a string. This is different from the
toolchain.scan_resources function. Now the functions are being used
correctly within the `find_tests` function in test_api
2016-09-02 11:32:11 -05:00
64928b02df
Merge pull request #2244 from bridadan/test-discovery-refactor
...
Test discovery refactor
2016-07-26 15:45:41 -05:00
f9d38bff37
Merge pull request #2062 from bridadan/shorten-test-binary-name
...
Changing name of test binaries to just test case folder name.
2016-07-26 00:24:08 -05:00
1618e608e7
Test discovery now uses build and toolchain logic
...
Previously, test discovery throught test.py used its own logic to find
tests. This was mostly a subset of the rules used during the build
process. It respected .mbedignore files, but it did not respect TARGET_,
TOOLCHAIN_, and FEATURES_ directories. This change now uses the same logic
used during building to respect these folders. Tests under these folders
that are not being pulled in for the given target will be ignored.
2016-07-25 14:43:52 -05:00
74b7f9e923
mbed Online Build System support:
...
* added/improved global chroot support
* added RESPONSE_FILES flag to support optional response files (on linux the cmd param length is 2 megabytes). Default True
* added unified handling for archive and link response file (similar to includes)
* added COMPILE_C_AS_CPP flag to support compiling of c files as cpp. Default False
* added mbedToolchain.init() for post __init__ hooks
* added caching to mbedToolchain.need_update() to reduce IO hits
* added support to identify compiler warning/error column (supports ARMCC, GCC and IAR). Errors/warnings now report file@line,col
* added global TOOLCHAIN_PATHS which allows overriding/changing of the toolchain paths. Also simplified ARM-related paths
* added target.json to mbed library release (by @0xc0170)* migrated compile_worker() to utils.py for lightweight thread initialization
* improved run_cmd() performance by removing unnecessary check about the command being executed (should be checked once in the relevant toolchain instead)
* removed remnants of Goanna support (should be reimplemented as hooks to compile/link/archive instead)
* fixes for Python 2.7 compatibility (by @0xc0170)
* fixes for Exporters (by @0xc0170)
2016-07-19 11:14:42 +01:00
0ec8199c55
Revert "Revert "Generalize flag handling""
...
This reverts commit 33cec194c6
2016-07-07 15:29:36 -05:00
12a01f61ca
Add summary for test building
...
When users are building tests with `mbed compile --tests` whey will by default
get memory map breakdown report.
This can be suppresed in the future with command line switch. For now it is
visible each time users build test cases.
List is sorted by project name created with `build_project` API.
Changes:
* `build_project` now returns tuple (this breaks build_api.build_project API!)
* Memmap script got a aggregation function to print summary from small 'JSON'
partial reports.
* Report is generated by `test_api.build_tests` function only!
Example:
```
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| name | target | toolchain | total_flash | static_ram | stack | heap | total_ram |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| features-feature_ipv4-tests-mbedmicro-net-nist_internet_time_service | K64F | GCC_ARM | 132136 | 62288 | 32768 | 65536 | 160592 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_echo | K64F | GCC_ARM | 125613 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_hello_world | K64F | GCC_ARM | 125949 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-udp_echo_client | K64F | GCC_ARM | 123613 | 62276 | 32768 | 65536 | 160580 |
| features-storage-tests-cfstore-add_del | K64F | GCC_ARM | 96080 | 13052 | 32768 | 65540 | 111360 |
| features-storage-tests-cfstore-close | K64F | GCC_ARM | 95520 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-create | K64F | GCC_ARM | 99144 | 13036 | 32768 | 65540 | 111344 |
| features-storage-tests-cfstore-example1 | K64F | GCC_ARM | 98592 | 12368 | 32768 | 65536 | 110672 |
| features-storage-tests-cfstore-example2 | K64F | GCC_ARM | 95232 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example3 | K64F | GCC_ARM | 95264 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-example4 | K64F | GCC_ARM | 92632 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example5 | K64F | GCC_ARM | 92344 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-find | K64F | GCC_ARM | 96344 | 13028 | 32768 | 65540 | 111336 |
| features-storage-tests-cfstore-find2 | K64F | GCC_ARM | 93192 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-flash | K64F | GCC_ARM | 97784 | 12532 | 32768 | 65540 | 110840 |
| features-storage-tests-cfstore-flush | K64F | GCC_ARM | 96464 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-flush2 | K64F | GCC_ARM | 95056 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-init | K64F | GCC_ARM | 93120 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-misc | K64F | GCC_ARM | 96808 | 12516 | 32768 | 65540 | 110824 |
| features-storage-tests-cfstore-open | K64F | GCC_ARM | 98632 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-read | K64F | GCC_ARM | 94112 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-write | K64F | GCC_ARM | 94488 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-flash_journal-basicapi | K64F | GCC_ARM | 104712 | 21236 | 32768 | 65540 | 119544 |
| frameworks-utest-tests-unit_tests-basic_test | K64F | GCC_ARM | 71534 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_async_validate | K64F | GCC_ARM | 74598 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_control_async | K64F | GCC_ARM | 74630 | 11476 | 32768 | 65540 | 109784 |
| frameworks-utest-tests-unit_tests-case_control_repeat | K64F | GCC_ARM | 72790 | 11452 | 32768 | 65540 | 109760 |
| frameworks-utest-tests-unit_tests-case_selection | K64F | GCC_ARM | 72302 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_setup_failure | K64F | GCC_ARM | 72630 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_teardown_failure | K64F | GCC_ARM | 72790 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-control_type | K64F | GCC_ARM | 82462 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_async_scheduler | K64F | GCC_ARM | 72182 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_scheduler | K64F | GCC_ARM | 71998 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_assertion_failure_test_setup | K64F | GCC_ARM | 71710 | 11460 | 32768 | 65540 | 109768 |
| frameworks-utest-tests-unit_tests-test_setup_case_selection_failure | K64F | GCC_ARM | 71702 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_setup_failure | K64F | GCC_ARM | 71710 | 11468 | 32768 | 65540 | 109776 |
| tests-integration-basic | K64F | GCC_ARM | 67566 | 10780 | 32768 | 65540 | 109088 |
| tests-integration-threaded_blinky | K64F | GCC_ARM | 68326 | 10780 | 32768 | 65540 | 109088 |
| tests-mbed_drivers-c_strings | K64F | GCC_ARM | 74438 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-callback | K64F | GCC_ARM | 88310 | 11972 | 32768 | 65540 | 110280 |
| tests-mbed_drivers-dev_null | K64F | GCC_ARM | 90213 | 10784 | 32768 | 65540 | 109092 |
| tests-mbed_drivers-echo | K64F | GCC_ARM | 71918 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-generic_tests | K64F | GCC_ARM | 77624 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-rtc | K64F | GCC_ARM | 85854 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-stl_features | K64F | GCC_ARM | 80726 | 11476 | 32768 | 65540 | 109784 |
| tests-mbed_drivers-ticker | K64F | GCC_ARM | 70974 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_2 | K64F | GCC_ARM | 70790 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_3 | K64F | GCC_ARM | 71038 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-timeout | K64F | GCC_ARM | 70886 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-wait_us | K64F | GCC_ARM | 70414 | 11308 | 32768 | 65540 | 109616 |
| tests-mbedmicro-mbed-attributes | K64F | GCC_ARM | 71534 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-mbed-call_before_main | K64F | GCC_ARM | 73112 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-cpp | K64F | GCC_ARM | 73400 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-div | K64F | GCC_ARM | 73176 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-basic | K64F | GCC_ARM | 68390 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-isr | K64F | GCC_ARM | 74480 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-mail | K64F | GCC_ARM | 74992 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-mutex | K64F | GCC_ARM | 74048 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-queue | K64F | GCC_ARM | 74912 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-semaphore | K64F | GCC_ARM | 74296 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-signals | K64F | GCC_ARM | 74328 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-threads | K64F | GCC_ARM | 75214 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-rtos-mbed-timer | K64F | GCC_ARM | 68430 | 10780 | 32768 | 65540 | 109088 |
| tests-storage_abstraction-basicapi | K64F | GCC_ARM | 107808 | 28908 | 32768 | 65540 | 127216 |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
```
Refactored after code review
Refactored parse() function
Polishing
Moved memory usage reporting function call to test.py to group all reporters in one place
Bug-fix: on ARM toolchain we were not fetching statistics from last element of memap result list
2016-07-07 08:35:20 +01:00
Cruz Monrreal
Cruz Monrreal II
Andrew Leech
Jimmy Brisson
Christopher Haster
Jenny Plunkett
Sarah Marsh
David Walters
Martin Kojtal
Marcelo Salazar
Brian Daniels
Sam Grove
ARM
Conor Keegan
sarahmarshy
Mihail Stoyanov
Przemek Wirkus