b3522fe9d4
Merge pull request #2855 from bridadan/handle-exceptions-better
...
[Tools] Handling Exceptions better when building tests
2016-09-30 15:23:08 -05:00
d88852d150
Replace -o with profiles
2016-09-29 10:20:05 -05:00
cadcc80371
Handling Exceptions better when building tests
...
Previously, when building tests with test.py, if an exception occurred, the
error message would be masked by the function 'build_tests'. This commit
handles NotSupportedExceptions and ToolExceptions, but lets all other
Exceptions propigate up to the caller function. In most cases, this is the
CLI scripts, which will print a traceback. This will allow us to better
debug the python tools if errors occur.
2016-09-28 14:13:23 -05:00
7c72a22508
Add test command switch for app config file
2016-09-10 15:29:19 +01:00
da5a19f149
Merge pull request #2630 from sarahmarshy/patch-4
...
Test names not dependent on disk location of root
2016-09-10 07:25:04 -05:00
ac11d94724
Test names not dependent on disk location of root
...
#2613 should be merged first
Using test_api, I found that the test names were dependent on where mbed-os (if that is the root) is stored on disk if you provide anything other than '.' as the root directory.
This would change names like:
```
repos-mbed-os-example-blinky-mbed-os-features-storage-feature_storage-tests-cfstore-example3
```
to
```
features-storage-feature_storage-tests-cfstore-example3
```
2016-09-06 10:23:32 -05:00
8b9a6dc19d
Correctly providing directories to build_apis
...
The shared `prepare_toolchain` and `scan_resources` functions in build_api
expect a list, not a string. This is different from the
toolchain.scan_resources function. Now the functions are being used
correctly within the `find_tests` function in test_api
2016-09-02 11:32:11 -05:00
64928b02df
Merge pull request #2244 from bridadan/test-discovery-refactor
...
Test discovery refactor
2016-07-26 15:45:41 -05:00
f9d38bff37
Merge pull request #2062 from bridadan/shorten-test-binary-name
...
Changing name of test binaries to just test case folder name.
2016-07-26 00:24:08 -05:00
1618e608e7
Test discovery now uses build and toolchain logic
...
Previously, test discovery throught test.py used its own logic to find
tests. This was mostly a subset of the rules used during the build
process. It respected .mbedignore files, but it did not respect TARGET_,
TOOLCHAIN_, and FEATURES_ directories. This change now uses the same logic
used during building to respect these folders. Tests under these folders
that are not being pulled in for the given target will be ignored.
2016-07-25 14:43:52 -05:00
74b7f9e923
mbed Online Build System support:
...
* added/improved global chroot support
* added RESPONSE_FILES flag to support optional response files (on linux the cmd param length is 2 megabytes). Default True
* added unified handling for archive and link response file (similar to includes)
* added COMPILE_C_AS_CPP flag to support compiling of c files as cpp. Default False
* added mbedToolchain.init() for post __init__ hooks
* added caching to mbedToolchain.need_update() to reduce IO hits
* added support to identify compiler warning/error column (supports ARMCC, GCC and IAR). Errors/warnings now report file@line,col
* added global TOOLCHAIN_PATHS which allows overriding/changing of the toolchain paths. Also simplified ARM-related paths
* added target.json to mbed library release (by @0xc0170)* migrated compile_worker() to utils.py for lightweight thread initialization
* improved run_cmd() performance by removing unnecessary check about the command being executed (should be checked once in the relevant toolchain instead)
* removed remnants of Goanna support (should be reimplemented as hooks to compile/link/archive instead)
* fixes for Python 2.7 compatibility (by @0xc0170)
* fixes for Exporters (by @0xc0170)
2016-07-19 11:14:42 +01:00
0ec8199c55
Revert "Revert "Generalize flag handling""
...
This reverts commit 33cec194c6
2016-07-07 15:29:36 -05:00
12a01f61ca
Add summary for test building
...
When users are building tests with `mbed compile --tests` whey will by default
get memory map breakdown report.
This can be suppresed in the future with command line switch. For now it is
visible each time users build test cases.
List is sorted by project name created with `build_project` API.
Changes:
* `build_project` now returns tuple (this breaks build_api.build_project API!)
* Memmap script got a aggregation function to print summary from small 'JSON'
partial reports.
* Report is generated by `test_api.build_tests` function only!
Example:
```
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| name | target | toolchain | total_flash | static_ram | stack | heap | total_ram |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| features-feature_ipv4-tests-mbedmicro-net-nist_internet_time_service | K64F | GCC_ARM | 132136 | 62288 | 32768 | 65536 | 160592 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_echo | K64F | GCC_ARM | 125613 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_hello_world | K64F | GCC_ARM | 125949 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-udp_echo_client | K64F | GCC_ARM | 123613 | 62276 | 32768 | 65536 | 160580 |
| features-storage-tests-cfstore-add_del | K64F | GCC_ARM | 96080 | 13052 | 32768 | 65540 | 111360 |
| features-storage-tests-cfstore-close | K64F | GCC_ARM | 95520 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-create | K64F | GCC_ARM | 99144 | 13036 | 32768 | 65540 | 111344 |
| features-storage-tests-cfstore-example1 | K64F | GCC_ARM | 98592 | 12368 | 32768 | 65536 | 110672 |
| features-storage-tests-cfstore-example2 | K64F | GCC_ARM | 95232 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example3 | K64F | GCC_ARM | 95264 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-example4 | K64F | GCC_ARM | 92632 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example5 | K64F | GCC_ARM | 92344 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-find | K64F | GCC_ARM | 96344 | 13028 | 32768 | 65540 | 111336 |
| features-storage-tests-cfstore-find2 | K64F | GCC_ARM | 93192 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-flash | K64F | GCC_ARM | 97784 | 12532 | 32768 | 65540 | 110840 |
| features-storage-tests-cfstore-flush | K64F | GCC_ARM | 96464 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-flush2 | K64F | GCC_ARM | 95056 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-init | K64F | GCC_ARM | 93120 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-misc | K64F | GCC_ARM | 96808 | 12516 | 32768 | 65540 | 110824 |
| features-storage-tests-cfstore-open | K64F | GCC_ARM | 98632 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-read | K64F | GCC_ARM | 94112 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-write | K64F | GCC_ARM | 94488 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-flash_journal-basicapi | K64F | GCC_ARM | 104712 | 21236 | 32768 | 65540 | 119544 |
| frameworks-utest-tests-unit_tests-basic_test | K64F | GCC_ARM | 71534 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_async_validate | K64F | GCC_ARM | 74598 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_control_async | K64F | GCC_ARM | 74630 | 11476 | 32768 | 65540 | 109784 |
| frameworks-utest-tests-unit_tests-case_control_repeat | K64F | GCC_ARM | 72790 | 11452 | 32768 | 65540 | 109760 |
| frameworks-utest-tests-unit_tests-case_selection | K64F | GCC_ARM | 72302 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_setup_failure | K64F | GCC_ARM | 72630 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_teardown_failure | K64F | GCC_ARM | 72790 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-control_type | K64F | GCC_ARM | 82462 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_async_scheduler | K64F | GCC_ARM | 72182 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_scheduler | K64F | GCC_ARM | 71998 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_assertion_failure_test_setup | K64F | GCC_ARM | 71710 | 11460 | 32768 | 65540 | 109768 |
| frameworks-utest-tests-unit_tests-test_setup_case_selection_failure | K64F | GCC_ARM | 71702 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_setup_failure | K64F | GCC_ARM | 71710 | 11468 | 32768 | 65540 | 109776 |
| tests-integration-basic | K64F | GCC_ARM | 67566 | 10780 | 32768 | 65540 | 109088 |
| tests-integration-threaded_blinky | K64F | GCC_ARM | 68326 | 10780 | 32768 | 65540 | 109088 |
| tests-mbed_drivers-c_strings | K64F | GCC_ARM | 74438 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-callback | K64F | GCC_ARM | 88310 | 11972 | 32768 | 65540 | 110280 |
| tests-mbed_drivers-dev_null | K64F | GCC_ARM | 90213 | 10784 | 32768 | 65540 | 109092 |
| tests-mbed_drivers-echo | K64F | GCC_ARM | 71918 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-generic_tests | K64F | GCC_ARM | 77624 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-rtc | K64F | GCC_ARM | 85854 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-stl_features | K64F | GCC_ARM | 80726 | 11476 | 32768 | 65540 | 109784 |
| tests-mbed_drivers-ticker | K64F | GCC_ARM | 70974 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_2 | K64F | GCC_ARM | 70790 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_3 | K64F | GCC_ARM | 71038 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-timeout | K64F | GCC_ARM | 70886 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-wait_us | K64F | GCC_ARM | 70414 | 11308 | 32768 | 65540 | 109616 |
| tests-mbedmicro-mbed-attributes | K64F | GCC_ARM | 71534 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-mbed-call_before_main | K64F | GCC_ARM | 73112 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-cpp | K64F | GCC_ARM | 73400 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-div | K64F | GCC_ARM | 73176 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-basic | K64F | GCC_ARM | 68390 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-isr | K64F | GCC_ARM | 74480 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-mail | K64F | GCC_ARM | 74992 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-mutex | K64F | GCC_ARM | 74048 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-queue | K64F | GCC_ARM | 74912 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-semaphore | K64F | GCC_ARM | 74296 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-signals | K64F | GCC_ARM | 74328 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-threads | K64F | GCC_ARM | 75214 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-rtos-mbed-timer | K64F | GCC_ARM | 68430 | 10780 | 32768 | 65540 | 109088 |
| tests-storage_abstraction-basicapi | K64F | GCC_ARM | 107808 | 28908 | 32768 | 65540 | 127216 |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
```
Refactored after code review
Refactored parse() function
Polishing
Moved memory usage reporting function call to test.py to group all reporters in one place
Bug-fix: on ARM toolchain we were not fetching statistics from last element of memap result list
2016-07-07 08:35:20 +01:00
458b46c803
Merge pull request #1873 from theotherjimmy/file-reporter
...
Adds a simple text file report format
2016-07-06 16:45:43 -05:00
36a577e1ef
Make build_tests work with target names, not just Target instances
...
With this change, custom targets defined by the application being
tested in its mbed_app.json file can be used with tests. Note that
`build_project` already accepts both target names and instances, so the
call to `build_project` inside `build_tests` will still work.
2016-07-05 15:47:26 +03:00
b7186bb604
Test spec paths are now relative to tool execution directory
...
This commit also normalizes the paths produced by the the test spec to
Unix style paths to increase portability of the file.
2016-06-30 09:32:13 -05:00
27aea960bc
Changing name of test binaries to just test case folder name.
...
This mitigates the Windows paths issue by shortening the test binary name to just the
test case folder name instead of the full unique test name used by the
tools. This doesn't solve the Windows path limit of 260 characters, but it
does reduce the characters used.
2016-06-28 18:48:13 -05:00
855cc91550
Force program_cycle_s to be used as an attribue everywhere
2016-06-27 16:01:52 -05:00
33cec194c6
Revert "Generalize flag handling"
2016-06-24 16:39:01 -05:00
2fc4d64b07
Brought test_api.py and users into the new order of argument parsing
2016-06-24 14:22:48 -05:00
6906b97ba0
Adds a simple text file report format
...
Should resolve #1583
2016-06-21 16:20:19 -05:00
1bcd64301a
Lower case test names, blob matching test names, and sort test names for prints
2016-06-14 02:14:45 +01:00
f622c591e8
Handling exceptions throughout test flow
2016-06-14 02:14:44 +01:00
6796025e78
Fixes for logs and test logic in tools
2016-06-14 02:14:42 +01:00
f6acb1ffb4
Forward ported changes to tools
2016-06-10 15:19:02 +01:00
773dab514e
Add --source option to both build.py and make.py. Also add test.py script for compiling and listing tests
2016-06-09 23:11:23 +01:00
87a978c4e8
Move private_settings.py to root mbed_settings.py. Various updates to reflect the path changes
2016-06-09 22:24:05 +01:00
3d45b2cdbb
Renamed workspace_tools folder to tools
2016-06-09 21:34:53 +01:00
Sam Grove
Jimmy Brisson
Brian Daniels
Conor Keegan
sarahmarshy
Mihail Stoyanov
Przemek Wirkus
Bogdan Marinescu