3ea625c8eb
Merge pull request #2161 from PrzemekWirkus/devel_memap_summary_fields
...
[Tools] Replace in memap CSV report fields
2016-07-18 11:41:18 +01:00
4e1ac4dba2
Merge pull request #2179 from sarahmarshy/progen_build_tests_v2
...
IDE build tests with progen
2016-07-16 17:54:46 +01:00
7251a8ccae
Merge pull request #2176 from 0xc0170/fix_#2171
...
Exporters - progen TARGETS lazy evaluated
2016-07-15 23:55:48 +01:00
b969fa5bb1
IDE build tests with progen
2016-07-15 13:25:35 -05:00
f95265f38d
Exporters - progen TARGETS lazy evaluated
...
To speed up project.py, use @property for TARGETS.
When exporters are used, it would ask progen for all targets support
(number of tools x number of targets), this takes a lot of time, thus
lazily evaluated, and TARGETS are for backaward compatibility, not currently
used by project.py but by other scripts to check if a target is supported.
Profiling:
```
python tools\project.py -m K64F -i uvision -n MBED_10 -b
Before the change:
1 0.000 0.000 0.010 0.010 definitions.py:113(get_tool_definition)
824 0.004 0.000 2.990 0.004 definitions.py:124(is_supported)
1 0.000 0.000 0.000 0.000 definitions.py:145(get_debugger)
1 0.002 0.002 0.044 0.044 definitions.py:15(<module>)
384 0.002 0.000 2.986 0.008 definitions.py:26(_load_record)
1 0.000 0.000 0.000 0.000 definitions.py:32(ProGenMcus)
992 0.055 0.000 1.959 0.002 definitions.py:34(__init__)
1376 0.002 0.000 0.003 0.000 definitions.py:40(get_mcus)
384 0.001 0.000 1.070 0.003 definitions.py:43(get_mcu_record)
1 0.000 0.000 0.000 0.000 definitions.py:50(ProGenTargets)
992 0.000 0.000 0.000 0.000 definitions.py:52(__init__)
1196 0.002 0.000 0.004 0.000 definitions.py:55(get_targets)
204 0.001 0.000 1.922 0.009 definitions.py:58(get_mcu_record)
1 0.000 0.000 0.000 0.000 definitions.py:67(get_debugger)
1 0.000 0.000 0.000 0.000 definitions.py:74(ProGenDef)
992 0.008 0.000 1.973 0.002 definitions.py:83(__init__)
Days : 0
Hours : 0
Minutes : 0
Seconds : 4
Milliseconds : 723
Ticks : 47237302
TotalDays : 5.46728032407407E-05
TotalHours : 0.00131214727777778
TotalMinutes : 0.0787288366666667
TotalSeconds : 4.7237302
TotalMilliseconds : 4723.7302
After the change:
1 0.000 0.000 0.010 0.010 definitions.py:113(get_tool_definition)
2 0.000 0.000 0.025 0.012 definitions.py:124(is_supported)
1 0.000 0.000 0.000 0.000 definitions.py:145(get_debugger)
1 0.002 0.002 0.042 0.042 definitions.py:15(<module>)
3 0.000 0.000 0.035 0.012 definitions.py:26(_load_record)
1 0.000 0.000 0.000 0.000 definitions.py:32(ProGenMcus)
2 0.000 0.000 0.005 0.002 definitions.py:34(__init__)
5 0.000 0.000 0.000 0.000 definitions.py:40(get_mcus)
3 0.000 0.000 0.000 0.000 definitions.py:43(get_mcu_record)
1 0.000 0.000 0.000 0.000 definitions.py:50(ProGenTargets)
2 0.000 0.000 0.000 0.000 definitions.py:52(__init__)
5 0.000 0.000 0.000 0.000 definitions.py:55(get_targets)
3 0.000 0.000 0.035 0.012 definitions.py:58(get_mcu_record)
1 0.000 0.000 0.000 0.000 definitions.py:67(get_debugger)
1 0.000 0.000 0.000 0.000 definitions.py:74(ProGenDef)
2 0.000 0.000 0.005 0.003 definitions.py:83(__init__)
Days : 0
Hours : 0
Minutes : 0
Seconds : 1
Milliseconds : 178
Ticks : 11780618
TotalDays : 1.3634974537037E-05
TotalHours : 0.000327239388888889
TotalMinutes : 0.0196343633333333
TotalSeconds : 1.1780618
TotalMilliseconds : 1178.0618
```
2016-07-15 14:02:57 +01:00
3d20b10b4d
Merge pull request #2172 from theotherjimmy/enfoce-supported-toolchains
...
[tools-build.py]Enforce supported toolchains
2016-07-15 09:02:14 +01:00
3a2b752169
Merge pull request #2174 from theotherjimmy/return-from-build
...
[tools/build_api] Retrun truthy value on success from build_*
2016-07-15 09:01:31 +01:00
eca0c69e6c
Merge pull request #2141 from theotherjimmy/project-argparse
...
[Project.py] Make -i and -m optional
2016-07-14 18:00:20 -05:00
6aa2bbb801
[tools/build_api] Retrun truthy value on success from build_*
...
before:
```
$ python tools/build.py -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.39)s
Build skipped:
* GCC_ARM::NRF51_DK
* ARM::NRF51_DK
```
after:
```
$ python tools/build.py -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.43)s
Build successes:
* GCC_ARM::NRF51_DK
* ARM::NRF51_DK
```
2016-07-14 13:55:51 -05:00
24fcb4700b
[tools-build.py]Enforce supported toolchains
...
by skipping unsupported combinations
2016-07-14 13:29:55 -05:00
d0d023ab58
Merge pull request #2122 from theotherjimmy/colorized-output
...
Colorized the short Warnings/Errors generated by the toolchains
2016-07-13 14:21:15 +03:00
bba13ddad2
Replace in memap CSV report fields
...
Changes:
* This change impacts #2047
* In code summary columns were shortened and names changed a bit,
see [here](https://github.com/mbedmicro/mbed/pull/2047/files?w=1#diff-f9cb941bde5647a5763e18fc220efc51R410 )
* There is a need to allign memap CSV report fields (summary columns) with what
is in code now. This discrapency may create problems when generating reports
in the future. All columns should have the same name.
* memap CSV sumamry fields names changes:
'total_static_ram' -> 'static_ram'
'allocated_heap' -> 'heap'
'allocated_stack' -> 'stack'
See example of discrepancy:
'total_static_ram' vs 'static_ram'
'allocated_heap' vs 'heap'
'allocated_stack' vs 'stack'
What is in code and presented with prettytable:
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| name | target | toolchain | static_ram | stack | heap | total_ram | total_flash |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| tests-mbed_drivers-c_strings | K64F | GCC_ARM | 11468 | 32768 | 65540 | 109776 | 74182 |
| tests-mbed_drivers-callback | K64F | GCC_ARM | 11980 | 32768 | 65540 | 110288 | 88406 |
| tests-mbedmicro-rtos-mbed-threads | K64F | GCC_ARM | 11468 | 32768 | 65540 | 109776 | 75918 |
| tests-mbedmicro-rtos-mbed-timer | K64F | GCC_ARM | 10788 | 32768 | 65540 | 109096 | 68534 |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
vs CSV:
```
total_static_ram,allocated_heap,allocated_stack,total_ram,total_flash
10788,65540,32768,109096,67486
```
vs memstats_map.JSON:
```json
[
{
"module": "Fill",
"size": {
".data": 4,
".bss": 2257,
".text": 173
}
},
{
"summary": {
"total_flash": 67486,
"allocated_stack": 32768,
"total_static_ram": 10788,
"total_ram": 109096,
"allocated_heap": 65540
}
}
]
```
2016-07-13 11:25:04 +01:00
e9d225bb96
Merge pull request #2151 from mbedmicro/verbose_json_errors
...
More verrbose errors for JSON parsing
2016-07-13 10:29:31 +03:00
33dd3fe8ad
More verrbose errors for JSON parsing
...
This commit adds information about the location of problematic JSON
files when reporting a JSON parsing error.
Before this commit:
```
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
After this commit:
```
Error parsing 'core/mbed_lib.json':
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
2016-07-12 18:52:34 +03:00
9704edfca7
Fix add_py_targets and tests
...
Previously, add_py_targets assumed that it is not an error to add an
existing target if that target was previously added using
add_py_targets. This was done to aid testing, but it was weird, error
prone and broke with the latest changes to the caching mechanism. With
this commit:
- it is always an error to add a target that was previously added, which
is much more consistent.
- tests for the configuration mechanism use the newly added
'set_targets_json_location' function to clear the internal caches in
targets.py (and thus all previously added custom targets). As a side
effect, this commit also tests the 'set_targets_json_location'
function itself.
2016-07-12 14:42:21 +03:00
117b21df8f
It's now possible to specify the location of targets.json
...
With this change, it becomes possible to use targets.py with any
targets.json, not just the default one in ../hal/targets.json.
targets.py will still be initialized with the default targets.json, but
the code can then call "set_targets_json_location" to specify the new
location of targets.json. set_targets_json_location modifies all the
data "in place"; that is, it doesn't create a new TARGET_MAP,
TARGET_NAMES or alike, but rather modified the existing ones. This is
important, because code using this construct:
from tools.targets import TARGET_MAP
can work unmodified with this change.
2016-07-12 13:03:47 +03:00
50dbce9e74
Fixes to function caching in targets.py
...
Now funnctions are looked up in the cache using a (function name,
arguments) key, which makes it possible to cache different invocations
of the functions (with different arguments).
Also applied the @cached attribute to get_target.
2016-07-12 13:03:40 +03:00
cebe292c5e
[Project.py] Make -i and -m optional
...
Resolves #2135
2016-07-11 10:38:50 -05:00
e2c88373ee
Replace color definitions with colorama constants
2016-07-11 09:41:23 -05:00
bac254c7a3
Add documentation of the notify parameter to mbedToolchain
2016-07-11 09:41:23 -05:00
12492c17b8
Call colorized notify when --color for build.py, make.py, test.py
2016-07-11 09:41:23 -05:00
6b8bde9471
Update event dict with toolchain, use toolchain for wrapping
2016-07-11 09:41:23 -05:00
21f025e601
Add a setting for colorization map so that it may be overwritten by the user
2016-07-11 09:41:23 -05:00
3aacdb4421
Implement colorization in it's own file
2016-07-11 09:41:23 -05:00
f4254c4fee
Add colorization option to default options parser
...
should be picked up by most cli's
2016-07-11 09:41:14 -05:00
61c63b3789
Argument error when memap's input file does not exist
2016-07-07 15:29:36 -05:00
122fa93a70
Allow make.py to accpet just --source
2016-07-07 15:29:36 -05:00
6d0716c69b
Ignore extra MCUs and toolchains in get_config.py
2016-07-07 15:29:36 -05:00
254dccdad3
Remove checking for existance of build directory
2016-07-07 15:29:36 -05:00
7a4bee8c0e
nargs="*" -> action="append"
2016-07-07 15:29:36 -05:00
bca12206f1
Allow --source to be specified more than once
2016-07-07 15:29:36 -05:00
4c5bfaa3d8
Allow --source with -n and -p, or alone; name projects conditionally
2016-07-07 15:29:36 -05:00
5f18943641
Revert parsing of mutiple ides on command line
2016-07-07 15:29:36 -05:00
9040d27967
Rework the force types to match case
...
for example, argparse_force_uppercase_type will now correctly parse uARM to uARM
2016-07-07 15:29:36 -05:00
b4b514ea2f
Correct test_name_known, now use argparse_many for lists
...
also updated all consumers of test_name_known
2016-07-07 15:29:36 -05:00
5052e97b17
Force conversion of all -i, -m, -t to the correct case
2016-07-07 15:29:36 -05:00
6fda53b8b8
Add documentation for the argparser utilities
2016-07-07 15:29:36 -05:00
59ae690f9b
Correct style in tests.py
2016-07-07 15:29:36 -05:00
d6bee561ab
Squash weird import bug
2016-07-07 15:29:36 -05:00
0ec8199c55
Revert "Revert "Generalize flag handling""
...
This reverts commit 33cec194c6
2016-07-07 15:29:36 -05:00
12a01f61ca
Add summary for test building
...
When users are building tests with `mbed compile --tests` whey will by default
get memory map breakdown report.
This can be suppresed in the future with command line switch. For now it is
visible each time users build test cases.
List is sorted by project name created with `build_project` API.
Changes:
* `build_project` now returns tuple (this breaks build_api.build_project API!)
* Memmap script got a aggregation function to print summary from small 'JSON'
partial reports.
* Report is generated by `test_api.build_tests` function only!
Example:
```
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| name | target | toolchain | total_flash | static_ram | stack | heap | total_ram |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| features-feature_ipv4-tests-mbedmicro-net-nist_internet_time_service | K64F | GCC_ARM | 132136 | 62288 | 32768 | 65536 | 160592 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_echo | K64F | GCC_ARM | 125613 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_hello_world | K64F | GCC_ARM | 125949 | 62448 | 32768 | 65540 | 160756 |
| features-feature_ipv4-tests-mbedmicro-net-udp_echo_client | K64F | GCC_ARM | 123613 | 62276 | 32768 | 65536 | 160580 |
| features-storage-tests-cfstore-add_del | K64F | GCC_ARM | 96080 | 13052 | 32768 | 65540 | 111360 |
| features-storage-tests-cfstore-close | K64F | GCC_ARM | 95520 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-create | K64F | GCC_ARM | 99144 | 13036 | 32768 | 65540 | 111344 |
| features-storage-tests-cfstore-example1 | K64F | GCC_ARM | 98592 | 12368 | 32768 | 65536 | 110672 |
| features-storage-tests-cfstore-example2 | K64F | GCC_ARM | 95232 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example3 | K64F | GCC_ARM | 95264 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-example4 | K64F | GCC_ARM | 92632 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-example5 | K64F | GCC_ARM | 92344 | 11856 | 32768 | 65536 | 110160 |
| features-storage-tests-cfstore-find | K64F | GCC_ARM | 96344 | 13028 | 32768 | 65540 | 111336 |
| features-storage-tests-cfstore-find2 | K64F | GCC_ARM | 93192 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-flash | K64F | GCC_ARM | 97784 | 12532 | 32768 | 65540 | 110840 |
| features-storage-tests-cfstore-flush | K64F | GCC_ARM | 96464 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-flush2 | K64F | GCC_ARM | 95056 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-cfstore-init | K64F | GCC_ARM | 93120 | 12012 | 32768 | 65540 | 110320 |
| features-storage-tests-cfstore-misc | K64F | GCC_ARM | 96808 | 12516 | 32768 | 65540 | 110824 |
| features-storage-tests-cfstore-open | K64F | GCC_ARM | 98632 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-read | K64F | GCC_ARM | 94112 | 12540 | 32768 | 65540 | 110848 |
| features-storage-tests-cfstore-write | K64F | GCC_ARM | 94488 | 12004 | 32768 | 65540 | 110312 |
| features-storage-tests-flash_journal-basicapi | K64F | GCC_ARM | 104712 | 21236 | 32768 | 65540 | 119544 |
| frameworks-utest-tests-unit_tests-basic_test | K64F | GCC_ARM | 71534 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_async_validate | K64F | GCC_ARM | 74598 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_control_async | K64F | GCC_ARM | 74630 | 11476 | 32768 | 65540 | 109784 |
| frameworks-utest-tests-unit_tests-case_control_repeat | K64F | GCC_ARM | 72790 | 11452 | 32768 | 65540 | 109760 |
| frameworks-utest-tests-unit_tests-case_selection | K64F | GCC_ARM | 72302 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_setup_failure | K64F | GCC_ARM | 72630 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-case_teardown_failure | K64F | GCC_ARM | 72790 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-control_type | K64F | GCC_ARM | 82462 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_async_scheduler | K64F | GCC_ARM | 72182 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-minimal_scheduler | K64F | GCC_ARM | 71998 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_assertion_failure_test_setup | K64F | GCC_ARM | 71710 | 11460 | 32768 | 65540 | 109768 |
| frameworks-utest-tests-unit_tests-test_setup_case_selection_failure | K64F | GCC_ARM | 71702 | 11468 | 32768 | 65540 | 109776 |
| frameworks-utest-tests-unit_tests-test_setup_failure | K64F | GCC_ARM | 71710 | 11468 | 32768 | 65540 | 109776 |
| tests-integration-basic | K64F | GCC_ARM | 67566 | 10780 | 32768 | 65540 | 109088 |
| tests-integration-threaded_blinky | K64F | GCC_ARM | 68326 | 10780 | 32768 | 65540 | 109088 |
| tests-mbed_drivers-c_strings | K64F | GCC_ARM | 74438 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-callback | K64F | GCC_ARM | 88310 | 11972 | 32768 | 65540 | 110280 |
| tests-mbed_drivers-dev_null | K64F | GCC_ARM | 90213 | 10784 | 32768 | 65540 | 109092 |
| tests-mbed_drivers-echo | K64F | GCC_ARM | 71918 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-generic_tests | K64F | GCC_ARM | 77624 | 11468 | 32768 | 65540 | 109776 |
| tests-mbed_drivers-rtc | K64F | GCC_ARM | 85854 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-stl_features | K64F | GCC_ARM | 80726 | 11476 | 32768 | 65540 | 109784 |
| tests-mbed_drivers-ticker | K64F | GCC_ARM | 70974 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_2 | K64F | GCC_ARM | 70790 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-ticker_3 | K64F | GCC_ARM | 71038 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-timeout | K64F | GCC_ARM | 70886 | 11308 | 32768 | 65540 | 109616 |
| tests-mbed_drivers-wait_us | K64F | GCC_ARM | 70414 | 11308 | 32768 | 65540 | 109616 |
| tests-mbedmicro-mbed-attributes | K64F | GCC_ARM | 71534 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-mbed-call_before_main | K64F | GCC_ARM | 73112 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-cpp | K64F | GCC_ARM | 73400 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-mbed-div | K64F | GCC_ARM | 73176 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-basic | K64F | GCC_ARM | 68390 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-isr | K64F | GCC_ARM | 74480 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-mail | K64F | GCC_ARM | 74992 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-mutex | K64F | GCC_ARM | 74048 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-queue | K64F | GCC_ARM | 74912 | 11300 | 32768 | 65540 | 109608 |
| tests-mbedmicro-rtos-mbed-semaphore | K64F | GCC_ARM | 74296 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-signals | K64F | GCC_ARM | 74328 | 10780 | 32768 | 65540 | 109088 |
| tests-mbedmicro-rtos-mbed-threads | K64F | GCC_ARM | 75214 | 11460 | 32768 | 65540 | 109768 |
| tests-mbedmicro-rtos-mbed-timer | K64F | GCC_ARM | 68430 | 10780 | 32768 | 65540 | 109088 |
| tests-storage_abstraction-basicapi | K64F | GCC_ARM | 107808 | 28908 | 32768 | 65540 | 127216 |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
```
Refactored after code review
Refactored parse() function
Polishing
Moved memory usage reporting function call to test.py to group all reporters in one place
Bug-fix: on ARM toolchain we were not fetching statistics from last element of memap result list
2016-07-07 08:35:20 +01:00
458b46c803
Merge pull request #1873 from theotherjimmy/file-reporter
...
Adds a simple text file report format
2016-07-06 16:45:43 -05:00
36ff1f73ce
Merge pull request #2106 from theotherjimmy/fix-makefile
...
[Exporter-gcc_arm] do builds from project directory when --source is specified
2016-07-06 15:33:28 -05:00
3697dedb78
Reverse order of relativizing paths and unixifing paths
...
'cause windows silly path separator is the same as the escape character
2016-07-06 13:54:33 -05:00
3cf8bfbfb7
Allow spaces in paths passed to mkdir and rm on bash
2016-07-06 12:51:14 -05:00
fbccf8dfc9
Force shell use when in bash/sh mode
2016-07-06 12:49:38 -05:00
c2d66865be
Correct batch detection algorimthm
...
There are two shell-like things that we care about
bash: does environment varible expansion with $
batch: does environment varible expansion by surrounding something with %
so to detect batch, try to expand a variable on the shell with $.
If you get a $ back, you are in batch!
2016-07-06 11:32:01 -05:00
e237d157dd
Merge pull request #2113 from ohagendorf/gcc_export
...
[bugfix gcc_arm] build+make <-> export project+make: different results
2016-07-06 13:49:10 +01:00
bd614a2a11
[bugfix gcc_arm] build+make <-> export project+make: different results
...
when in target.json "default_build": "small" is configured
- build.py+make.py
- uses linker option --specs=nano.specs
- macro MBED_RTOS_SINGLE_THREAD is defined
- exporting with project.py + make Makefile
- doesn't use the linker option --specs=nano.specs
- doesn't contain macro MBED_RTOS_SINGLE_THREAD
2016-07-06 13:35:09 +02:00
7f2f0bc5c6
repairing a minor typo in exporters.py
2016-07-06 13:34:59 +02:00
Martin Kojtal
Mihail Stoyanov
Sarah Marsh
Sam Grove
Jimmy Brisson
Bogdan Marinescu
Przemek Wirkus
Bogdan Marinescu
Jimmy Brisson
Olaf Hagendorf