Commit Graph

294 Commits (95c17a4ff4425e18a7302d3749b8eedf1539f36c)

Author SHA1 Message Date
Sam Grove f817ec136c Merge pull request #2142 from theotherjimmy/anti-remove-repo
[tools-build,make,test] Disallow parent of repo as --build args
2016-07-21 22:11:28 -05:00
Sam Grove b10276cbef Merge pull request #2173 from theotherjimmy/abstract-base-class
[toolchains]Abstractify the mbedToolchain base class.
2016-07-21 22:10:49 -05:00
Sam Grove 702707dee7 Merge pull request #2166 from theotherjimmy/prevent-traceback-during-export
Preventing a traceback when a library is not built during export
2016-07-21 22:09:52 -05:00
Martin Kojtal dd010097ba Merge pull request #2159 from jeromecoutant/PR_dev_NUCLEO_F207ZG
New STM32 board : NUCLEO_F207ZG
2016-07-21 11:13:33 +01:00
Bogdan Marinescu 51540d08d5 Merge pull request #2177 from mbedmicro/build_on_config_change
Rebuild sources if configuration is changed
2016-07-20 14:52:08 +03:00
Jimmy Brisson 9759382166 prevent test form removeing the root dir 2016-07-19 09:33:03 -05:00
Jimmy Brisson 52855b53cf [tools-build,make,project] Disallow parent of repo as --build args
Resolves #2081
2016-07-19 09:32:44 -05:00
Jimmy Brisson 684644b1b1 [project.py] Preventing a traceback when a library is not built during export 2016-07-19 08:58:24 -05:00
Bogdan Marinescu fd757d3b84 Merge pull request #2180 from mbedmicro/online-build-system
[Tools] mbed Online Build System support and improvements
2016-07-19 16:30:48 +03:00
Mihail Stoyanov ad87f9da34 Document code and mark which methods are used by the online build system 2016-07-19 11:16:05 +01:00
Mihail Stoyanov 74b7f9e923 mbed Online Build System support:
* added/improved global chroot support
* added RESPONSE_FILES flag to support optional response files (on linux the cmd param length is 2 megabytes). Default True
* added unified handling for archive and link response file (similar to includes)
* added COMPILE_C_AS_CPP flag to support compiling of c files as cpp. Default False
* added mbedToolchain.init() for post __init__ hooks
* added caching to mbedToolchain.need_update() to reduce IO hits
* added support to identify compiler warning/error column (supports ARMCC, GCC and IAR). Errors/warnings now report file@line,col
* added global TOOLCHAIN_PATHS which allows overriding/changing of the toolchain paths. Also simplified ARM-related paths
* added target.json to mbed library release (by @0xc0170)* migrated compile_worker() to utils.py for lightweight thread initialization
* improved run_cmd() performance by removing unnecessary check about the command being executed (should be checked once in the relevant toolchain instead)
* removed remnants of Goanna support (should be reimplemented as hooks to compile/link/archive instead)
* fixes for Python 2.7 compatibility (by @0xc0170)
* fixes for Exporters (by @0xc0170)
2016-07-19 11:14:42 +01:00
Martin Kojtal 5cd73170ba Merge pull request #2185 from yogpan01/master
Changing -O2 parameter to -Os
2016-07-19 10:42:34 +01:00
Bogdan Marinescu 8ab89c1131 Merge pull request #2175 from mbedmicro/release
Release v122
2016-07-19 11:34:15 +03:00
Bogdan Marinescu 474915c22c Merge pull request #2186 from 0xc0170/fix_issue2183
Fix issue2183
2016-07-19 11:31:35 +03:00
0xc0170 edf015ad92 Merge branch 'master' into release
Conflicts:
	tools/export/uvision4.py
	tools/export/uvision5.py
2016-07-19 08:44:34 +01:00
0xc0170 028f722868 uvision5 - remove init, not used 2016-07-19 08:30:35 +01:00
0xc0170 f80b62ed00 exporter - fix TARGETS property via descriptors
Fixes #2183 issue.
2016-07-19 08:30:21 +01:00
0xc0170 bf2ad16c69 project - zip and src check if empty fix
Regression from #2179. Fixes #2184.
2016-07-18 15:47:01 +01:00
Yogesh Pande e6043daa04 Changing -O2 parameter to -Os
This commit includes
- Changing build option parameter from -O2 to -Os to reduce flash size of the
  final binary pacakge.
  The original parameter of -O2 is kept to circumvent the error
  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46762 .
 But this error has been long fixed and released in latest 4.9.x GCC compiler.

 This change is necessary to reduce the generated binary size of mbed Client component to
avoid problem of getting images over 512 kB which is causing application to crash because of
Configuration Store problem https://github.com/ARMmbed/configuration-store/issues/21
2016-07-18 13:59:39 +03:00
Martin Kojtal 3ea625c8eb Merge pull request #2161 from PrzemekWirkus/devel_memap_summary_fields
[Tools] Replace in memap CSV report fields
2016-07-18 11:41:18 +01:00
Mihail Stoyanov 4e1ac4dba2 Merge pull request #2179 from sarahmarshy/progen_build_tests_v2
IDE build tests with progen
2016-07-16 17:54:46 +01:00
Martin Kojtal 7251a8ccae Merge pull request #2176 from 0xc0170/fix_#2171
Exporters - progen TARGETS lazy evaluated
2016-07-15 23:55:48 +01:00
Sarah Marsh b969fa5bb1 IDE build tests with progen 2016-07-15 13:25:35 -05:00
Jimmy Brisson bccbe48f3f Updated all docstrings to be compatible with PEP 257
It's nicer to be compatible with the standards.

I have added one more field to the description of each function:
Side effects. This feild contains the expected side effects of
running a particular method. If the side effects of a method are
None, it is expected that the function does not change anything in
any envoronment and that running it multiple times with the same
arguments will produce the same result every time. That is when
Side effects is non, the method is expected to be pure.
2016-07-15 10:51:47 -05:00
Jimmy Brisson 4fc8c56410 Replace pass with rais NotImplemented in each abstact method
The methods themselfes do nothing within the abstract base class
mbedToolchain. This change enforces that toolchains do not call
these methods that contain no implementation.
2016-07-15 10:17:35 -05:00
Jimmy Brisson f6b8cf05b0 Add a test that exercises the abstract base class checking
The test simply tries to instantiate each class exposed as part of the
TOOLCHAIN_CLASSES map. This is a test that each toolchain class
implements the required API, as the mbedToolchain will not allow the
instantiation of a subclass without a part of that API.
2016-07-15 10:10:40 -05:00
Jimmy Brisson 17833e8f58 Abstractify the mbedToolchain base class
The mbedToolchain class calls many members of it's subclasses, expecting
them to implement a particular API. This change adds a requirement to
each subclass that requires them to implement this expected API.

The API consists of these methods:
 - parse_dependencies
 - parse_ouptut
 - get_config_option
 - compile_c
 - compile_cpp
 - link
 - archive
 - binary
2016-07-15 10:07:56 -05:00
Bogdan Marinescu a558c49240 Rebuild sources if configuration is changed
This commit adds a check for configuration data changes. If a change in
configuration data is detected, all the sources in the tree are rebuilt.

This is a fix for #2073. #2162 was originally proposed as a fix, but it
was agreed that `--preinclude` is a more convenient way to include
configuration data, since it can be used to change the behaviour of
source files that don't include "mbed_config.h" directly, which is a big
advantage when importing 3rd party source trees. Compared to #2162, this
commit has the disadvantage of rebuilding all the source files if a
configuration change is detected, but it was agreed that the advantage
of using `--preinclude` outweighs the disadvantage of the increased
compilation time.
2016-07-15 16:37:46 +03:00
0xc0170 f95265f38d Exporters - progen TARGETS lazy evaluated
To speed up project.py, use @property for TARGETS.
When exporters are used, it would ask progen for all targets support
(number of tools x number of targets), this takes a lot of  time, thus
lazily evaluated, and TARGETS are for backaward compatibility, not currently
used by project.py but by other scripts to check if a target is supported.

Profiling:
```
python tools\project.py -m K64F -i uvision -n MBED_10 -b

Before the change:
       1    0.000    0.000    0.010    0.010 definitions.py:113(get_tool_definition)
      824    0.004    0.000    2.990    0.004 definitions.py:124(is_supported)
        1    0.000    0.000    0.000    0.000 definitions.py:145(get_debugger)
        1    0.002    0.002    0.044    0.044 definitions.py:15(<module>)
      384    0.002    0.000    2.986    0.008 definitions.py:26(_load_record)
        1    0.000    0.000    0.000    0.000 definitions.py:32(ProGenMcus)
      992    0.055    0.000    1.959    0.002 definitions.py:34(__init__)
     1376    0.002    0.000    0.003    0.000 definitions.py:40(get_mcus)
      384    0.001    0.000    1.070    0.003 definitions.py:43(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:50(ProGenTargets)
      992    0.000    0.000    0.000    0.000 definitions.py:52(__init__)
     1196    0.002    0.000    0.004    0.000 definitions.py:55(get_targets)
      204    0.001    0.000    1.922    0.009 definitions.py:58(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:67(get_debugger)
        1    0.000    0.000    0.000    0.000 definitions.py:74(ProGenDef)
      992    0.008    0.000    1.973    0.002 definitions.py:83(__init__)

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 4
Milliseconds      : 723
Ticks             : 47237302
TotalDays         : 5.46728032407407E-05
TotalHours        : 0.00131214727777778
TotalMinutes      : 0.0787288366666667
TotalSeconds      : 4.7237302
TotalMilliseconds : 4723.7302

After the change:

        1    0.000    0.000    0.010    0.010 definitions.py:113(get_tool_definition)
        2    0.000    0.000    0.025    0.012 definitions.py:124(is_supported)
        1    0.000    0.000    0.000    0.000 definitions.py:145(get_debugger)
        1    0.002    0.002    0.042    0.042 definitions.py:15(<module>)
        3    0.000    0.000    0.035    0.012 definitions.py:26(_load_record)
        1    0.000    0.000    0.000    0.000 definitions.py:32(ProGenMcus)
        2    0.000    0.000    0.005    0.002 definitions.py:34(__init__)
        5    0.000    0.000    0.000    0.000 definitions.py:40(get_mcus)
        3    0.000    0.000    0.000    0.000 definitions.py:43(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:50(ProGenTargets)
        2    0.000    0.000    0.000    0.000 definitions.py:52(__init__)
        5    0.000    0.000    0.000    0.000 definitions.py:55(get_targets)
        3    0.000    0.000    0.035    0.012 definitions.py:58(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:67(get_debugger)
        1    0.000    0.000    0.000    0.000 definitions.py:74(ProGenDef)
        2    0.000    0.000    0.005    0.003 definitions.py:83(__init__)

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 178
Ticks             : 11780618
TotalDays         : 1.3634974537037E-05
TotalHours        : 0.000327239388888889
TotalMinutes      : 0.0196343633333333
TotalSeconds      : 1.1780618
TotalMilliseconds : 1178.0618
```
2016-07-15 14:02:57 +01:00
Martin Kojtal 3d20b10b4d Merge pull request #2172 from theotherjimmy/enfoce-supported-toolchains
[tools-build.py]Enforce supported toolchains
2016-07-15 09:02:14 +01:00
Martin Kojtal 3a2b752169 Merge pull request #2174 from theotherjimmy/return-from-build
[tools/build_api] Retrun truthy value on success from build_*
2016-07-15 09:01:31 +01:00
Sam Grove eca0c69e6c Merge pull request #2141 from theotherjimmy/project-argparse
[Project.py] Make -i and -m optional
2016-07-14 18:00:20 -05:00
Jimmy Brisson 6aa2bbb801 [tools/build_api] Retrun truthy value on success from build_*
before:
```
$  python tools/build.py  -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.39)s

Build skipped:
  * GCC_ARM::NRF51_DK
  * ARM::NRF51_DK

```
after:
```
$  python tools/build.py  -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.43)s

Build successes:
  * GCC_ARM::NRF51_DK
  * ARM::NRF51_DK

```
2016-07-14 13:55:51 -05:00
Jimmy Brisson 24fcb4700b [tools-build.py]Enforce supported toolchains
by skipping unsupported combinations
2016-07-14 13:29:55 -05:00
0xc0170 125754bf31 Exporter uvision - fix build_dir, only windows slashes allowed
uvision errors otherwise, not possible to create a path.
2016-07-14 14:21:05 +01:00
0xc0170 9c3801b7b5 Config - fix - use basetring to check if a target is string 2016-07-13 14:59:48 +01:00
Bogdan Marinescu d0d023ab58 Merge pull request #2122 from theotherjimmy/colorized-output
Colorized the short Warnings/Errors generated by the toolchains
2016-07-13 14:21:15 +03:00
Przemek Wirkus bba13ddad2 Replace in memap CSV report fields
Changes:
* This change impacts #2047
* In code summary columns were shortened and names changed a bit,
  see [here](https://github.com/mbedmicro/mbed/pull/2047/files?w=1#diff-f9cb941bde5647a5763e18fc220efc51R410)
* There is a need to allign memap CSV report fields (summary columns) with what
  is in code now. This discrapency may create problems when generating reports
  in the future. All columns should have the same name.
* memap CSV sumamry fields names changes:
  'total_static_ram' -> 'static_ram'
  'allocated_heap' -> 'heap'
  'allocated_stack' -> 'stack'

See example of discrepancy:

'total_static_ram' vs 'static_ram'
'allocated_heap' vs 'heap'
'allocated_stack' vs 'stack'

What is in code and presented with prettytable:

+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| name                                   | target | toolchain | static_ram | stack |  heap | total_ram | total_flash |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| tests-mbed_drivers-c_strings           | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       74182 |
| tests-mbed_drivers-callback            | K64F   | GCC_ARM   |      11980 | 32768 | 65540 |    110288 |       88406 |
| tests-mbedmicro-rtos-mbed-threads      | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       75918 |
| tests-mbedmicro-rtos-mbed-timer        | K64F   | GCC_ARM   |      10788 | 32768 | 65540 |    109096 |       68534 |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+

vs CSV:

```
total_static_ram,allocated_heap,allocated_stack,total_ram,total_flash
10788,65540,32768,109096,67486
```

vs memstats_map.JSON:

```json
[
    {
        "module": "Fill",
        "size": {
            ".data": 4,
            ".bss": 2257,
            ".text": 173
        }
    },

    {
        "summary": {
            "total_flash": 67486,
            "allocated_stack": 32768,
            "total_static_ram": 10788,
            "total_ram": 109096,
            "allocated_heap": 65540
        }
    }
]
```
2016-07-13 11:25:04 +01:00
Bogdan Marinescu e9d225bb96 Merge pull request #2151 from mbedmicro/verbose_json_errors
More verrbose errors for JSON parsing
2016-07-13 10:29:31 +03:00
Bogdan Marinescu 33dd3fe8ad More verrbose errors for JSON parsing
This commit adds information about the location of problematic JSON
files when reporting a JSON parsing error.

Before this commit:

```
[ERROR] Expecting property name: line 7 column 9 (char 188)
```

After this commit:

```
Error parsing 'core/mbed_lib.json':
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
2016-07-12 18:52:34 +03:00
jeromecoutant 8d3cdfea59 tools update to add NUCLEO_F207ZG 2016-07-12 16:38:11 +02:00
Bogdan Marinescu 9704edfca7 Fix add_py_targets and tests
Previously, add_py_targets assumed that it is not an error to add an
existing target if that target was previously added using
add_py_targets. This was done to aid testing, but it was weird, error
prone and broke with the latest changes to the caching mechanism. With
this commit:

- it is always an error to add a target that was previously added, which
  is much more consistent.
- tests for the configuration mechanism use the newly added
  'set_targets_json_location' function to clear the internal caches in
  targets.py (and thus all previously added custom targets). As a side
  effect, this commit also tests the 'set_targets_json_location'
  function itself.
2016-07-12 14:42:21 +03:00
Bogdan Marinescu 117b21df8f It's now possible to specify the location of targets.json
With this change, it becomes possible to use targets.py with any
targets.json, not just the default one in ../hal/targets.json.
targets.py will still be initialized with the default targets.json, but
the code can then call "set_targets_json_location" to specify the new
location of targets.json. set_targets_json_location modifies all the
data "in place"; that is, it doesn't create a new TARGET_MAP,
TARGET_NAMES or alike, but rather modified the existing ones. This is
important, because code using this construct:

from tools.targets import TARGET_MAP

can work unmodified with this change.
2016-07-12 13:03:47 +03:00
Jimmy Brisson 50dbce9e74 Fixes to function caching in targets.py
Now funnctions are looked up in the cache using a (function name,
arguments) key, which makes it possible to cache different invocations
of the functions (with different arguments).
Also applied the @cached attribute to get_target.
2016-07-12 13:03:40 +03:00
Jimmy Brisson cebe292c5e [Project.py] Make -i and -m optional
Resolves #2135
2016-07-11 10:38:50 -05:00
Jimmy Brisson e2c88373ee Replace color definitions with colorama constants 2016-07-11 09:41:23 -05:00
Jimmy Brisson bac254c7a3 Add documentation of the notify parameter to mbedToolchain 2016-07-11 09:41:23 -05:00
Jimmy Brisson 12492c17b8 Call colorized notify when --color for build.py, make.py, test.py 2016-07-11 09:41:23 -05:00
Jimmy Brisson 6b8bde9471 Update event dict with toolchain, use toolchain for wrapping 2016-07-11 09:41:23 -05:00
Jimmy Brisson 21f025e601 Add a setting for colorization map so that it may be overwritten by the user 2016-07-11 09:41:23 -05:00