Commit Graph

304 Commits (db99e726e006ca3260769d6cd444f51eefec1708)

Author SHA1 Message Date
Sam Grove 03e99031e7 Merge pull request #2225 from theotherjimmy/fix-project-args-traceback
Fix #2223
2016-07-22 18:03:25 -05:00
Brian Daniels 51c575dcc6 Fixed traceback in build_api (typo of resources variable in build_library) 2016-07-22 17:49:54 -05:00
Sam Grove aec3695cc5 Merge pull request #2188 from screamerbg/build-common
[Tools] Unify common routines in build_api.py
2016-07-22 17:04:25 -05:00
Mihail Stoyanov ac86584c42 Unify common routines in build_api.py
Introduce prepare_toolchain() which handles:
 * deduplication of src_paths
 * config initialization
 * toolchain initialization
 * toolchain flags
 * returns toolchain
Introduce scan_resources() which handles:
 * scanning of sources
 * scanning of include dirs
 * inclusion of addition include dirs
 * returns resources
2016-07-22 22:21:39 +01:00
Sam Grove 1880daef0d Merge pull request #2215 from geky/tools-cumulative-overrides
[tools] Modified config to aggregate cumulative overrides against targets
2016-07-22 16:16:41 -05:00
Jimmy Brisson a603bb0e01 Fix "project.py --help - typerror type argument unexpected" (#2223) 2016-07-22 12:59:57 -05:00
Christopher Haster a5ea143c77 Updated config tests to match change in error message 2016-07-22 00:35:13 -05:00
Christopher Haster 9bd7a2fe72 Added proper support for the extra_labels target override
Due to dependencies between the config/target systems, the config
has been modified to preparse the mbed_app.json for overrides.
This means that attributes with dependencies in the targets are
only valid at the application level. A hard error has been
added for these attributes if they are supplied at the library
level.
2016-07-22 00:35:13 -05:00
Christopher Haster f2051b217a Modified config to explicitly update cumulative attributes in targets
Modified '_process_config_and_overrides' to update cumulative
attributes based on cumulative overrides found during walking config
files.

This should remove most of the disparity between the config target
cumulative overrides and cumulative attributes set in targets.json.
These cumulative overrides are updated after the global target
instantiation though, which may cause internal build tool state
dependent on cumulative attributes to not correctly reflect all
cumulative overrides.
2016-07-22 00:35:13 -05:00
Christopher Haster 0fcb20e8c1 Generalized handling of cumulative attributes in target overrides
In tools/config.py, Config now aggregates all cumulative attributes in
target overrides.

{
  "target_overrides": {
    "K64F": {
      "target.features_add": ["UVISOR"],
      "target.extra_labels_add": ["UVISOR_SUPPORTED"],
      "target.macros_add": ["UVISOR_MACRO"],
      "target.device_has_add": ["UVISOR_HAS"]
    }
  }
}

Note: no action is performed on overrides yet
2016-07-22 00:35:13 -05:00
Sam Grove f817ec136c Merge pull request #2142 from theotherjimmy/anti-remove-repo
[tools-build,make,test] Disallow parent of repo as --build args
2016-07-21 22:11:28 -05:00
Sam Grove b10276cbef Merge pull request #2173 from theotherjimmy/abstract-base-class
[toolchains]Abstractify the mbedToolchain base class.
2016-07-21 22:10:49 -05:00
Sam Grove 702707dee7 Merge pull request #2166 from theotherjimmy/prevent-traceback-during-export
Preventing a traceback when a library is not built during export
2016-07-21 22:09:52 -05:00
Martin Kojtal dd010097ba Merge pull request #2159 from jeromecoutant/PR_dev_NUCLEO_F207ZG
New STM32 board : NUCLEO_F207ZG
2016-07-21 11:13:33 +01:00
Bogdan Marinescu 51540d08d5 Merge pull request #2177 from mbedmicro/build_on_config_change
Rebuild sources if configuration is changed
2016-07-20 14:52:08 +03:00
Jimmy Brisson 9759382166 prevent test form removeing the root dir 2016-07-19 09:33:03 -05:00
Jimmy Brisson 52855b53cf [tools-build,make,project] Disallow parent of repo as --build args
Resolves #2081
2016-07-19 09:32:44 -05:00
Jimmy Brisson 684644b1b1 [project.py] Preventing a traceback when a library is not built during export 2016-07-19 08:58:24 -05:00
Bogdan Marinescu fd757d3b84 Merge pull request #2180 from mbedmicro/online-build-system
[Tools] mbed Online Build System support and improvements
2016-07-19 16:30:48 +03:00
Mihail Stoyanov ad87f9da34 Document code and mark which methods are used by the online build system 2016-07-19 11:16:05 +01:00
Mihail Stoyanov 74b7f9e923 mbed Online Build System support:
* added/improved global chroot support
* added RESPONSE_FILES flag to support optional response files (on linux the cmd param length is 2 megabytes). Default True
* added unified handling for archive and link response file (similar to includes)
* added COMPILE_C_AS_CPP flag to support compiling of c files as cpp. Default False
* added mbedToolchain.init() for post __init__ hooks
* added caching to mbedToolchain.need_update() to reduce IO hits
* added support to identify compiler warning/error column (supports ARMCC, GCC and IAR). Errors/warnings now report file@line,col
* added global TOOLCHAIN_PATHS which allows overriding/changing of the toolchain paths. Also simplified ARM-related paths
* added target.json to mbed library release (by @0xc0170)* migrated compile_worker() to utils.py for lightweight thread initialization
* improved run_cmd() performance by removing unnecessary check about the command being executed (should be checked once in the relevant toolchain instead)
* removed remnants of Goanna support (should be reimplemented as hooks to compile/link/archive instead)
* fixes for Python 2.7 compatibility (by @0xc0170)
* fixes for Exporters (by @0xc0170)
2016-07-19 11:14:42 +01:00
Martin Kojtal 5cd73170ba Merge pull request #2185 from yogpan01/master
Changing -O2 parameter to -Os
2016-07-19 10:42:34 +01:00
Bogdan Marinescu 8ab89c1131 Merge pull request #2175 from mbedmicro/release
Release v122
2016-07-19 11:34:15 +03:00
Bogdan Marinescu 474915c22c Merge pull request #2186 from 0xc0170/fix_issue2183
Fix issue2183
2016-07-19 11:31:35 +03:00
0xc0170 edf015ad92 Merge branch 'master' into release
Conflicts:
	tools/export/uvision4.py
	tools/export/uvision5.py
2016-07-19 08:44:34 +01:00
0xc0170 028f722868 uvision5 - remove init, not used 2016-07-19 08:30:35 +01:00
0xc0170 f80b62ed00 exporter - fix TARGETS property via descriptors
Fixes #2183 issue.
2016-07-19 08:30:21 +01:00
0xc0170 bf2ad16c69 project - zip and src check if empty fix
Regression from #2179. Fixes #2184.
2016-07-18 15:47:01 +01:00
Yogesh Pande e6043daa04 Changing -O2 parameter to -Os
This commit includes
- Changing build option parameter from -O2 to -Os to reduce flash size of the
  final binary pacakge.
  The original parameter of -O2 is kept to circumvent the error
  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46762 .
 But this error has been long fixed and released in latest 4.9.x GCC compiler.

 This change is necessary to reduce the generated binary size of mbed Client component to
avoid problem of getting images over 512 kB which is causing application to crash because of
Configuration Store problem https://github.com/ARMmbed/configuration-store/issues/21
2016-07-18 13:59:39 +03:00
Martin Kojtal 3ea625c8eb Merge pull request #2161 from PrzemekWirkus/devel_memap_summary_fields
[Tools] Replace in memap CSV report fields
2016-07-18 11:41:18 +01:00
Mihail Stoyanov 4e1ac4dba2 Merge pull request #2179 from sarahmarshy/progen_build_tests_v2
IDE build tests with progen
2016-07-16 17:54:46 +01:00
Martin Kojtal 7251a8ccae Merge pull request #2176 from 0xc0170/fix_#2171
Exporters - progen TARGETS lazy evaluated
2016-07-15 23:55:48 +01:00
Sarah Marsh b969fa5bb1 IDE build tests with progen 2016-07-15 13:25:35 -05:00
Jimmy Brisson bccbe48f3f Updated all docstrings to be compatible with PEP 257
It's nicer to be compatible with the standards.

I have added one more field to the description of each function:
Side effects. This feild contains the expected side effects of
running a particular method. If the side effects of a method are
None, it is expected that the function does not change anything in
any envoronment and that running it multiple times with the same
arguments will produce the same result every time. That is when
Side effects is non, the method is expected to be pure.
2016-07-15 10:51:47 -05:00
Jimmy Brisson 4fc8c56410 Replace pass with rais NotImplemented in each abstact method
The methods themselfes do nothing within the abstract base class
mbedToolchain. This change enforces that toolchains do not call
these methods that contain no implementation.
2016-07-15 10:17:35 -05:00
Jimmy Brisson f6b8cf05b0 Add a test that exercises the abstract base class checking
The test simply tries to instantiate each class exposed as part of the
TOOLCHAIN_CLASSES map. This is a test that each toolchain class
implements the required API, as the mbedToolchain will not allow the
instantiation of a subclass without a part of that API.
2016-07-15 10:10:40 -05:00
Jimmy Brisson 17833e8f58 Abstractify the mbedToolchain base class
The mbedToolchain class calls many members of it's subclasses, expecting
them to implement a particular API. This change adds a requirement to
each subclass that requires them to implement this expected API.

The API consists of these methods:
 - parse_dependencies
 - parse_ouptut
 - get_config_option
 - compile_c
 - compile_cpp
 - link
 - archive
 - binary
2016-07-15 10:07:56 -05:00
Bogdan Marinescu a558c49240 Rebuild sources if configuration is changed
This commit adds a check for configuration data changes. If a change in
configuration data is detected, all the sources in the tree are rebuilt.

This is a fix for #2073. #2162 was originally proposed as a fix, but it
was agreed that `--preinclude` is a more convenient way to include
configuration data, since it can be used to change the behaviour of
source files that don't include "mbed_config.h" directly, which is a big
advantage when importing 3rd party source trees. Compared to #2162, this
commit has the disadvantage of rebuilding all the source files if a
configuration change is detected, but it was agreed that the advantage
of using `--preinclude` outweighs the disadvantage of the increased
compilation time.
2016-07-15 16:37:46 +03:00
0xc0170 f95265f38d Exporters - progen TARGETS lazy evaluated
To speed up project.py, use @property for TARGETS.
When exporters are used, it would ask progen for all targets support
(number of tools x number of targets), this takes a lot of  time, thus
lazily evaluated, and TARGETS are for backaward compatibility, not currently
used by project.py but by other scripts to check if a target is supported.

Profiling:
```
python tools\project.py -m K64F -i uvision -n MBED_10 -b

Before the change:
       1    0.000    0.000    0.010    0.010 definitions.py:113(get_tool_definition)
      824    0.004    0.000    2.990    0.004 definitions.py:124(is_supported)
        1    0.000    0.000    0.000    0.000 definitions.py:145(get_debugger)
        1    0.002    0.002    0.044    0.044 definitions.py:15(<module>)
      384    0.002    0.000    2.986    0.008 definitions.py:26(_load_record)
        1    0.000    0.000    0.000    0.000 definitions.py:32(ProGenMcus)
      992    0.055    0.000    1.959    0.002 definitions.py:34(__init__)
     1376    0.002    0.000    0.003    0.000 definitions.py:40(get_mcus)
      384    0.001    0.000    1.070    0.003 definitions.py:43(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:50(ProGenTargets)
      992    0.000    0.000    0.000    0.000 definitions.py:52(__init__)
     1196    0.002    0.000    0.004    0.000 definitions.py:55(get_targets)
      204    0.001    0.000    1.922    0.009 definitions.py:58(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:67(get_debugger)
        1    0.000    0.000    0.000    0.000 definitions.py:74(ProGenDef)
      992    0.008    0.000    1.973    0.002 definitions.py:83(__init__)

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 4
Milliseconds      : 723
Ticks             : 47237302
TotalDays         : 5.46728032407407E-05
TotalHours        : 0.00131214727777778
TotalMinutes      : 0.0787288366666667
TotalSeconds      : 4.7237302
TotalMilliseconds : 4723.7302

After the change:

        1    0.000    0.000    0.010    0.010 definitions.py:113(get_tool_definition)
        2    0.000    0.000    0.025    0.012 definitions.py:124(is_supported)
        1    0.000    0.000    0.000    0.000 definitions.py:145(get_debugger)
        1    0.002    0.002    0.042    0.042 definitions.py:15(<module>)
        3    0.000    0.000    0.035    0.012 definitions.py:26(_load_record)
        1    0.000    0.000    0.000    0.000 definitions.py:32(ProGenMcus)
        2    0.000    0.000    0.005    0.002 definitions.py:34(__init__)
        5    0.000    0.000    0.000    0.000 definitions.py:40(get_mcus)
        3    0.000    0.000    0.000    0.000 definitions.py:43(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:50(ProGenTargets)
        2    0.000    0.000    0.000    0.000 definitions.py:52(__init__)
        5    0.000    0.000    0.000    0.000 definitions.py:55(get_targets)
        3    0.000    0.000    0.035    0.012 definitions.py:58(get_mcu_record)
        1    0.000    0.000    0.000    0.000 definitions.py:67(get_debugger)
        1    0.000    0.000    0.000    0.000 definitions.py:74(ProGenDef)
        2    0.000    0.000    0.005    0.003 definitions.py:83(__init__)

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 1
Milliseconds      : 178
Ticks             : 11780618
TotalDays         : 1.3634974537037E-05
TotalHours        : 0.000327239388888889
TotalMinutes      : 0.0196343633333333
TotalSeconds      : 1.1780618
TotalMilliseconds : 1178.0618
```
2016-07-15 14:02:57 +01:00
Martin Kojtal 3d20b10b4d Merge pull request #2172 from theotherjimmy/enfoce-supported-toolchains
[tools-build.py]Enforce supported toolchains
2016-07-15 09:02:14 +01:00
Martin Kojtal 3a2b752169 Merge pull request #2174 from theotherjimmy/return-from-build
[tools/build_api] Retrun truthy value on success from build_*
2016-07-15 09:01:31 +01:00
Sam Grove eca0c69e6c Merge pull request #2141 from theotherjimmy/project-argparse
[Project.py] Make -i and -m optional
2016-07-14 18:00:20 -05:00
Jimmy Brisson 6aa2bbb801 [tools/build_api] Retrun truthy value on success from build_*
before:
```
$  python tools/build.py  -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.39)s

Build skipped:
  * GCC_ARM::NRF51_DK
  * ARM::NRF51_DK

```
after:
```
$  python tools/build.py  -m NRF51_DK --source . -t GCC_ARM,ARM --build build --silent
[SNIP]
Completed in: (3.43)s

Build successes:
  * GCC_ARM::NRF51_DK
  * ARM::NRF51_DK

```
2016-07-14 13:55:51 -05:00
Jimmy Brisson 24fcb4700b [tools-build.py]Enforce supported toolchains
by skipping unsupported combinations
2016-07-14 13:29:55 -05:00
0xc0170 125754bf31 Exporter uvision - fix build_dir, only windows slashes allowed
uvision errors otherwise, not possible to create a path.
2016-07-14 14:21:05 +01:00
0xc0170 9c3801b7b5 Config - fix - use basetring to check if a target is string 2016-07-13 14:59:48 +01:00
Bogdan Marinescu d0d023ab58 Merge pull request #2122 from theotherjimmy/colorized-output
Colorized the short Warnings/Errors generated by the toolchains
2016-07-13 14:21:15 +03:00
Przemek Wirkus bba13ddad2 Replace in memap CSV report fields
Changes:
* This change impacts #2047
* In code summary columns were shortened and names changed a bit,
  see [here](https://github.com/mbedmicro/mbed/pull/2047/files?w=1#diff-f9cb941bde5647a5763e18fc220efc51R410)
* There is a need to allign memap CSV report fields (summary columns) with what
  is in code now. This discrapency may create problems when generating reports
  in the future. All columns should have the same name.
* memap CSV sumamry fields names changes:
  'total_static_ram' -> 'static_ram'
  'allocated_heap' -> 'heap'
  'allocated_stack' -> 'stack'

See example of discrepancy:

'total_static_ram' vs 'static_ram'
'allocated_heap' vs 'heap'
'allocated_stack' vs 'stack'

What is in code and presented with prettytable:

+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| name                                   | target | toolchain | static_ram | stack |  heap | total_ram | total_flash |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| tests-mbed_drivers-c_strings           | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       74182 |
| tests-mbed_drivers-callback            | K64F   | GCC_ARM   |      11980 | 32768 | 65540 |    110288 |       88406 |
| tests-mbedmicro-rtos-mbed-threads      | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       75918 |
| tests-mbedmicro-rtos-mbed-timer        | K64F   | GCC_ARM   |      10788 | 32768 | 65540 |    109096 |       68534 |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+

vs CSV:

```
total_static_ram,allocated_heap,allocated_stack,total_ram,total_flash
10788,65540,32768,109096,67486
```

vs memstats_map.JSON:

```json
[
    {
        "module": "Fill",
        "size": {
            ".data": 4,
            ".bss": 2257,
            ".text": 173
        }
    },

    {
        "summary": {
            "total_flash": 67486,
            "allocated_stack": 32768,
            "total_static_ram": 10788,
            "total_ram": 109096,
            "allocated_heap": 65540
        }
    }
]
```
2016-07-13 11:25:04 +01:00
Bogdan Marinescu e9d225bb96 Merge pull request #2151 from mbedmicro/verbose_json_errors
More verrbose errors for JSON parsing
2016-07-13 10:29:31 +03:00
Bogdan Marinescu 33dd3fe8ad More verrbose errors for JSON parsing
This commit adds information about the location of problematic JSON
files when reporting a JSON parsing error.

Before this commit:

```
[ERROR] Expecting property name: line 7 column 9 (char 188)
```

After this commit:

```
Error parsing 'core/mbed_lib.json':
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
2016-07-12 18:52:34 +03:00