The previous code used a variable that was internal to the Target class.
This commit removes the argument to `set_targets_json_location`
completely, which forces Target to use the default locatio internally.
Custom targets were the origin of a number of issues with the mbed
tools, so it was decided that they need to be removed. This PR does just
that (and moves the "custom_targets" part of the config system tests
into a separate, per-test "targets.json" file to preserve the test
functionality).
uVision4 is no longer maintained by the Keil team (support was stopped
long time ago).
This is why we need to make the uvision exporter able to generate uVision5
projects by default (which is based on software packs).
Also, adding note that the uvision4 exporter is now deprecated and will be
removed in the future.
Simplicity Studio exporter doesnt really support header files in the root directory (looking at you, mbed_config.h), so this is a workaround for that issue.
The previous way of finding group names found the encompassing directory of each file. If the project is exported from the online compiler, this resulted in temporary folder names like tmpyKKWv_ showing up as group names.
I propose defaulting to the project name if the file is in the project root.
The dict allows the user of the exporter api to specify the result directory
of particular groups of scanned dirs. This will be used by the online exporters
to spoof everything being in the same directory when they are not. It may also
be used by tests, if they would like to export something that looks exactly
like a normal project.
Affects these ides:
- Atmel Studio
- Code Red (I don't think we support this)
- Coide
- DS-5
- E2Studio
- EMblocks
- KDS
- Simplicity v3
- SW 4 STM32
also corrects flags usage in EMBlocks
Makes several broad changes:
- removes dead code that dealt with the online build system
- replaces export function with a much simpler one that:
- does not copy any sources
- the zip file hits the disk
- the mbed_config.h hits the disk
- the project files hit the disk
- nothing else hits the disk
- exporters use Resource object scanned with a toolchain
- progen exporters don't optionally build a project instead they have a
build function that may be called afterwards
- much of the code passes pylint (have a score of 9 or above):
- project.py
- project_api.py
- export/__init__.py
- export/exporters.py
- test/export/build_test.py
#2613 should be merged first
Using test_api, I found that the test names were dependent on where mbed-os (if that is the root) is stored on disk if you provide anything other than '.' as the root directory.
This would change names like:
```
repos-mbed-os-example-blinky-mbed-os-features-storage-feature_storage-tests-cfstore-example3
```
to
```
features-storage-feature_storage-tests-cfstore-example3
```
The shared `prepare_toolchain` and `scan_resources` functions in build_api
expect a list, not a string. This is different from the
toolchain.scan_resources function. Now the functions are being used
correctly within the `find_tests` function in test_api
This came up when building tests, but affects any library that's built
that uses config and included as "source" with the mbed tools.
JSON files are not copied by default when building a library, so when this
is built it loses the config data associated with the library. This commit
copies all JSON files into the build directory when building libraries.
This leads to two mbed_config.h files existing if the build directories
are different between the library and application build. This is the case
when building tests, so an option build_library was added to remove the
mbed_config.h file when compilation is done. This disabled by default when
building libraries, but it is enabled when building tests.
These two flags are already tested in tools/toolchains/gcc.py but can't
be used at the moment because they are not part of the argument list of
the option parameter.
Keep track of the current size allocated, maximum size allocated,
number of allocations, failed allocations and total size allocated for
both GCC and ARM. Report the maximum size allocated at the end of
testing.
Also, add a test to verify heap metrics are working as expected.
The output of the GCC compiler is such that the toolchain regex sometimes
got hung up on the ':' charcter being printed in front of the drive letter when
running on Windows. This PR changes the matching logic to be more flexible
by using 'search' to check the entire string for a match, not just the
beginning of the string.
`build_lib` wasn't aware of configuration data, so some of the source
files that depends on configuration data fail to compile. This commit
fixes that by making `build_lib` aware of configuration data.
Fixes#2360.
New error:
[Error] Toolchain path does not exist for IAR.
Current value: /default/path/that/doesnt/exist
(System exit before any build system calls)
Previously, build_mbed_libs didn't take into account the configuration.
This is a problem when doing a binary release, since mbed_lib.json
contains configuration data relevant to the binary release.
The micro:bit is running rtos tests, even though this is a 16K device
that should not be using multiple threads. This patch removes it from
the 2.0 RTOS tests.
Now "all" depend on the project merged with the soft device,
the goal merged has been renamed into the file it generate and now depend
on the realization of $(PROJECT).hex
At some point, the actual code that did the "clean" for build_library was
removed. This also affected building and cleaning tests. This adds this
capability back to the build API.
Unlike other libraries, uvisor is currently integrated into the build
system, with special cases based on its addition as a feature. This
suggests we should also have tests for these special cases.
Previously a scanning for config terminated on comparison of the feature
set, however at some point the set was changed to a list. This resulted
in non-deterministic failures based on list order.
Except for IAR, it does not support the --preinclude option for assembly
files but all of the other exporters can. It was less code to remove it
in IAR than to explicitly add it in all of the other exporters.
This PR enforces that the default_build in a target's definition in
hal/targets.json is set to standard to be included in the version 5
release.
This is being enforced because small builds do not support
multi-threading. This makes the capabilities more consistent across all
boards in the version 5 release.
NOTE: This removes the MOTE_L152RC and the LPC11U68
from the version 5 release list because the 'default_build' for these
targets is currently set to 'small'.
The key 'release' in hal/targets.json is ambiguous. This changes the key
to 'release_versions' to emphasize that the entries should be version
numbers/strings and that it should be an array, not a singular value.
This modifies the behavior of 'mcu_toolchain_matrix' in build_api.py. It
now prints release version support in the matrix. It also defaults to only
showing targets in mbed OS 5. You can still show older release versions by
modifying the 'release_version' parameter of the function.
Previously, the condition for including a target in a release was decided
by a 'release' key being set to 'true' in hal/targets.json. This doesn't
have enough granularity when we release multiple versions of mbed. This PR
changes the 'release' key to an array of strings, where each member is a
version that the target supports. Currently the valid versions are '2' and
'5'.
This PR also adds more robust checking for invalid target configurations
in a release. This is enforced whenever the release list is built from the
data, preventing invalid targets from coming into the release.
Finally, it updates the build_release.py script to use the new api for
fetching release targets.
Move the library files used by both nanostack and client into a common
location. This allows both client and nanostack to be built stand-alone.
The patch moves the following libraries into a common location:
-mbed-client-c
-mbed-client-randlib
-mbed-trace
-nanostack-hal-mbed-cmsis-rtos
-nanostack-libservice
-sal-stack-nanostack-eventloop
And moves the library 'coap-service' from client into nanostack.
Previously, test discovery throught test.py used its own logic to find
tests. This was mostly a subset of the rules used during the build
process. It respected .mbedignore files, but it did not respect TARGET_,
TOOLCHAIN_, and FEATURES_ directories. This change now uses the same logic
used during building to respect these folders. Tests under these folders
that are not being pulled in for the given target will be ignored.
Due to dependencies between the config/target systems, the config
has been modified to preparse the mbed_app.json for overrides.
This means that attributes with dependencies in the targets are
only valid at the application level. A hard error has been
added for these attributes if they are supplied at the library
level.
Modified '_process_config_and_overrides' to update cumulative
attributes based on cumulative overrides found during walking config
files.
This should remove most of the disparity between the config target
cumulative overrides and cumulative attributes set in targets.json.
These cumulative overrides are updated after the global target
instantiation though, which may cause internal build tool state
dependent on cumulative attributes to not correctly reflect all
cumulative overrides.
In tools/config.py, Config now aggregates all cumulative attributes in
target overrides.
{
"target_overrides": {
"K64F": {
"target.features_add": ["UVISOR"],
"target.extra_labels_add": ["UVISOR_SUPPORTED"],
"target.macros_add": ["UVISOR_MACRO"],
"target.device_has_add": ["UVISOR_HAS"]
}
}
}
Note: no action is performed on overrides yet
* added/improved global chroot support
* added RESPONSE_FILES flag to support optional response files (on linux the cmd param length is 2 megabytes). Default True
* added unified handling for archive and link response file (similar to includes)
* added COMPILE_C_AS_CPP flag to support compiling of c files as cpp. Default False
* added mbedToolchain.init() for post __init__ hooks
* added caching to mbedToolchain.need_update() to reduce IO hits
* added support to identify compiler warning/error column (supports ARMCC, GCC and IAR). Errors/warnings now report file@line,col
* added global TOOLCHAIN_PATHS which allows overriding/changing of the toolchain paths. Also simplified ARM-related paths
* added target.json to mbed library release (by @0xc0170)* migrated compile_worker() to utils.py for lightweight thread initialization
* improved run_cmd() performance by removing unnecessary check about the command being executed (should be checked once in the relevant toolchain instead)
* removed remnants of Goanna support (should be reimplemented as hooks to compile/link/archive instead)
* fixes for Python 2.7 compatibility (by @0xc0170)
* fixes for Exporters (by @0xc0170)
This commit includes
- Changing build option parameter from -O2 to -Os to reduce flash size of the
final binary pacakge.
The original parameter of -O2 is kept to circumvent the error
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46762 .
But this error has been long fixed and released in latest 4.9.x GCC compiler.
This change is necessary to reduce the generated binary size of mbed Client component to
avoid problem of getting images over 512 kB which is causing application to crash because of
Configuration Store problem https://github.com/ARMmbed/configuration-store/issues/21
It's nicer to be compatible with the standards.
I have added one more field to the description of each function:
Side effects. This feild contains the expected side effects of
running a particular method. If the side effects of a method are
None, it is expected that the function does not change anything in
any envoronment and that running it multiple times with the same
arguments will produce the same result every time. That is when
Side effects is non, the method is expected to be pure.
The methods themselfes do nothing within the abstract base class
mbedToolchain. This change enforces that toolchains do not call
these methods that contain no implementation.
The test simply tries to instantiate each class exposed as part of the
TOOLCHAIN_CLASSES map. This is a test that each toolchain class
implements the required API, as the mbedToolchain will not allow the
instantiation of a subclass without a part of that API.
The mbedToolchain class calls many members of it's subclasses, expecting
them to implement a particular API. This change adds a requirement to
each subclass that requires them to implement this expected API.
The API consists of these methods:
- parse_dependencies
- parse_ouptut
- get_config_option
- compile_c
- compile_cpp
- link
- archive
- binary
This commit adds a check for configuration data changes. If a change in
configuration data is detected, all the sources in the tree are rebuilt.
This is a fix for #2073. #2162 was originally proposed as a fix, but it
was agreed that `--preinclude` is a more convenient way to include
configuration data, since it can be used to change the behaviour of
source files that don't include "mbed_config.h" directly, which is a big
advantage when importing 3rd party source trees. Compared to #2162, this
commit has the disadvantage of rebuilding all the source files if a
configuration change is detected, but it was agreed that the advantage
of using `--preinclude` outweighs the disadvantage of the increased
compilation time.
Remove special case targets which have floating point enabled in
Newlib Nano from gcc.py. To enable floating point printf/scanf
for a target configure it to use the 'standard' default build in
targets.json rather than 'small'.
This commit adds information about the location of problematic JSON
files when reporting a JSON parsing error.
Before this commit:
```
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
After this commit:
```
Error parsing 'core/mbed_lib.json':
[ERROR] Expecting property name: line 7 column 9 (char 188)
```
Previously, add_py_targets assumed that it is not an error to add an
existing target if that target was previously added using
add_py_targets. This was done to aid testing, but it was weird, error
prone and broke with the latest changes to the caching mechanism. With
this commit:
- it is always an error to add a target that was previously added, which
is much more consistent.
- tests for the configuration mechanism use the newly added
'set_targets_json_location' function to clear the internal caches in
targets.py (and thus all previously added custom targets). As a side
effect, this commit also tests the 'set_targets_json_location'
function itself.
With this change, it becomes possible to use targets.py with any
targets.json, not just the default one in ../hal/targets.json.
targets.py will still be initialized with the default targets.json, but
the code can then call "set_targets_json_location" to specify the new
location of targets.json. set_targets_json_location modifies all the
data "in place"; that is, it doesn't create a new TARGET_MAP,
TARGET_NAMES or alike, but rather modified the existing ones. This is
important, because code using this construct:
from tools.targets import TARGET_MAP
can work unmodified with this change.
Now funnctions are looked up in the cache using a (function name,
arguments) key, which makes it possible to cache different invocations
of the functions (with different arguments).
Also applied the @cached attribute to get_target.
There are two shell-like things that we care about
bash: does environment varible expansion with $
batch: does environment varible expansion by surrounding something with %
so to detect batch, try to expand a variable on the shell with $.
If you get a $ back, you are in batch!
when in target.json "default_build": "small" is configured
- build.py+make.py
- uses linker option --specs=nano.specs
- macro MBED_RTOS_SINGLE_THREAD is defined
- exporting with project.py + make Makefile
- doesn't use the linker option --specs=nano.specs
- doesn't contain macro MBED_RTOS_SINGLE_THREAD
With this patch in place, tests.py uses targets names instead of target
instances, which makes it possible to use application defined targets
with tests.
With this change, custom targets defined by the application being
tested in its mbed_app.json file can be used with tests. Note that
`build_project` already accepts both target names and instances, so the
call to `build_project` inside `build_tests` will still work.
The configuration object is now created early in the build_project
function. This way, if there's a mbed_app.json that contains a custom
target, that target is taken into account. This is useful (for example)
when compiling tests for an application that defines a custom target.
- removing redundancy as discussed in PR #2087:
- in target.json the core option can have only this values : "Cortex-M0", "Cortex-M0+", "Cortex-M1", "Cortex-M3", "Cortex-M4", "Cortex-M7", "Cortex-A9" - Cortex-M4F and Cortex-M7F removed
- in target.json an additional fpu option with values: "single" and "double" can be used
- build and export scripts are changed to handle this
- tested (compiling, running on hardware) with nucleo_f767 (cortex-m7 with double precision fpu), nucleo_f746 (cortex-m7 with single precision fpu), nucleo_f446 and nucleo_l467 (cortex-m4 with single precision fpu), teensy31 (cortex-m4 without fpu - only build test), nucleo_l073 (cortex-m0)
- singletest results are added to PR #2087 comments
- creating new core name Cortex_M7F_DP for a target with a double precision fpu
- adding new core name to arm.py to set compiler/linker flags to a double precision fpu when configured in target.json
- up to now: gcc wrote flag for a double precision fpu -> target with STM32F746 didn't run when using double variables - mcu has only single precision fpu
- changing gcc.py to use single precision for Cortex-M7 und double precision for Cortex_M7F_DP
tested with NUCLEO_F746, NUCLEO_F767 and build.py+make.py and exporting with project.py + compiling/flashing
- iar.py need a similar extention - I didn't change that yet because
- did not run at the moment - python exception
- currently worked on in PR #1948
Sam Grove
jeromecoutant
Marcelo Salazar
Jimmy Brisson
Bogdan Marinescu
Sarah Marsh
Bogdan Marinescu
Steven Cooreman
Conor Keegan
OzzySan
Tony Wu
sarahmarshy
ohagendorf
Michel Jaouen
Brian Daniels
Radhika
Jeremy Brodt
Vincent Coubard
Martin Kojtal
Russ Butler
Mahadevan Mahesh
Martin Kojtal
Yoshihiro TSUBOI
Mihail Stoyanov
Russ Butler
Christopher Haster
Simon Hughes
Yogesh Pande
Mihail Stoyanov
Przemek Wirkus
Jimmy Brisson
Erwan GOURIOU
Erwan Gouriou
Laurent Meunier