Commit Graph

55 Commits (db965c16527add8ad6cf9ae14022d67f0b6a33d2)

Author SHA1 Message Date
Jimmy Brisson c93a2bfa4c Python2+3: Use absolute import where possible 2018-02-05 11:07:37 -06:00
Jimmy Brisson 4322bee175 Python2+3: tests passing 2018-02-05 11:04:36 -06:00
Jimmy Brisson 45bdd98a85 Python2+3: tests import correctly 2018-02-05 11:04:36 -06:00
Jimmy Brisson 10a91216f5 Python2+3: The rest of mbed compile 2018-02-05 11:04:01 -06:00
Jimmy Brisson 68737f2762 Python2+3: Things import 2018-02-05 11:03:44 -06:00
Christopher Haster 471d99c68f Added pretty bar printing for compile output
Looks like this:

    Building project mbed-os-prettyoutput (ARCH_PRO, GCC_ARM)
    Scan: .
    Scan: env
    Scan: mbed
    Scan: FEATURE_LWIP
    Text 70.5KB Data 2.72KB BSS 7.43KB                 ROM 73.2KB RAM 10.1KB
    ROM [|||||||                                             ]  73.2KB/512KB
    RAM [||||||||||||||||                                    ]   10.1KB/32KB
    Image: BUILD/ARCH_PRO/GCC_ARM/mbed-os-prettyoutput.bin

If you build a target without a cmsis-pack it looks like this:

    Building project mbed-os-prettyoutput (ARM_BEETLE_SOC, GCC_ARM)
    Scan: .
    Scan: env
    Scan: mbed
    Scan: FEATURE_BLE
    Text 99KB Data 2.84KB BSS 13KB ROM 102KB RAM 15.8KB
    Image: BUILD/ARM_BEETLE_SOC/GCC_ARM/mbed-os-prettyoutput.bin

And the old behaviour of displaying the memap table can be brought back
by passing the --stats-depth parameter
2018-01-25 11:46:23 -06:00
Jimmy Brisson 6a8be65904 Use non-scoped imports in memap 2018-01-02 14:19:00 -06:00
Jimmy Brisson 0c08fabbae Correct Gcc builtin module naming 2018-01-02 13:37:25 -06:00
Jimmy Brisson 527c0a2554 Rework parsing dispatch
Parsing dispatch now uses a metaclass as a Trait and a series of classes
that implement the trait for dispatching parsing. This structure gives
each parser it's own namespace, avoiding tacking on suffixes to each
attribute indicating which parser uses it.
2018-01-02 13:11:38 -06:00
Jimmy Brisson fc9bd60d4c Format memap 2018-01-02 10:43:10 -06:00
Jimmy Brisson 003913b22f Use os.sep and os.join instead of string ops 2018-01-02 10:31:00 -06:00
Jimmy Brisson 23c7af799f Handle multiple args per line in IAR cmd parser 2018-01-02 10:29:53 -06:00
Deepika 6804361d76 Build failed as code section was not included in map.
Fix by Jimmy
2017-12-20 09:49:43 -06:00
Jimmy Brisson 0524ba40bd Remove debug print from memap IAR impl 2017-10-03 13:46:30 -05:00
Jimmy Brisson 727256a05d Remove prefix stuff from GCC memap parser
It's just not used anymore
2017-09-20 11:00:02 -05:00
Jimmy Brisson 3d5bea337f Test the module_add function 2017-09-20 10:49:12 -05:00
Jimmy Brisson fdd4ae3056 Parse full paths from ARM map file
Now we don't have to scan!

This is a 20% speed improvement
2017-09-18 17:18:01 -05:00
Jimmy Brisson 6d135c26b3 Remove warnings for Zero sized sections
We just don't care if we don't know where they go
2017-09-18 15:45:05 -05:00
Jimmy Brisson 0d2a0a0c1d Parse filenames from IAR map file
Instead of scanning.

Is a 8ms/15% speedup.
2017-09-18 15:45:03 -05:00
Jimmy Brisson 2114ccd5a1 Refactor memap for speed 2017-09-18 15:24:18 -05:00
Jimmy Brisson 7eccec425f Parse libraries with memap-arm 2017-09-14 09:50:47 -05:00
Jimmy Brisson 98b4768434 Initial toolchain class for ARMC6
While ARMC6 does use the same linker (armlink) as ARM Compiler 5, it
is not compatible.

The reason for this incompatibility are twofold:
 * armlink may invoke the C preprocessor by adding a shebang
   (`#!`) to the top of their input files.
 * ARMC6 and ARMC5 differ in how you invoke the preprocessor:
   * ARMC5: `#! armcc -E`
   * ARMC6: `#! armclang -E`

This forces the tools to rewrite the shebang if it's wrong.

This does not yet handle dependencies properly
2017-09-11 13:17:54 -05:00
Marcelo Salazar 7053ef9412 memap enhancements with depth level configurable
Removed heap/stack from report
Add --stats-depth option to mbed compile
Fix minor issues identified in review
Fix bug when parsing armcc libs
Fix code style
Fix minor issues on existing tests
Fix memap for automated TESTS
Fix stats-depth parameter for tests
2017-08-06 10:30:04 +01:00
Sam Grove 69de788f94 Merge pull request #3712 from JojoS62/enable-GCC_CR-in-memmap
add GCC_CR as toolchain for memmap
2017-02-17 09:11:22 -06:00
JojoS 649d8a1d15 add GCC_CR as toolchain for memmap 2017-02-07 15:05:32 +01:00
javier-moreno-tridonic-com 8ff17b9235 Update memap.py 2017-01-23 17:02:15 +01:00
Sam Grove 2fe28f0da1 [tools] update comments in memmap about build folder name 2016-11-18 00:22:58 -06:00
Sam Grove 6bd44c59d3 Merge pull request #2990 from bridadan/parallel-test-build
[tools] Parallel building of tests
2016-10-19 05:41:24 -05:00
Russ Butler 7a50832550 Revert "Allow max-depth specification in memap"
This reverts commit 8f9f4eefed since
it broke memory map info for IAR, ARMCC and on windows, GCC_ARM.
2016-10-12 14:49:32 -05:00
Brian Daniels 39086722e8 Removing dependency on StringIO, only change return type of table export 2016-10-11 18:38:52 -05:00
Brian Daniels 655c37715c Handling output of parallelized test building.
This makes use of the reports generated by the building of tests to
prevent output from interleaving when the build is parallelized. This
required some changes to memap to return a generated string from
the 'generate_output' function. I also had an option to stop the prints
from memap to prevent text from interleaving
2016-10-11 15:34:20 -05:00
Bartek Szatkowski a7d9dbec69 Fix file path quoting while constructing CSV file in memmap.py
Comma is a valid char in file paths for Posix systems.
2016-10-07 16:33:43 +01:00
Jimmy Brisson 8f9f4eefed Allow max-depth specification in memap 2016-10-05 12:36:30 -05:00
Sam Grove 23f70906eb Merge pull request #2845 from javier-moreno-tridonic-com/feature-detailed-memap
[Tools] Added -d (--detailed) paremeter to unfold 'Misc' contents in memap.py table
2016-09-30 18:54:45 -05:00
Jimmy Brisson 2b36d2de5c Moved detailed memap parameter to consturctor 2016-09-30 15:44:20 -05:00
Javier Gomez Moreno e0e25c1bed The parameter is now stored in an object variable instead of a global variable 2016-09-30 08:45:20 +02:00
Brian Daniels abb6d2681e Adding memap tests and adding to travis ci 2016-09-28 17:27:25 -05:00
Javier Gomez Moreno f8af872870 Added -d paremeter to unfold 'Misc' contents in memap.py 2016-09-28 11:58:27 +02:00
Brian Daniels e8b06874a3 Making memap output consistent across output formats
This commit fixes an issue where the output from memap.py was not
consistent across all output formats. This issue stemmed from the fact
that a few important calculations were being performed at output
generation time. This has been moved to the 'parse' function and saved for
future use by the 'generate' functions.

Because this commit saves more data to the MemapParser instance, there
were some name collisions. The public member 'mem_summary' has been
renamed to 'mem_report'. 'mem_report' contains the data structure used by
the json generator. This includes both the section data and the memory
summary. The 'mem_summary' member now just contains the summary. The
summary includes total allocated heap, total static RAM, etc.
2016-09-27 16:51:16 -05:00
Marcelo Salazar 56604a04c8 Fixed bug on memap (non-default output path)
Removed code that checks if the path to the map file
matches with the selected toolchain.
2016-09-02 19:05:27 +03:00
Jimmy Brisson 74dd5efd48 Print module information in 'table' output format 2016-08-18 13:25:58 -05:00
Jimmy Brisson cb5e61028d Format and add docstrings to memmap 2016-08-16 11:48:39 -05:00
Przemek Wirkus bba13ddad2 Replace in memap CSV report fields
Changes:
* This change impacts #2047
* In code summary columns were shortened and names changed a bit,
  see [here](https://github.com/mbedmicro/mbed/pull/2047/files?w=1#diff-f9cb941bde5647a5763e18fc220efc51R410)
* There is a need to allign memap CSV report fields (summary columns) with what
  is in code now. This discrapency may create problems when generating reports
  in the future. All columns should have the same name.
* memap CSV sumamry fields names changes:
  'total_static_ram' -> 'static_ram'
  'allocated_heap' -> 'heap'
  'allocated_stack' -> 'stack'

See example of discrepancy:

'total_static_ram' vs 'static_ram'
'allocated_heap' vs 'heap'
'allocated_stack' vs 'stack'

What is in code and presented with prettytable:

+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| name                                   | target | toolchain | static_ram | stack |  heap | total_ram | total_flash |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+
| tests-mbed_drivers-c_strings           | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       74182 |
| tests-mbed_drivers-callback            | K64F   | GCC_ARM   |      11980 | 32768 | 65540 |    110288 |       88406 |
| tests-mbedmicro-rtos-mbed-threads      | K64F   | GCC_ARM   |      11468 | 32768 | 65540 |    109776 |       75918 |
| tests-mbedmicro-rtos-mbed-timer        | K64F   | GCC_ARM   |      10788 | 32768 | 65540 |    109096 |       68534 |
+----------------------------------------+--------+-----------+------------+-------+-------+-----------+-------------+

vs CSV:

```
total_static_ram,allocated_heap,allocated_stack,total_ram,total_flash
10788,65540,32768,109096,67486
```

vs memstats_map.JSON:

```json
[
    {
        "module": "Fill",
        "size": {
            ".data": 4,
            ".bss": 2257,
            ".text": 173
        }
    },

    {
        "summary": {
            "total_flash": 67486,
            "allocated_stack": 32768,
            "total_static_ram": 10788,
            "total_ram": 109096,
            "allocated_heap": 65540
        }
    }
]
```
2016-07-13 11:25:04 +01:00
Jimmy Brisson 61c63b3789 Argument error when memap's input file does not exist 2016-07-07 15:29:36 -05:00
Jimmy Brisson 0ec8199c55 Revert "Revert "Generalize flag handling""
This reverts commit 33cec194c6.
2016-07-07 15:29:36 -05:00
Przemek Wirkus 12a01f61ca Add summary for test building
When users are building tests with `mbed compile --tests` whey will by default
get memory map breakdown report.
This can be suppresed in the future with command line switch. For now it is
visible each time users build test cases.
List is sorted by project name created with `build_project` API.

Changes:
* `build_project` now returns tuple (this breaks build_api.build_project API!)
* Memmap script got a aggregation function to print summary from small 'JSON'
  partial reports.
* Report is generated by `test_api.build_tests` function only!

Example:
```
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| name                                                                 | target | toolchain | total_flash | static_ram | stack |  heap | total_ram |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
| features-feature_ipv4-tests-mbedmicro-net-nist_internet_time_service | K64F   | GCC_ARM   |      132136 |      62288 | 32768 | 65536 |    160592 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_echo            | K64F   | GCC_ARM   |      125613 |      62448 | 32768 | 65540 |    160756 |
| features-feature_ipv4-tests-mbedmicro-net-tcp_client_hello_world     | K64F   | GCC_ARM   |      125949 |      62448 | 32768 | 65540 |    160756 |
| features-feature_ipv4-tests-mbedmicro-net-udp_echo_client            | K64F   | GCC_ARM   |      123613 |      62276 | 32768 | 65536 |    160580 |
| features-storage-tests-cfstore-add_del                               | K64F   | GCC_ARM   |       96080 |      13052 | 32768 | 65540 |    111360 |
| features-storage-tests-cfstore-close                                 | K64F   | GCC_ARM   |       95520 |      12004 | 32768 | 65540 |    110312 |
| features-storage-tests-cfstore-create                                | K64F   | GCC_ARM   |       99144 |      13036 | 32768 | 65540 |    111344 |
| features-storage-tests-cfstore-example1                              | K64F   | GCC_ARM   |       98592 |      12368 | 32768 | 65536 |    110672 |
| features-storage-tests-cfstore-example2                              | K64F   | GCC_ARM   |       95232 |      12012 | 32768 | 65540 |    110320 |
| features-storage-tests-cfstore-example3                              | K64F   | GCC_ARM   |       95264 |      11856 | 32768 | 65536 |    110160 |
| features-storage-tests-cfstore-example4                              | K64F   | GCC_ARM   |       92632 |      12012 | 32768 | 65540 |    110320 |
| features-storage-tests-cfstore-example5                              | K64F   | GCC_ARM   |       92344 |      11856 | 32768 | 65536 |    110160 |
| features-storage-tests-cfstore-find                                  | K64F   | GCC_ARM   |       96344 |      13028 | 32768 | 65540 |    111336 |
| features-storage-tests-cfstore-find2                                 | K64F   | GCC_ARM   |       93192 |      12004 | 32768 | 65540 |    110312 |
| features-storage-tests-cfstore-flash                                 | K64F   | GCC_ARM   |       97784 |      12532 | 32768 | 65540 |    110840 |
| features-storage-tests-cfstore-flush                                 | K64F   | GCC_ARM   |       96464 |      12012 | 32768 | 65540 |    110320 |
| features-storage-tests-cfstore-flush2                                | K64F   | GCC_ARM   |       95056 |      12004 | 32768 | 65540 |    110312 |
| features-storage-tests-cfstore-init                                  | K64F   | GCC_ARM   |       93120 |      12012 | 32768 | 65540 |    110320 |
| features-storage-tests-cfstore-misc                                  | K64F   | GCC_ARM   |       96808 |      12516 | 32768 | 65540 |    110824 |
| features-storage-tests-cfstore-open                                  | K64F   | GCC_ARM   |       98632 |      12540 | 32768 | 65540 |    110848 |
| features-storage-tests-cfstore-read                                  | K64F   | GCC_ARM   |       94112 |      12540 | 32768 | 65540 |    110848 |
| features-storage-tests-cfstore-write                                 | K64F   | GCC_ARM   |       94488 |      12004 | 32768 | 65540 |    110312 |
| features-storage-tests-flash_journal-basicapi                        | K64F   | GCC_ARM   |      104712 |      21236 | 32768 | 65540 |    119544 |
| frameworks-utest-tests-unit_tests-basic_test                         | K64F   | GCC_ARM   |       71534 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-case_async_validate                | K64F   | GCC_ARM   |       74598 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-case_control_async                 | K64F   | GCC_ARM   |       74630 |      11476 | 32768 | 65540 |    109784 |
| frameworks-utest-tests-unit_tests-case_control_repeat                | K64F   | GCC_ARM   |       72790 |      11452 | 32768 | 65540 |    109760 |
| frameworks-utest-tests-unit_tests-case_selection                     | K64F   | GCC_ARM   |       72302 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-case_setup_failure                 | K64F   | GCC_ARM   |       72630 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-case_teardown_failure              | K64F   | GCC_ARM   |       72790 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-control_type                       | K64F   | GCC_ARM   |       82462 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-minimal_async_scheduler            | K64F   | GCC_ARM   |       72182 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-minimal_scheduler                  | K64F   | GCC_ARM   |       71998 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-test_assertion_failure_test_setup  | K64F   | GCC_ARM   |       71710 |      11460 | 32768 | 65540 |    109768 |
| frameworks-utest-tests-unit_tests-test_setup_case_selection_failure  | K64F   | GCC_ARM   |       71702 |      11468 | 32768 | 65540 |    109776 |
| frameworks-utest-tests-unit_tests-test_setup_failure                 | K64F   | GCC_ARM   |       71710 |      11468 | 32768 | 65540 |    109776 |
| tests-integration-basic                                              | K64F   | GCC_ARM   |       67566 |      10780 | 32768 | 65540 |    109088 |
| tests-integration-threaded_blinky                                    | K64F   | GCC_ARM   |       68326 |      10780 | 32768 | 65540 |    109088 |
| tests-mbed_drivers-c_strings                                         | K64F   | GCC_ARM   |       74438 |      11468 | 32768 | 65540 |    109776 |
| tests-mbed_drivers-callback                                          | K64F   | GCC_ARM   |       88310 |      11972 | 32768 | 65540 |    110280 |
| tests-mbed_drivers-dev_null                                          | K64F   | GCC_ARM   |       90213 |      10784 | 32768 | 65540 |    109092 |
| tests-mbed_drivers-echo                                              | K64F   | GCC_ARM   |       71918 |      11468 | 32768 | 65540 |    109776 |
| tests-mbed_drivers-generic_tests                                     | K64F   | GCC_ARM   |       77624 |      11468 | 32768 | 65540 |    109776 |
| tests-mbed_drivers-rtc                                               | K64F   | GCC_ARM   |       85854 |      11308 | 32768 | 65540 |    109616 |
| tests-mbed_drivers-stl_features                                      | K64F   | GCC_ARM   |       80726 |      11476 | 32768 | 65540 |    109784 |
| tests-mbed_drivers-ticker                                            | K64F   | GCC_ARM   |       70974 |      11308 | 32768 | 65540 |    109616 |
| tests-mbed_drivers-ticker_2                                          | K64F   | GCC_ARM   |       70790 |      11308 | 32768 | 65540 |    109616 |
| tests-mbed_drivers-ticker_3                                          | K64F   | GCC_ARM   |       71038 |      11308 | 32768 | 65540 |    109616 |
| tests-mbed_drivers-timeout                                           | K64F   | GCC_ARM   |       70886 |      11308 | 32768 | 65540 |    109616 |
| tests-mbed_drivers-wait_us                                           | K64F   | GCC_ARM   |       70414 |      11308 | 32768 | 65540 |    109616 |
| tests-mbedmicro-mbed-attributes                                      | K64F   | GCC_ARM   |       71534 |      11460 | 32768 | 65540 |    109768 |
| tests-mbedmicro-mbed-call_before_main                                | K64F   | GCC_ARM   |       73112 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-mbed-cpp                                             | K64F   | GCC_ARM   |       73400 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-mbed-div                                             | K64F   | GCC_ARM   |       73176 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-basic                                      | K64F   | GCC_ARM   |       68390 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-isr                                        | K64F   | GCC_ARM   |       74480 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-mail                                       | K64F   | GCC_ARM   |       74992 |      11300 | 32768 | 65540 |    109608 |
| tests-mbedmicro-rtos-mbed-mutex                                      | K64F   | GCC_ARM   |       74048 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-queue                                      | K64F   | GCC_ARM   |       74912 |      11300 | 32768 | 65540 |    109608 |
| tests-mbedmicro-rtos-mbed-semaphore                                  | K64F   | GCC_ARM   |       74296 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-signals                                    | K64F   | GCC_ARM   |       74328 |      10780 | 32768 | 65540 |    109088 |
| tests-mbedmicro-rtos-mbed-threads                                    | K64F   | GCC_ARM   |       75214 |      11460 | 32768 | 65540 |    109768 |
| tests-mbedmicro-rtos-mbed-timer                                      | K64F   | GCC_ARM   |       68430 |      10780 | 32768 | 65540 |    109088 |
| tests-storage_abstraction-basicapi                                   | K64F   | GCC_ARM   |      107808 |      28908 | 32768 | 65540 |    127216 |
+----------------------------------------------------------------------+--------+-----------+-------------+------------+-------+-------+-----------+
```

Refactored after code review

Refactored parse() function

Polishing

Moved memory usage reporting function call to test.py to group all reporters in one place

Bug-fix: on ARM toolchain we were not fetching statistics from last element of memap result list
2016-07-07 08:35:20 +01:00
Marcelo Salazar d50139c8e3 Minor tweaks 2016-06-27 15:02:05 +01:00
Marcelo Salazar fbd8a41fed Reverted changes and improved wording 2016-06-27 13:31:56 +01:00
Marcelo Salazar c0f53ccb21 [memap] Removing stack/heap and minor improvements
- Removing stack & heap (dynamic) RAM information
  This information was misleading and shouldn't be shown in memap.
  E.g. each task may have its own stack region configured at run time.

- Adding 'bytes' unit in the total memory info

- Right aligment of numbers, so it is easier to compare numbers
2016-06-27 11:44:01 +01:00
Sam Grove 33cec194c6 Revert "Generalize flag handling" 2016-06-24 16:39:01 -05:00