mbed-os/TEST_APPS

Running Icetea tests located under mbed-os

Structure

mbed-os has a folder called TEST_APPS that contains everything related to Icetea testing. There are currently 3 folders:

• device - contains all the different test applications you can flash to your board
• icetea-plugins - contains plugins that are being used by some of the testcases, needed for the test execution
• testcases - contains Icetea testcases written in Python

The testcases depends on test applications

Preparing your work environment

Prerequisities

You need Icetea and mbed-cli that supports Icetea, installed.

Selecting the network interface to use

Depending on a device, there might be a default network interface type defined in the mbed-os/targets/targets.json, which is used to locate a test-config file by default. If there is not, or you want to use a different interface than the default, you need to provide a relevant test-config -file to the mbed test with --test-config option. The test-config file contains the necessary information for the test application, there are some test-config files located under mbed-os/tools/test_configs. Devices which have their network drivers residing inside mbed-os can use generic test_configs like HeapBlockDeviceAndEthernetInterface.json and HeapBlockDeviceAndWifiInterface.json. Otherwise you need to use a device specific test-config.

Running the tests

Now that the interface has been selected you can run the icetea tests from the mbed-os root on your command line by

>mbed test -m <target> -t <toolchain> --icetea

This command will compile the mbed-os, then compiles the test applications, creates a test suite and then starts running the tests.

If you want only to run some specific tests, you can use the -n -option. You can list multiple tests by separating them by comma (,).

>mbed test -m <target> -t <toolchain> --icetea -n test1,test2

Running the tests with specifig test-config

Some devices may offer multiple network interfaces to operate with. For example, UBLOX_EVK_ODIN_W2 offers ethernet and Wi-Fi capabilities. The tests can be run for either one of those using already existing test-config -files.

To run the tests with Wi-Fi interface: >mbed test -m UBLOX_EVK_ODIN_W2 -t <toolchain> --icetea --test-config tools/test_configs/HeapBlockDeviceAndWifiInterface.json

To run the tests with ethernet interface: >mbed test -m UBLOX_EVK_ODIN_W2 -t <toolchain> --icetea --test-config tools/test_configs/HeapBlockDeviceAndEthernetInterface.json

Providing Wi-Fi access point information

If you are using Wi-Fi interface for running the tests, you need to provide also information about the used access point. The information can be provided in the used test-config file.

Example of access point information:

    "target_overrides": {
        "*": {
            "target.network-default-interface-type": "WIFI",
            "nsapi.default-wifi-ssid": "\"ssid\"",
            "nsapi.default-wifi-password": "\"password\"",
            "nsapi.default-wifi-security": "WPA_WPA2"
        }
    }

Test results

Icetea prints the results from the test run to the command line, and the final result looks similar to this.

+--------------------------------+---------+-------------+-------------+-----------+----------+
| Testcase                       | Verdict | Fail Reason | Skip Reason | platforms | duration |
+--------------------------------+---------+-------------+-------------+-----------+----------+
| test_cmdline                   |   pass  |             |             |    K64F   |  8.555   |
| UDPSOCKET_BIND_PORT            |   pass  |             |             |    K64F   |  19.614  |
| TCPSOCKET_BIND_PORT            |   pass  |             |             |    K64F   |  15.852  |
| TCPSERVER_ACCEPT               |   pass  |             |             |    K64F   |  41.629  |
| TCPSOCKET_ECHOTEST_BURST_SHORT |   pass  |             |             |    K64F   |  19.926  |
+--------------------------------+---------+-------------+-------------+-----------+----------+
+---------------+----------------+
|    Summary    |                |
+---------------+----------------+
| Final Verdict |      PASS      |
|     count     |       5        |
|    passrate   |    100.00 %    |
|      pass     |       5        |
|    Duration   | 0:01:45.576000 |
+---------------+----------------+

The results from the tests can also be found in the mbed-os/log folder. You probably want to add the log folder to your .mbedignore file to prevent issues with build commands becoming too long over the time.