toem - U009 Logging and Tracing 4 - Combine multiple sources

impulse 2.1

U009 Logging and Tracing 4 - Combine multiple sources

Attach, View Anaylze and Solve!

In the first two articles it was assumed that all log data comes from one file. But the fact is that complex systems with multiple domains/cores usually have more than a single log! - e.g. a hardware log (systemC), firmware traces from multiple cores and DSPs and application logs.

This article explains how to combine log and trace data from multiple sources. We use the Log4J input from the previous article (systemA) and combine it with CSV data (systemB). Both sources provide a ms time-stamp. But unfortunately the start point differs :-(.

CSV Data

The CSV Data and Log Reader is a configurable reader for all typical kinds of CSV (Comma-Separated Values) based data and log formats. CSV is a common data exchange format that is widely supported by consumer, business, and scientific applications. With easy to use dialogs you can define configurations (separators, first line, labels, data types,...) for your specific formats. This article will help you configure and use the reader.

The systemB input contains a ms time-stamp, a message and 2 float signals.

1588251996590;" ACPI: PCI Interrupt Link [LNKF";-0.7264048716430498;1.0569470760874955
1588251996590;"   node   0: [mem 0x0000000000001000-0x000000000009bfff";-0.7264048716430498;1.0569470760874955
1588251996592;" xhci_hcd 0000:00:14.0: Host supports USB 3.0 SuperSpeed";0.9272212259680832;2.4757980243910085

RL02 CSV Reader

Multi Adapter Port

The Multi Adapter Port and its derivatives "Multi Resource Port" and "Multi File/Pipe Port" combine multiple adapters into one port. You can present and analyze signals from multiple inputs inside of one view. The Multi Resource Port makes it possible to combine several resource adapters. A common resource root can be defined and used as a base in all adapters.

R012 Multi Resource Port

H130 Attach - How to add a port

A Signals Port can be opened by the impulse Viewer like a normal file. Instead of extracting the signal data from the file, it is read from an external source (e.g. TCP, a debug adapter or a bus interface).

To add a port you need to have the Signals Ports view visible (Window > Show View > impulse > Signal Ports).;
Click the "Add new Port" toolbar button of the "Signal Ports" view and select the port type. ;
Configure the port settings and press ok.

H131 Attach - How to configure existing ports

Ports can be managed most easily with the "Signal Ports" view: (Window > Show View > impulse > Signal Ports). Alternatively you can also use the preferences (Preferences ->impulse > Ports).

Press the "Edit Port" button of the "Signal Ports" view or use the context menu of the port/adapter element.
If you have already opened the port in a viewer, press the "Edit Port" button in the viewers toolbar.

H132 Attach - How to open and start a port

The port may read the signal data in one go or spread over a period of time (on-line). impulse allows the analysis of signal data while new data is still arriving.

To open a port you need to have the Signals Ports view visible (Window > Show View > impulse > Signal Ports).
Opening a port means opening the viewer with the signal content of the port. Press the "Open Port" button of the "Signal Ports" view or double-click the port item.
To start the port, press the "Run/Stop" in the tool-bar.

Screen Cast: Logging and Tracing 4 - Combine multiple sources

Download Tutorial Examples (Preference Wallet Uxxx.walMZ)

The CSV Reader Configuration

The CSV Reader requires a configuration (CSV Configuration) to work. A configuration contains information about the char set, the delimiter, labels and and the actual data fields. Press "Add" and select "CSV Configuration" to create a new reader configuration:

Char Set: Select the char set that is used by the log file (or "Default").
Domain Base: Select the domain base. This represents is smallest domain change (e.g. 1 ms).
Updates: This reader has some sample or standard configurations after installation. After an update of impulse, these configurations may also have been updated. To exclude a configuration from updating (e.g. because you have adapted the configuration for your application), you can set the flag "Do not update with latest version".
First row: Indicated the first row (starting with 1) of data or labels field.
No of columns: No of expected columns. If the no of columns is smaller than expected, an error will be thrown.
First row has labels: Check if the first row contains labels. These labels will be used to name the generated signals or members. If not given, standard names (s1 ,s2,..) will be taken.
Separator: The character to splits data and label fields.
Quote: Select if your content uses double-, single- or no quote meta-characters. Quotes are used to enclose text content with separator characters or quotes.
Mode:Select if the reader shall create a signal for each data column, or one struct signal with a member for each data column or log signals.

Name and configure the signals

Select member names and types for all columns that shall create a signal.

Type in the member names. If you leave the field blank, either a standard name is taken (e.g. s1) or an optional label defines the signal name.
Define the member type and a descriptor.

Configure the domain value source and format

Date: You need to define the date format. Use the content proposals and http://docs.oracle.com/javase/6/docs/api/java/text/SimpleDateFormat.html. The parsing of the date value results into milliseconds after January 1, 1970 00:00:00 GMT.
Float domain value: Parse a float value, optionally parse the domain unit from the source value or assign a unit.
Integer domain value: Parse an integer value, optionally parse the domain unit from the source value or assign a unit.

There may be CSV logs without a domain value. In this case you may want to use:

Incrementing: Increment the domain value with every new sample (all signals).
Incrementing (per Signal): Increment the domain value with every new sample per signals.
Reception time: Use the current time in milliseconds after January 1, 1970 00:00:00 GMT as domain value.

RL02 CSV Reader

Not in sync !

Unfortunately the time-stamps are not synchronous. The dependency of the two logs shall be done by a log sample with the message "EISA bus registered". At the point of this sample, the system shall start with a delay of 10ms.

Synchronize the systemB adapter

To synchronize systemB, open the adapter and check "Enable Sync", then click on "Edit in eclipse script editor" to open the editor and copy/paste the script below.

The system will call the synchronisation script continuously while reading the input until the method isync.setSynced(position) is called. The script first request a pointer to a given signal. If the signal is available, it tries to move the pointer to the valid position. If successful it calculates the delta and calls isync.setSynced.

// base: root cell of type ICell (usually a record cell)
// insertPoint: root cell of this port of type PortScope  
// isync: sync interface of type IPortSync  
// console: console output of type MessageConsoleStream

var log2 /*:ISamplePointer:*/ = isync.getPointer('systemA\\de.toem.impulse.test.secondary.Log2');

if (log2 != null) {
    var checked = 0;
    var synced = false;
    while (log2.hasNext() && checked < 100) {
        var msg = log2.stringValueOf("Message");
        if (msg != null && msg.equals("EISA bus registered")) {
        	
            isync.setSynced(log2.getPosition().add(DomainValue.valueOf("10ms")));
            synced = true;
            console.println("Synced: " + reset.getPosition());
            break;
        }
        checked++;
        log2.goNext();
    }
    if (!synced)
        console.println("Sync not found!");
} else
    console.println("Signal not found!");

The console shows the output:

Sync not found!
Sync message at: 752ms

11 Synchronisation