Intel(R) Trace Hub (TH)

Intel Trace Hub 개요

Intel(R) Trace Hub(TH)는 여러 hardware·software source의 trace data를 생성하고 전환해 다양한 output port로 내보내는 hardware block 집합이다. Data는 System Trace Protocol(MIPI STPv2)로 encode되며 full-system debugging을 목적으로 한다. Hardware 세부 사항은 Intel Trace Hub developer manual을 참조한다.

구성 요소는 trace source, trace destination(output), switch인 Global Trace Hub(GTH)다. 이 device들은 전용 `intel_th` bus에 놓이며 sysfs attribute로 발견하고 설정할 수 있다.

지원되는 subdevice는 software trace source인 STH, system memory output인 MSU, 외부 debug host로 보내는 PTI, 중앙 switch인 GTH다.

Output device의 공통 attribute는 `Documentation/ABI/testing/sysfs-bus-intel_th-output-devices`에 설명돼 있다. 핵심 `active` attribute는 해당 output device로의 trace 출력을 켜거나 끈다.

GTH는 `masters` attribute group으로 서로 다른 STP master를 서로 다른 output port에 보낸다. 자세한 interface는 `Documentation/ABI/testing/sysfs-bus-intel_th-devices-gth`에 있다.

STH는 stm class device를 등록하고 사용자 공간과 kernel 공간의 software trace source에 interface를 제공한다. 자세한 내용은 `Documentation/trace/stm.rst`를 참조한다.

MSU는 trace data를 system memory buffer에 모은다. 이후 device node의 `read()` 또는 `mmap()` interface로 읽거나 data를 소비·중계하는 software sink driver로 보낼 수 있다.

Intel Trace Hub는 동작을 위해 특별한 사용자 공간 software가 필요하지 않다. Sysfs attribute와 device node만으로 설정, 시작, 수집을 수행할 수 있다.

.. SPDX-License-Identifier: GPL-2.0

=======================
Intel(R) Trace Hub (TH)
=======================

Overview
--------

Intel(R) Trace Hub (TH) is a set of hardware blocks that produce,
switch and output trace data from multiple hardware and software
sources over several types of trace output ports encoded in System
Trace Protocol (MIPI STPv2) and is intended to perform full system
debugging. For more information on the hardware, see Intel(R) Trace
Hub developer's manual [1].

It consists of trace sources, trace destinations (outputs) and a
switch (Global Trace Hub, GTH). These devices are placed on a bus of
their own ("intel_th"), where they can be discovered and configured
via sysfs attributes.

Currently, the following Intel TH subdevices (blocks) are supported:
  - Software Trace Hub (STH), trace source, which is a System Trace
    Module (STM) device,
  - Memory Storage Unit (MSU), trace output, which allows storing
    trace hub output in system memory,
  - Parallel Trace Interface output (PTI), trace output to an external
    debug host via a PTI port,
  - Global Trace Hub (GTH), which is a switch and a central component
    of Intel(R) Trace Hub architecture.

Common attributes for output devices are described in
Documentation/ABI/testing/sysfs-bus-intel_th-output-devices, the most
notable of them is "active", which enables or disables trace output
into that particular output device.

GTH allows directing different STP masters into different output ports
via its "masters" attribute group. More detailed GTH interface
description is at Documentation/ABI/testing/sysfs-bus-intel_th-devices-gth.

STH registers an stm class device, through which it provides interface
to userspace and kernelspace software trace sources. See
Documentation/trace/stm.rst for more information on that.

MSU can be configured to collect trace data into a system memory
buffer, which can later on be read from its device nodes via read() or
mmap() interface and directed to a "software sink" driver that will
consume the data and/or relay it further.

On the whole, Intel(R) Trace Hub does not require any special
userspace software to function; everything can be configured, started
and collected via sysfs attributes, and device nodes.

[1] https://software.intel.com/sites/default/files/managed/d3/3c/intel-th-developer-manual.pdf

Bus와 subdevice 이름

System의 Intel TH device마다 독립 bus를 만들고 enumeration 순서를 나타내는 id를 부여한다. `intel_th` bus의 모든 subdevice 이름은 이 id로 시작하며, 예를 들어 `0-gth`, `0-msc0`, `0-msc1`, `0-pti`, `0-sth`처럼 device 이름과 선택적 index가 뒤따른다.

Output device에는 `/dev/intel_thN` 아래 device node도 생긴다. `N`은 Intel TH device id이며, 예를 들어 id 0인 MSU memory buffer는 `/dev/intel_th0/msc{0,1}`에서 접근한다.

Bus and Subdevices
------------------

For each Intel TH device in the system a bus of its own is
created and assigned an id number that reflects the order in which TH
devices were enumerated. All TH subdevices (devices on intel_th bus)
begin with this id: 0-gth, 0-msc0, 0-msc1, 0-pti, 0-sth, which is
followed by device's name and an optional index.

Output devices also get a device node in /dev/intel_thN, where N is
the Intel TH device id. For example, MSU's memory buffers, when
allocated, are accessible via /dev/intel_th0/msc{0,1}.

MSU trace 수집 예제

빠른 예제는 첫 MSU의 GTH port를 확인하고 master 33을 그 port로 route한다. 두 window에 각각 64 page를 할당한 multiblock buffer를 만들고 wrapping과 trace 출력을 활성화한다. Data가 쌓인 뒤 출력을 멈추고 device node에서 STP trace를 수집한다.

# figure out which GTH port is the first memory controller::

	$ cat /sys/bus/intel_th/devices/0-msc0/port
	0

# looks like it's port 0, configure master 33 to send data to port 0::

	$ echo 0 > /sys/bus/intel_th/devices/0-gth/masters/33

# allocate a 2-windowed multiblock buffer on the first memory
# controller, each with 64 pages::

	$ echo multi > /sys/bus/intel_th/devices/0-msc0/mode
	$ echo 64,64 > /sys/bus/intel_th/devices/0-msc0/nr_pages

# enable wrapping for this controller, too::

	$ echo 1 > /sys/bus/intel_th/devices/0-msc0/wrap

# and enable tracing into this port::

	$ echo 1 > /sys/bus/intel_th/devices/0-msc0/active

# .. send data to master 33, see stm.txt for more details ..
# .. wait for traces to pile up ..
# .. and stop the trace::

	$ echo 0 > /sys/bus/intel_th/devices/0-msc0/active

# and now you can collect the trace from the device node::

	$ cat /dev/intel_th0/msc0 > my_stp_trace

Quick example
-------------

# figure out which GTH port is the first memory controller::

	$ cat /sys/bus/intel_th/devices/0-msc0/port
	0

# looks like it's port 0, configure master 33 to send data to port 0::

	$ echo 0 > /sys/bus/intel_th/devices/0-gth/masters/33

# allocate a 2-windowed multiblock buffer on the first memory
# controller, each with 64 pages::

	$ echo multi > /sys/bus/intel_th/devices/0-msc0/mode
	$ echo 64,64 > /sys/bus/intel_th/devices/0-msc0/nr_pages

# enable wrapping for this controller, too::

	$ echo 1 > /sys/bus/intel_th/devices/0-msc0/wrap

# and enable tracing into this port::

	$ echo 1 > /sys/bus/intel_th/devices/0-msc0/active

# .. send data to master 33, see stm.txt for more details ..
# .. wait for traces to pile up ..
# .. and stop the trace::

	$ echo 0 > /sys/bus/intel_th/devices/0-msc0/active

# and now you can collect the trace from the device node::

	$ cat /dev/intel_th0/msc0 > my_stp_trace

Host debugger mode

Hardware debug interface로 연결한 remote debug host에서 Trace Hub를 설정하고 capture를 제어할 수 있다. 같은 interface가 Intel Trace Hub 제어와 debug host로의 trace data 전송을 모두 담당한다.

이 배치를 driver에 알려 capture와 port configuration을 건드리지 않게 해야 debug host의 설정 접근과 충돌하지 않는다. 이 mode에서 driver가 하는 일은 stm class device인 STH에 software trace를 모으는 것뿐이다. 수신 decoder가 인식할 master/channel mapping은 사용자가 설정해야 한다.

Host mode는 `intel_th` kernel module의 `host_mode` parameter를 `y`로 설정해 켠다. 그러면 virtual output device가 `intel_th` bus에 나타나지 않고 GTH의 trace 설정·capture 제어 attribute group도 노출되지 않는다. STH는 평소처럼 동작한다.

Host Debugger Mode
------------------

It is possible to configure the Trace Hub and control its trace
capture from a remote debug host, which should be connected via one of
the hardware debugging interfaces, which will then be used to both
control Intel Trace Hub and transfer its trace data to the debug host.

The driver needs to be told that such an arrangement is taking place
so that it does not touch any capture/port configuration and avoids
conflicting with the debug host's configuration accesses. The only
activity that the driver will perform in this mode is collecting
software traces to the Software Trace Hub (an stm class device). The
user is still responsible for setting up adequate master/channel
mappings that the decoder on the receiving end would recognize.

In order to enable the host mode, set the 'host_mode' parameter of the
'intel_th' kernel module to 'y'. None of the virtual output devices
will show up on the intel_th bus. Also, trace configuration and
capture controlling attribute groups of the 'gth' device will not be
exposed. The 'sth' device will operate as usual.

Software sink

MSU driver는 다른 driver가 trace data의 software sink로 등록할 수 있는 in-kernel API를 제공한다. Sink driver는 USB device controller나 network card 같은 다른 device로 data를 내보낼 수 있다.

API는 window가 가득 찼음을 sink에 알리고 그 window를 lock하는 부분과, window의 locked state를 추적하며 sink가 unlock 완료를 MSU에 알리는 부분으로 나뉜다. Full window가 lock되면 MSU는 다음 unlocked window로 자동 전환하고, 사용할 window가 없으면 trace capture를 멈춘다.

예제 `msu-sink` driver는 full window를 즉시 unlock해 MSU를 circular buffer처럼 계속 실행한다. `multi` mode와 달리 첫 window만이 아니라 buffer의 모든 window를 채운다. `msu-sink.ko`가 load된 상태에서 `mode` 파일에 `sink`를 쓰면 활성화된다.

Software Sinks
--------------

The Memory Storage Unit (MSU) driver provides an in-kernel API for
drivers to register themselves as software sinks for the trace data.
Such drivers can further export the data via other devices, such as
USB device controllers or network cards.

The API has two main parts::
 - notifying the software sink that a particular window is full, and
   "locking" that window, that is, making it unavailable for the trace
   collection; when this happens, the MSU driver will automatically
   switch to the next window in the buffer if it is unlocked, or stop
   the trace capture if it's not;
 - tracking the "locked" state of windows and providing a way for the
   software sink driver to notify the MSU driver when a window is
   unlocked and can be used again to collect trace data.

An example sink driver, msu-sink illustrates the implementation of a
software sink. Functionally, it simply unlocks windows as soon as they
are full, keeping the MSU running in a circular buffer mode. Unlike the
"multi" mode, it will fill out all the windows in the buffer as opposed
to just the first one. It can be enabled by writing "sink" to the "mode"
file (assuming msu-sink.ko is loaded).