Documentation/driver-api/rapidio/rapidio.rst GitHub 원문 ↗

Linux 6.18.37 · Driver API

Linux RapidIO Subsystem

RapidIO core object, subsystem initialization, enumeration·discovery 역할과 basic scanner routing algorithm을 다루는 전문 번역입니다.

Source pathDocumentation/driver-api/rapidio/rapidio.rst
Source versionLinux v6.18.37
TranslationDUJINLABS 전문 번역 + 해설

요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.

1. 요약·해설

원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.

요약과 해설

rapidio.rst:1-362

Linux RapidIO subsystem은 mport, device, switch, network와 subsystem interface를 Linux device model에 통합합니다. Host는 destination ID와 lock을 사용해 network를 depth-first enumerate하고 route를 구성하며, agent는 host 완료 후 routing을 바꾸지 않고 topology를 discover합니다. Built-in·module 및 kernel·userspace 시작 방식을 모두 지원합니다.

문서 구성
원문 줄내용
1-97Subsystem 개요와 core object
98-112Mport 등록과 초기화
113-238Enumeration·discovery 시작 방식과 role
239-306Basic DFS enumeration과 agent discovery
307-348새 scanner와 switch module dependency
349-362References

2. 영어 원문 전체

번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.

원문 전체 펼치기
1 ============
2 Introduction
3 ============
4
5 The RapidIO standard is a packet-based fabric interconnect standard designed for
6 use in embedded systems. Development of the RapidIO standard is directed by the
7 RapidIO Trade Association (RTA). The current version of the RapidIO specification
8 is publicly available for download from the RTA web-site [1].
9
10 This document describes the basics of the Linux RapidIO subsystem and provides
11 information on its major components.
12
13 1 Overview
14 ==========
15
16 Because the RapidIO subsystem follows the Linux device model it is integrated
17 into the kernel similarly to other buses by defining RapidIO-specific device and
18 bus types and registering them within the device model.
19
20 The Linux RapidIO subsystem is architecture independent and therefore defines
21 architecture-specific interfaces that provide support for common RapidIO
22 subsystem operations.
23
24 2. Core Components
25 ==================
26
27 A typical RapidIO network is a combination of endpoints and switches.
28 Each of these components is represented in the subsystem by an associated data
29 structure. The core logical components of the RapidIO subsystem are defined
30 in include/linux/rio.h file.
31
32 2.1 Master Port
33 ---------------
34
35 A master port (or mport) is a RapidIO interface controller that is local to the
36 processor executing the Linux code. A master port generates and receives RapidIO
37 packets (transactions). In the RapidIO subsystem each master port is represented
38 by a rio_mport data structure. This structure contains master port specific
39 resources such as mailboxes and doorbells. The rio_mport also includes a unique
40 host device ID that is valid when a master port is configured as an enumerating
41 host.
42
43 RapidIO master ports are serviced by subsystem specific mport device drivers
44 that provide functionality defined for this subsystem. To provide a hardware
45 independent interface for RapidIO subsystem operations, rio_mport structure
46 includes rio_ops data structure which contains pointers to hardware specific
47 implementations of RapidIO functions.
48
49 2.2 Device
50 ----------
51
52 A RapidIO device is any endpoint (other than mport) or switch in the network.
53 All devices are presented in the RapidIO subsystem by corresponding rio_dev data
54 structure. Devices form one global device list and per-network device lists
55 (depending on number of available mports and networks).
56
57 2.3 Switch
58 ----------
59
60 A RapidIO switch is a special class of device that routes packets between its
61 ports towards their final destination. The packet destination port within a
62 switch is defined by an internal routing table. A switch is presented in the
63 RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
64 data structure, which contains switch specific information such as copy of the
65 routing table and pointers to switch specific functions.
66
67 The RapidIO subsystem defines the format and initialization method for subsystem
68 specific switch drivers that are designed to provide hardware-specific
69 implementation of common switch management routines.
70
71 2.4 Network
72 -----------
73
74 A RapidIO network is a combination of interconnected endpoint and switch devices.
75 Each RapidIO network known to the system is represented by corresponding rio_net
76 data structure. This structure includes lists of all devices and local master
77 ports that form the same network. It also contains a pointer to the default
78 master port that is used to communicate with devices within the network.
79
80 2.5 Device Drivers
81 ------------------
82
83 RapidIO device-specific drivers follow Linux Kernel Driver Model and are
84 intended to support specific RapidIO devices attached to the RapidIO network.
85
86 2.6 Subsystem Interfaces
87 ------------------------
88
89 RapidIO interconnect specification defines features that may be used to provide
90 one or more common service layers for all participating RapidIO devices. These
91 common services may act separately from device-specific drivers or be used by
92 device-specific drivers. Example of such service provider is the RIONET driver
93 which implements Ethernet-over-RapidIO interface. Because only one driver can be
94 registered for a device, all common RapidIO services have to be registered as
95 subsystem interfaces. This allows to have multiple common services attached to
96 the same device without blocking attachment of a device-specific driver.
97
98 3. Subsystem Initialization
99 ===========================
100
101 In order to initialize the RapidIO subsystem, a platform must initialize and
102 register at least one master port within the RapidIO network. To register mport
103 within the subsystem controller driver's initialization code calls function
104 rio_register_mport() for each available master port.
105
106 After all active master ports are registered with a RapidIO subsystem,
107 an enumeration and/or discovery routine may be called automatically or
108 by user-space command.
109
110 RapidIO subsystem can be configured to be built as a statically linked or
111 modular component of the kernel (see details below).
112
113 4. Enumeration and Discovery
114 ============================
115
116 4.1 Overview
117 ------------
118
119 RapidIO subsystem configuration options allow users to build enumeration and
120 discovery methods as statically linked components or loadable modules.
121 An enumeration/discovery method implementation and available input parameters
122 define how any given method can be attached to available RapidIO mports:
123 simply to all available mports OR individually to the specified mport device.
124
125 Depending on selected enumeration/discovery build configuration, there are
126 several methods to initiate an enumeration and/or discovery process:
127
128 (a) Statically linked enumeration and discovery process can be started
129 automatically during kernel initialization time using corresponding module
130 parameters. This was the original method used since introduction of RapidIO
131 subsystem. Now this method relies on enumerator module parameter which is
132 'rio-scan.scan' for existing basic enumeration/discovery method.
133 When automatic start of enumeration/discovery is used a user has to ensure
134 that all discovering endpoints are started before the enumerating endpoint
135 and are waiting for enumeration to be completed.
136 Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
137 endpoint waits for enumeration to be completed. If the specified timeout
138 expires the discovery process is terminated without obtaining RapidIO network
139 information. NOTE: a timed out discovery process may be restarted later using
140 a user-space command as it is described below (if the given endpoint was
141 enumerated successfully).
142
143 (b) Statically linked enumeration and discovery process can be started by
144 a command from user space. This initiation method provides more flexibility
145 for a system startup compared to the option (a) above. After all participating
146 endpoints have been successfully booted, an enumeration process shall be
147 started first by issuing a user-space command, after an enumeration is
148 completed a discovery process can be started on all remaining endpoints.
149
150 (c) Modular enumeration and discovery process can be started by a command from
151 user space. After an enumeration/discovery module is loaded, a network scan
152 process can be started by issuing a user-space command.
153 Similar to the option (b) above, an enumerator has to be started first.
154
155 (d) Modular enumeration and discovery process can be started by a module
156 initialization routine. In this case an enumerating module shall be loaded
157 first.
158
159 When a network scan process is started it calls an enumeration or discovery
160 routine depending on the configured role of a master port: host or agent.
161
162 Enumeration is performed by a master port if it is configured as a host port by
163 assigning a host destination ID greater than or equal to zero. The host
164 destination ID can be assigned to a master port using various methods depending
165 on RapidIO subsystem build configuration:
166
167 (a) For a statically linked RapidIO subsystem core use command line parameter
168 "rapidio.hdid=" with a list of destination ID assignments in order of mport
169 device registration. For example, in a system with two RapidIO controllers
170 the command line parameter "rapidio.hdid=-1,7" will result in assignment of
171 the host destination ID=7 to the second RapidIO controller, while the first
172 one will be assigned destination ID=-1.
173
174 (b) If the RapidIO subsystem core is built as a loadable module, in addition
175 to the method shown above, the host destination ID(s) can be specified using
176 traditional methods of passing module parameter "hdid=" during its loading:
177
178 - from command line: "modprobe rapidio hdid=-1,7", or
179 - from modprobe configuration file using configuration command "options",
180 like in this example: "options rapidio hdid=-1,7". An example of modprobe
181 configuration file is provided in the section below.
182
183 NOTES:
184 (i) if "hdid=" parameter is omitted all available mport will be assigned
185 destination ID = -1;
186
187 (ii) the "hdid=" parameter in systems with multiple mports can have
188 destination ID assignments omitted from the end of list (default = -1).
189
190 If the host device ID for a specific master port is set to -1, the discovery
191 process will be performed for it.
192
193 The enumeration and discovery routines use RapidIO maintenance transactions
194 to access the configuration space of devices.
195
196 NOTE: If RapidIO switch-specific device drivers are built as loadable modules
197 they must be loaded before enumeration/discovery process starts.
198 This requirement is cased by the fact that enumeration/discovery methods invoke
199 vendor-specific callbacks on early stages.
200
201 4.2 Automatic Start of Enumeration and Discovery
202 ------------------------------------------------
203
204 Automatic enumeration/discovery start method is applicable only to built-in
205 enumeration/discovery RapidIO configuration selection. To enable automatic
206 enumeration/discovery start by existing basic enumerator method set use boot
207 command line parameter "rio-scan.scan=1".
208
209 This configuration requires synchronized start of all RapidIO endpoints that
210 form a network which will be enumerated/discovered. Discovering endpoints have
211 to be started before an enumeration starts to ensure that all RapidIO
212 controllers have been initialized and are ready to be discovered. Configuration
213 parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
214 a discovering endpoint will wait for enumeration to be completed.
215
216 When automatic enumeration/discovery start is selected, basic method's
217 initialization routine calls rio_init_mports() to perform enumeration or
218 discovery for all known mport devices.
219
220 Depending on RapidIO network size and configuration this automatic
221 enumeration/discovery start method may be difficult to use due to the
222 requirement for synchronized start of all endpoints.
223
224 4.3 User-space Start of Enumeration and Discovery
225 -------------------------------------------------
226
227 User-space start of enumeration and discovery can be used with built-in and
228 modular build configurations. For user-space controlled start RapidIO subsystem
229 creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
230 an enumeration or discovery process on specific mport device, a user needs to
231 write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
232 sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
233 registration. For example for machine with single RapidIO controller, mport_ID
234 for that controller always will be 0.
235
236 To initiate RapidIO enumeration/discovery on all available mports a user may
237 write '-1' (or RIO_MPORT_ANY) into the scan attribute file.
238
239 4.4 Basic Enumeration Method
240 ----------------------------
241
242 This is an original enumeration/discovery method which is available since
243 first release of RapidIO subsystem code. The enumeration process is
244 implemented according to the enumeration algorithm outlined in the RapidIO
245 Interconnect Specification: Annex I [1].
246
247 This method can be configured as statically linked or loadable module.
248 The method's single parameter "scan" allows to trigger the enumeration/discovery
249 process from module initialization routine.
250
251 This enumeration/discovery method can be started only once and does not support
252 unloading if it is built as a module.
253
254 The enumeration process traverses the network using a recursive depth-first
255 algorithm. When a new device is found, the enumerator takes ownership of that
256 device by writing into the Host Device ID Lock CSR. It does this to ensure that
257 the enumerator has exclusive right to enumerate the device. If device ownership
258 is successfully acquired, the enumerator allocates a new rio_dev structure and
259 initializes it according to device capabilities.
260
261 If the device is an endpoint, a unique device ID is assigned to it and its value
262 is written into the device's Base Device ID CSR.
263
264 If the device is a switch, the enumerator allocates an additional rio_switch
265 structure to store switch specific information. Then the switch's vendor ID and
266 device ID are queried against a table of known RapidIO switches. Each switch
267 table entry contains a pointer to a switch-specific initialization routine that
268 initializes pointers to the rest of switch specific operations, and performs
269 hardware initialization if necessary. A RapidIO switch does not have a unique
270 device ID; it relies on hopcount and routing for device ID of an attached
271 endpoint if access to its configuration registers is required. If a switch (or
272 chain of switches) does not have any endpoint (except enumerator) attached to
273 it, a fake device ID will be assigned to configure a route to that switch.
274 In the case of a chain of switches without endpoint, one fake device ID is used
275 to configure a route through the entire chain and switches are differentiated by
276 their hopcount value.
277
278 For both endpoints and switches the enumerator writes a unique component tag
279 into device's Component Tag CSR. That unique value is used by the error
280 management notification mechanism to identify a device that is reporting an
281 error management event.
282
283 Enumeration beyond a switch is completed by iterating over each active egress
284 port of that switch. For each active link, a route to a default device ID
285 (0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
286 into the routing table. The algorithm recurs by calling itself with hopcount + 1
287 and the default device ID in order to access the device on the active port.
288
289 After the host has completed enumeration of the entire network it releases
290 devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
291 in the system, it sets the Discovered bit in the Port General Control CSR
292 to indicate that enumeration is completed and agents are allowed to execute
293 passive discovery of the network.
294
295 The discovery process is performed by agents and is similar to the enumeration
296 process that is described above. However, the discovery process is performed
297 without changes to the existing routing because agents only gather information
298 about RapidIO network structure and are building an internal map of discovered
299 devices. This way each Linux-based component of the RapidIO subsystem has
300 a complete view of the network. The discovery process can be performed
301 simultaneously by several agents. After initializing its RapidIO master port
302 each agent waits for enumeration completion by the host for the configured wait
303 time period. If this wait time period expires before enumeration is completed,
304 an agent skips RapidIO discovery and continues with remaining kernel
305 initialization.
306
307 4.5 Adding New Enumeration/Discovery Method
308 -------------------------------------------
309
310 RapidIO subsystem code organization allows addition of new enumeration/discovery
311 methods as new configuration options without significant impact to the core
312 RapidIO code.
313
314 A new enumeration/discovery method has to be attached to one or more mport
315 devices before an enumeration/discovery process can be started. Normally,
316 method's module initialization routine calls rio_register_scan() to attach
317 an enumerator to a specified mport device (or devices). The basic enumerator
318 implementation demonstrates this process.
319
320 4.6 Using Loadable RapidIO Switch Drivers
321 -----------------------------------------
322
323 In the case when RapidIO switch drivers are built as loadable modules a user
324 must ensure that they are loaded before the enumeration/discovery starts.
325 This process can be automated by specifying pre- or post- dependencies in the
326 RapidIO-specific modprobe configuration file as shown in the example below.
327
328 File /etc/modprobe.d/rapidio.conf::
329
330 # Configure RapidIO subsystem modules
331
332 # Set enumerator host destination ID (overrides kernel command line option)
333 options rapidio hdid=-1,2
334
335 # Load RapidIO switch drivers immediately after rapidio core module was loaded
336 softdep rapidio post: idt_gen2 idtcps tsi57x
337
338 # OR :
339
340 # Load RapidIO switch drivers just before rio-scan enumerator module is loaded
341 softdep rio-scan pre: idt_gen2 idtcps tsi57x
342
343 --------------------------
344
345 NOTE:
346 In the example above, one of "softdep" commands must be removed or
347 commented out to keep required module loading sequence.
348
349 5. References
350 =============
351
352 [1] RapidIO Trade Association. RapidIO Interconnect Specifications.
353 http://www.rapidio.org.
354
355 [2] Rapidio TA. Technology Comparisons.
356 http://www.rapidio.org/education/technology_comparisons/
357
358 [3] RapidIO support for Linux.
359 https://lwn.net/Articles/139118/
360
361 [4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
362 https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf
363

3. 한국어 전문 번역

영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.

RapidIO standard와 문서 범위

1-12

RapidIO standard는 embedded system에서 사용하도록 설계된 packet 기반 fabric interconnect standard입니다. RapidIO Trade Association(RTA)이 standard 개발을 주도하며 현재 RapidIO specification은 RTA web site의 reference [1]에서 공개적으로 내려받을 수 있습니다.

이 문서는 Linux RapidIO subsystem의 기초와 주요 component를 설명합니다.

RapidIO 소개
항목내용
InterconnectPacket-based fabric
대상Embedded systems
표준 기구RapidIO Trade Association (RTA)
문서 범위Linux subsystem 기초와 주요 component

============
Introduction
============

The RapidIO standard is a packet-based fabric interconnect standard designed for
use in embedded systems. Development of the RapidIO standard is directed by the
RapidIO Trade Association (RTA). The current version of the RapidIO specification
is publicly available for download from the RTA web-site [1].

This document describes the basics of the Linux RapidIO subsystem and provides
information on its major components.

Linux device model 통합

13-23

RapidIO subsystem은 Linux device model을 따릅니다. 다른 bus와 마찬가지로 RapidIO 전용 device type과 bus type을 정의하고 device model에 등록해 kernel에 통합합니다.

Linux RapidIO subsystem 자체는 architecture independent합니다. 공통 RapidIO subsystem operation을 지원하기 위해 architecture-specific interface를 별도로 정의합니다.

Device model 통합
RapidIO device and bus typesLinux device modelRapidIO subsystem core
Architecture-specific interfaceCommon RapidIO operationsArchitecture-independent core

공통 core는 architecture independent하고 platform 구현은 architecture-specific interface를 제공합니다.

1 Overview
==========

Because the RapidIO subsystem follows the Linux device model it is integrated
into the kernel similarly to other buses by defining RapidIO-specific device and
bus types and registering them within the device model.

The Linux RapidIO subsystem is architecture independent and therefore defines
architecture-specific interfaces that provide support for common RapidIO
subsystem operations.

Core component와 `include/linux/rio.h`

24-31

일반적인 RapidIO network는 endpoint와 switch의 조합입니다. Subsystem은 각 component를 대응하는 data structure로 표현합니다.

RapidIO subsystem의 core logical component는 `include/linux/rio.h`에 정의되어 있습니다.

Network component 표현
물리 componentSubsystem 표현
EndpointRapidIO core data structure
SwitchRapidIO core와 switch 확장 structure
정의 위치`include/linux/rio.h`

2. Core Components
==================

A typical RapidIO network is a combination of endpoints and switches.
Each of these components is represented in the subsystem by an associated data
structure. The core logical components of the RapidIO subsystem are defined
in include/linux/rio.h file.

Master port와 `rio_ops`

32-48

Master port 또는 mport는 Linux code를 실행하는 processor에 local인 RapidIO interface controller입니다. RapidIO packet, 즉 transaction을 생성하고 수신합니다.

Subsystem에서 각 master port는 `rio_mport` data structure로 표현됩니다. 이 structure는 mailbox와 doorbell 같은 mport-specific resource를 담고, mport가 enumerating host로 설정되었을 때 유효한 unique host device ID도 포함합니다.

RapidIO 전용 mport device driver가 master port를 서비스합니다. Hardware-independent subsystem interface를 제공하기 위해 `rio_mport` 안의 `rio_ops` structure가 hardware-specific RapidIO function 구현 pointer를 보관합니다.

Master port 구조
Local RapidIO controller`struct rio_mport`Mailboxes / doorbells / host device ID
Subsystem operation`struct rio_ops`Hardware-specific implementation

Local controller의 resource와 operation table을 `rio_mport`가 묶습니다.

2.1 Master Port
---------------

A master port (or mport) is a RapidIO interface controller that is local to the
processor executing the Linux code. A master port generates and receives RapidIO
packets (transactions). In the RapidIO subsystem each master port is represented
by a rio_mport data structure. This structure contains master port specific
resources such as mailboxes and doorbells. The rio_mport also includes a unique
host device ID that is valid when a master port is configured as an enumerating
host.

RapidIO master ports are serviced by subsystem specific mport device drivers
that provide functionality defined for this subsystem. To provide a hardware
independent interface for RapidIO subsystem operations, rio_mport structure
includes rio_ops data structure which contains pointers to hardware specific
implementations of RapidIO functions.

RapidIO device와 device list

49-56

RapidIO device는 mport를 제외한 network endpoint 또는 switch입니다. 모든 device는 대응하는 `rio_dev` data structure로 subsystem에 나타납니다.

Device는 하나의 global device list와 network별 device list를 구성합니다. Network별 list 수는 사용 가능한 mport와 network 수에 따라 달라집니다.

`rio_dev` list
범위구성
Global모든 RapidIO device
Per-network해당 network의 endpoint와 switch
제외Local mport

2.2 Device
----------

A RapidIO device is any endpoint (other than mport) or switch in the network.
All devices are presented in the RapidIO subsystem by corresponding rio_dev data
structure. Devices form one global device list and per-network device lists
(depending on number of available mports and networks).

Switch routing과 `rio_switch`

57-70

RapidIO switch는 packet을 port 사이에서 최종 destination 방향으로 routing하는 특수 device입니다. Switch 내부 routing table이 packet의 destination port를 결정합니다.

Subsystem은 switch를 `rio_dev`로 표현하고 `rio_switch` data structure를 추가해 확장합니다. `rio_switch`에는 routing table 사본과 switch-specific function pointer 같은 정보가 들어 있습니다.

RapidIO subsystem은 공통 switch management routine의 hardware-specific 구현을 제공하는 switch driver의 형식과 초기화 방법을 정의합니다.

Switch 표현과 routing
Incoming packetInternal routing tableDestination egress port
`struct rio_dev``struct rio_switch`Route copy and switch callbacks

Generic device 위에 switch-specific state와 operation을 확장합니다.

2.3 Switch
----------

A RapidIO switch is a special class of device that routes packets between its
ports towards their final destination. The packet destination port within a
switch is defined by an internal routing table. A switch is presented in the
RapidIO subsystem by rio_dev data structure expanded by additional rio_switch
data structure, which contains switch specific information such as copy of the
routing table and pointers to switch specific functions.

The RapidIO subsystem defines the format and initialization method for subsystem
specific switch drivers that are designed to provide hardware-specific
implementation of common switch management routines.

Network, device driver, subsystem interface

71-97

RapidIO network는 서로 연결된 endpoint와 switch device의 조합이며 `rio_net` data structure로 표현됩니다. 이 structure는 같은 network를 구성하는 모든 device와 local master port list를 담고, network 내 device와 통신할 때 사용할 default master port pointer도 가집니다.

RapidIO device-specific driver는 Linux Kernel Driver Model을 따르며 network에 연결된 특정 RapidIO device를 지원합니다.

RapidIO specification에는 모든 참여 device에 공통 service layer를 제공할 수 있는 기능이 있습니다. 이러한 service는 device-specific driver와 독립적으로 동작하거나 그 driver가 사용할 수 있습니다. 예로 RIONET driver는 Ethernet-over-RapidIO interface를 구현합니다.

한 device에는 driver 하나만 등록할 수 있으므로 공통 RapidIO service는 subsystem interface로 등록해야 합니다. 그러면 device-specific driver attachment를 막지 않으면서 같은 device에 여러 공통 service를 붙일 수 있습니다.

Core 역할 구분
구성자료 구조·형태역할
Network`struct rio_net`Device, mport, default mport 집합
Device driverLinux driver model특정 RapidIO device 지원
RIONETSubsystem interfaceEthernet-over-RapidIO
Common serviceSubsystem interface한 device에 여러 service 공존

2.4 Network
-----------

A RapidIO network is a combination of interconnected endpoint and switch devices.
Each RapidIO network known to the system is represented by corresponding rio_net
data structure. This structure includes lists of all devices and local master
ports that form the same network. It also contains a pointer to the default
master port that is used to communicate with devices within the network.

2.5 Device Drivers
------------------

RapidIO device-specific drivers follow Linux Kernel Driver Model and are
intended to support specific RapidIO devices attached to the RapidIO network.

2.6 Subsystem Interfaces
------------------------

RapidIO interconnect specification defines features that may be used to provide
one or more common service layers for all participating RapidIO devices. These
common services may act separately from device-specific drivers or be used by
device-specific drivers. Example of such service provider is the RIONET driver
which implements Ethernet-over-RapidIO interface. Because only one driver can be
registered for a device, all common RapidIO services have to be registered as
subsystem interfaces. This allows to have multiple common services attached to
the same device without blocking attachment of a device-specific driver.

Subsystem 초기화와 mport 등록

98-112

Platform이 RapidIO subsystem을 초기화하려면 RapidIO network 안의 master port를 최소 하나 초기화하고 등록해야 합니다. Controller driver initialization code는 사용 가능한 각 mport에 대해 `rio_register_mport()`를 호출합니다.

모든 active master port를 subsystem에 등록한 뒤 enumeration 또는 discovery routine을 자동으로 호출하거나 userspace command로 시작할 수 있습니다.

RapidIO subsystem은 kernel에 statically linked component 또는 modular component로 build할 수 있습니다.

Subsystem 초기화
Initialize local controller`rio_register_mport()` per mportAll active mports registeredEnumeration / discovery
RapidIO subsystemBuilt-in or module

Platform mport 등록이 끝난 후 network enumeration 또는 discovery를 시작합니다.

3. Subsystem Initialization
===========================

In order to initialize the RapidIO subsystem, a platform must initialize and
register at least one master port within the RapidIO network. To register mport
within the subsystem controller driver's initialization code calls function
rio_register_mport() for each available master port.

After all active master ports are registered with a RapidIO subsystem,
an enumeration and/or discovery routine may be called automatically or
by user-space command.

RapidIO subsystem can be configured to be built as a statically linked or
modular component of the kernel (see details below).

Enumeration·discovery 시작 방식

113-158

Configuration option에 따라 enumeration과 discovery method는 static component 또는 loadable module로 build할 수 있습니다. Method 구현과 input parameter는 모든 mport에 붙일지 특정 mport에만 붙일지를 결정합니다.

방식 (a)는 statically linked method를 kernel initialization 중 module parameter로 자동 시작합니다. Basic method는 `rio-scan.scan` parameter를 사용합니다. 모든 discovering endpoint가 enumerating endpoint보다 먼저 시작되어 완료를 기다려야 합니다.

`CONFIG_RAPIDIO_DISC_TIMEOUT`은 discovering endpoint가 enumeration 완료를 기다리는 시간을 정의합니다. Timeout이면 network 정보를 얻지 못한 채 discovery를 종료하지만, endpoint가 성공적으로 enumerate되었다면 나중에 userspace command로 다시 시작할 수 있습니다.

방식 (b)는 statically linked method를 userspace command로 시작합니다. 모든 endpoint boot 후 enumeration을 먼저 시작하고 완료된 뒤 나머지 endpoint의 discovery를 시작하므로 자동 방식보다 startup을 유연하게 제어할 수 있습니다.

방식 (c)는 modular method를 load한 뒤 userspace command로 network scan을 시작합니다. 이 경우에도 enumerator가 먼저 시작되어야 합니다. 방식 (d)는 module initialization routine이 modular method를 시작하며 enumerating module을 먼저 load해야 합니다.

Enumeration·discovery 시작 방식
방식Build시작 주체순서 제약
(a)StaticKernel init parameterDiscoverer 먼저 준비
(b)StaticUserspace commandEnumerator 후 discoverer
(c)ModuleUserspace commandModule load 후 enumerator 먼저
(d)ModuleModule initEnumerating module 먼저 load

4. Enumeration and Discovery
============================

4.1 Overview
------------

RapidIO subsystem configuration options allow users to build enumeration and
discovery methods as statically linked components or loadable modules.
An enumeration/discovery method implementation and available input parameters
define how any given method can be attached to available RapidIO mports:
simply to all available mports OR individually to the specified mport device.

Depending on selected enumeration/discovery build configuration, there are
several methods to initiate an enumeration and/or discovery process:

  (a) Statically linked enumeration and discovery process can be started
  automatically during kernel initialization time using corresponding module
  parameters. This was the original method used since introduction of RapidIO
  subsystem. Now this method relies on enumerator module parameter which is
  'rio-scan.scan' for existing basic enumeration/discovery method.
  When automatic start of enumeration/discovery is used a user has to ensure
  that all discovering endpoints are started before the enumerating endpoint
  and are waiting for enumeration to be completed.
  Configuration option CONFIG_RAPIDIO_DISC_TIMEOUT defines time that discovering
  endpoint waits for enumeration to be completed. If the specified timeout
  expires the discovery process is terminated without obtaining RapidIO network
  information. NOTE: a timed out discovery process may be restarted later using
  a user-space command as it is described below (if the given endpoint was
  enumerated successfully).

  (b) Statically linked enumeration and discovery process can be started by
  a command from user space. This initiation method provides more flexibility
  for a system startup compared to the option (a) above. After all participating
  endpoints have been successfully booted, an enumeration process shall be
  started first by issuing a user-space command, after an enumeration is
  completed a discovery process can be started on all remaining endpoints.

  (c) Modular enumeration and discovery process can be started by a command from
  user space. After an enumeration/discovery module is loaded, a network scan
  process can be started by issuing a user-space command.
  Similar to the option (b) above, an enumerator has to be started first.

  (d) Modular enumeration and discovery process can be started by a module
  initialization routine. In this case an enumerating module shall be loaded
  first.

Host·agent role과 `hdid` 지정

159-181

Network scan은 master port의 role에 따라 enumeration routine 또는 discovery routine을 호출합니다. Host destination ID가 0 이상이면 해당 master port는 host port로 설정되어 enumeration을 수행합니다.

Statically linked subsystem core에서는 kernel command-line `rapidio.hdid=`에 mport 등록 순서대로 destination ID list를 지정합니다. 두 controller에 `rapidio.hdid=-1,7`을 주면 첫 controller는 -1, 두 번째는 host destination ID 7을 받습니다.

Subsystem core가 loadable module이면 위 kernel command line 외에도 module load 시 `hdid=`를 전달할 수 있습니다. 예시는 `modprobe rapidio hdid=-1,7`이며 modprobe configuration에는 `options rapidio hdid=-1,7`을 사용합니다.

Host destination ID 결정
`rapidio.hdid=` or module `hdid=`Assign IDs by mport registration orderID >= 0Host enumeration
ID = -1Agent discovery

Destination ID가 0 이상이면 host enumeration, -1이면 agent discovery 역할입니다.

When a network scan process is started it calls an enumeration or discovery
routine depending on the configured role of a master port: host or agent.

Enumeration is performed by a master port if it is configured as a host port by
assigning a host destination ID greater than or equal to zero. The host
destination ID can be assigned to a master port using various methods depending
on RapidIO subsystem build configuration:

  (a) For a statically linked RapidIO subsystem core use command line parameter
  "rapidio.hdid=" with a list of destination ID assignments in order of mport
  device registration. For example, in a system with two RapidIO controllers
  the command line parameter "rapidio.hdid=-1,7" will result in assignment of
  the host destination ID=7 to the second RapidIO controller, while the first
  one will be assigned destination ID=-1.

  (b) If the RapidIO subsystem core is built as a loadable module, in addition
  to the method shown above, the host destination ID(s) can be specified using
  traditional methods of passing module parameter "hdid=" during its loading:

  - from command line: "modprobe rapidio hdid=-1,7", or
  - from modprobe configuration file using configuration command "options",
    like in this example: "options rapidio hdid=-1,7". An example of modprobe
    configuration file is provided in the section below.

`hdid` 기본값과 switch driver 선행 조건

182-200

`hdid=` parameter를 생략하면 모든 mport의 destination ID는 -1입니다. Mport가 여러 개인 system에서는 list 끝의 assignment를 생략할 수 있고 생략된 값도 기본 -1입니다.

특정 master port의 host device ID가 -1이면 그 port는 discovery를 수행합니다. Enumeration과 discovery routine은 RapidIO maintenance transaction으로 device configuration space에 접근합니다.

RapidIO switch-specific driver를 loadable module로 build했다면 enumeration 또는 discovery 시작 전에 load해야 합니다. Enumeration/discovery method가 초기 단계부터 vendor-specific callback을 호출하기 때문입니다.

Role과 선행 조건
조건결과
`hdid` 생략 또는 trailing assignment 생략Destination ID -1
Host device ID -1Discovery 수행
Configuration accessRapidIO maintenance transaction
Switch driver moduleScan 시작 전에 load


NOTES:
  (i) if "hdid=" parameter is omitted all available mport will be assigned
  destination ID = -1;

  (ii) the "hdid=" parameter in systems with multiple mports can have
  destination ID assignments omitted from the end of list (default = -1).

If the host device ID for a specific master port is set to -1, the discovery
process will be performed for it.

The enumeration and discovery routines use RapidIO maintenance transactions
to access the configuration space of devices.

NOTE: If RapidIO switch-specific device drivers are built as loadable modules
they must be loaded before enumeration/discovery process starts.
This requirement is cased by the fact that enumeration/discovery methods invoke
vendor-specific callbacks on early stages.

Automatic enumeration·discovery

201-223

Automatic start는 built-in enumeration/discovery configuration에서만 사용할 수 있습니다. 기존 basic enumerator의 자동 시작은 boot command line에 `rio-scan.scan=1`을 설정합니다.

Network를 구성하는 모든 RapidIO endpoint의 시작을 동기화해야 합니다. 모든 controller가 초기화되어 discovery 가능한 상태가 되도록 discovering endpoint를 enumeration 시작 전에 먼저 기동해야 합니다. `CONFIG_RAPIDIO_DISC_TIMEOUT`은 완료를 기다릴 시간(초)을 지정합니다.

Automatic start를 선택하면 basic method initialization routine이 `rio_init_mports()`를 호출해 알려진 모든 mport에서 enumeration 또는 discovery를 수행합니다.

Network 크기와 구성에 따라 모든 endpoint의 synchronized start 요구사항 때문에 자동 방식은 사용하기 어려울 수 있습니다.

Automatic scan
Boot `rio-scan.scan=1`Start discovering endpointsAll controllers ready`rio_init_mports()`Enumerate / discover all mports

Discovering endpoint 준비와 controller 초기화가 모두 끝난 뒤 host scan을 시작해야 합니다.

4.2 Automatic Start of Enumeration and Discovery
------------------------------------------------

Automatic enumeration/discovery start method is applicable only to built-in
enumeration/discovery RapidIO configuration selection. To enable automatic
enumeration/discovery start by existing basic enumerator method set use boot
command line parameter "rio-scan.scan=1".

This configuration requires synchronized start of all RapidIO endpoints that
form a network which will be enumerated/discovered. Discovering endpoints have
to be started before an enumeration starts to ensure that all RapidIO
controllers have been initialized and are ready to be discovered. Configuration
parameter CONFIG_RAPIDIO_DISC_TIMEOUT defines time (in seconds) which
a discovering endpoint will wait for enumeration to be completed.

When automatic enumeration/discovery start is selected, basic method's
initialization routine calls rio_init_mports() to perform enumeration or
discovery for all known mport devices.

Depending on RapidIO network size and configuration this automatic
enumeration/discovery start method may be difficult to use due to the
requirement for synchronized start of all endpoints.

Userspace scan attribute

224-238

Userspace start는 built-in과 modular configuration 모두에서 사용할 수 있습니다. RapidIO subsystem은 write-only sysfs attribute `/sys/bus/rapidio/scan`을 만듭니다.

특정 mport에서 enumeration 또는 discovery를 시작하려면 RapidIO destination ID가 아니라 `mport_ID`를 이 file에 씁니다. `mport_ID`는 mport 등록 중 0부터 `RIO_MAX_MPORTS` 범위에서 순차적으로 배정되며 controller 하나인 system에서는 항상 0입니다.

사용 가능한 모든 mport에서 scan하려면 `-1`, 즉 `RIO_MPORT_ANY`를 씁니다.

`/sys/bus/rapidio/scan`
입력대상
특정 `mport_ID`해당 registered mport
`0`Single-controller system의 controller
`-1` / `RIO_MPORT_ANY`모든 mport

4.3 User-space Start of Enumeration and Discovery
-------------------------------------------------

User-space start of enumeration and discovery can be used with built-in and
modular build configurations. For user-space controlled start RapidIO subsystem
creates the sysfs write-only attribute file '/sys/bus/rapidio/scan'. To initiate
an enumeration or discovery process on specific mport device, a user needs to
write mport_ID (not RapidIO destination ID) into that file. The mport_ID is a
sequential number (0 ... RIO_MAX_MPORTS) assigned during mport device
registration. For example for machine with single RapidIO controller, mport_ID
for that controller always will be 0.

To initiate RapidIO enumeration/discovery on all available mports a user may
write '-1' (or RIO_MPORT_ANY) into the scan attribute file.

Basic method와 device ownership

239-263

Basic enumeration/discovery method는 RapidIO subsystem 최초 release부터 제공된 original method이며 RapidIO Interconnect Specification Annex I [1]의 algorithm을 구현합니다.

Static 또는 loadable module로 구성할 수 있고 단일 `scan` parameter가 module initialization routine에서 process를 시작합니다. 이 method는 한 번만 시작할 수 있으며 module build에서도 unloading을 지원하지 않습니다.

Enumerator는 recursive depth-first algorithm으로 network를 순회합니다. 새 device를 찾으면 Host Device ID Lock CSR에 써서 ownership을 얻고 exclusive enumeration 권한을 확보합니다.

Ownership 획득에 성공하면 새 `rio_dev`를 할당해 capability에 따라 초기화합니다. Endpoint에는 unique device ID를 배정하고 Base Device ID CSR에 기록합니다.

Basic depth-first enumeration
Recursive depth-first scanFind deviceWrite Host Device ID Lock CSRAllocate `rio_dev`Initialize capabilities
EndpointAssign unique device IDWrite Base Device ID CSR

Lock으로 device ownership을 확보한 뒤 자료 구조와 endpoint ID를 초기화합니다.

4.4 Basic Enumeration Method
----------------------------

This is an original enumeration/discovery method which is available since
first release of RapidIO subsystem code. The enumeration process is
implemented according to the enumeration algorithm outlined in the RapidIO
Interconnect Specification: Annex I [1].

This method can be configured as statically linked or loadable module.
The method's single parameter "scan" allows to trigger the enumeration/discovery
process from module initialization routine.

This enumeration/discovery method can be started only once and does not support
unloading if it is built as a module.

The enumeration process traverses the network using a recursive depth-first
algorithm. When a new device is found, the enumerator takes ownership of that
device by writing into the Host Device ID Lock CSR. It does this to ensure that
the enumerator has exclusive right to enumerate the device. If device ownership
is successfully acquired, the enumerator allocates a new rio_dev structure and
initializes it according to device capabilities.

If the device is an endpoint, a unique device ID is assigned to it and its value
is written into the device's Base Device ID CSR.

Switch initialization과 recursive routing

264-288

Device가 switch이면 enumerator가 switch-specific 정보를 저장할 `rio_switch`를 추가로 할당합니다. Vendor ID와 device ID로 known RapidIO switch table을 조회하고, table entry의 initialization routine이 나머지 switch operation pointer와 필요한 hardware initialization을 설정합니다.

RapidIO switch 자체에는 unique device ID가 없습니다. Configuration register 접근에는 연결된 endpoint의 device ID를 위한 routing과 hopcount를 사용합니다. Enumerator 외 endpoint가 없는 switch 또는 switch chain에는 route 구성을 위한 fake device ID 하나를 배정하고 hopcount로 각 switch를 구분합니다.

Endpoint와 switch 모두에 unique component tag를 Component Tag CSR에 씁니다. Error-management notification은 이 값으로 event를 보고한 device를 식별합니다.

Switch 너머를 enumerate할 때 active egress port를 순회합니다. 각 active link의 routing table에 default device ID, 즉 8-bit system은 `0xFF`, 16-bit system은 `0xFFFF` route를 임시로 쓰고 `hopcount + 1`과 default ID로 algorithm을 재귀 호출합니다.

Switch enumeration
Allocate `rio_switch`Lookup vendor/device IDRun switch init callbackInitialize route operations
No attached endpointAssign fake device IDDifferentiate by hopcount
Active egress portTemporary default route`hopcount + 1`Recursive scan

Switch 식별, route 준비, active egress별 재귀 순회가 이어집니다.

If the device is a switch, the enumerator allocates an additional rio_switch
structure to store switch specific information. Then the switch's vendor ID and
device ID are queried against a table of known RapidIO switches. Each switch
table entry contains a pointer to a switch-specific initialization routine that
initializes pointers to the rest of switch specific operations, and performs
hardware initialization if necessary. A RapidIO switch does not have a unique
device ID; it relies on hopcount and routing for device ID of an attached
endpoint if access to its configuration registers is required. If a switch (or
chain of switches) does not have any endpoint (except enumerator) attached to
it, a fake device ID will be assigned to configure a route to that switch.
In the case of a chain of switches without endpoint, one fake device ID is used
to configure a route through the entire chain and switches are differentiated by
their hopcount value.

For both endpoints and switches the enumerator writes a unique component tag
into device's Component Tag CSR. That unique value is used by the error
management notification mechanism to identify a device that is reporting an
error management event.

Enumeration beyond a switch is completed by iterating over each active egress
port of that switch. For each active link, a route to a default device ID
(0xFF for 8-bit systems and 0xFFFF for 16-bit systems) is temporarily written
into the routing table. The algorithm recurs by calling itself with hopcount + 1
and the default device ID in order to access the device on the active port.

Enumeration 완료와 agent discovery

289-306

Host가 전체 network enumeration을 완료하면 `rio_clear_locks()`를 호출해 device ID lock을 지우고 device를 release합니다. 각 endpoint의 Port General Control CSR에서 Discovered bit를 설정해 완료를 알리고 agent가 passive discovery를 수행하도록 허용합니다.

Agent discovery는 enumeration과 유사하지만 기존 routing을 바꾸지 않습니다. Agent는 network 구조를 수집하고 discovered device의 internal map만 구성합니다. 따라서 각 Linux 기반 RapidIO component가 network 전체 view를 갖게 됩니다.

여러 agent가 discovery를 동시에 수행할 수 있습니다. 각 agent는 master port 초기화 후 설정된 wait time 동안 host enumeration 완료를 기다립니다. Timeout 전에 완료되지 않으면 discovery를 건너뛰고 나머지 kernel initialization을 계속합니다.

Host 완료와 agent discovery
Host enumeration complete`rio_clear_locks()`Set Discovered bitAgents may discover
AgentWait for hostRead topology without route changesBuild internal map
Wait timeoutSkip discoveryContinue kernel initialization

Host가 lock과 Discovered bit를 정리한 뒤 agent가 routing 변경 없이 topology를 수집합니다.

After the host has completed enumeration of the entire network it releases
devices by clearing device ID locks (calls rio_clear_locks()). For each endpoint
in the system, it sets the Discovered bit in the Port General Control CSR
to indicate that enumeration is completed and agents are allowed to execute
passive discovery of the network.

The discovery process is performed by agents and is similar to the enumeration
process that is described above. However, the discovery process is performed
without changes to the existing routing because agents only gather information
about RapidIO network structure and are building an internal map of discovered
devices. This way each Linux-based component of the RapidIO subsystem has
a complete view of the network. The discovery process can be performed
simultaneously by several agents. After initializing its RapidIO master port
each agent waits for enumeration completion by the host for the configured wait
time period. If this wait time period expires before enumeration is completed,
an agent skips RapidIO discovery and continues with remaining kernel
initialization.

새 enumeration·discovery method 추가

307-319

RapidIO subsystem code organization은 core code에 큰 영향을 주지 않고 새 enumeration/discovery method를 configuration option으로 추가할 수 있게 설계되었습니다.

새 method는 process 시작 전에 하나 이상의 mport device에 attach해야 합니다. 일반적으로 method module initialization routine이 `rio_register_scan()`을 호출해 지정한 mport 또는 mport 집합에 enumerator를 붙입니다. Basic enumerator 구현이 이 절차의 예입니다.

새 scan method 등록
New method configurationModule initialization`rio_register_scan()`Attach to one or more mportsStart scan

Method module이 mport에 scanner를 붙인 뒤 enumeration 또는 discovery를 시작할 수 있습니다.

4.5 Adding New Enumeration/Discovery Method
-------------------------------------------

RapidIO subsystem code organization allows addition of new enumeration/discovery
methods as new configuration options without significant impact to the core
RapidIO code.

A new enumeration/discovery method has to be attached to one or more mport
devices before an enumeration/discovery process can be started. Normally,
method's module initialization routine calls rio_register_scan() to attach
an enumerator to a specified mport device (or devices). The basic enumerator
implementation demonstrates this process.

Loadable switch driver 순서

320-348

RapidIO switch driver를 loadable module로 build한 경우 enumeration/discovery 전에 반드시 load해야 합니다. RapidIO 전용 modprobe configuration file의 pre-dependency 또는 post-dependency로 순서를 자동화할 수 있습니다.

예제 `/etc/modprobe.d/rapidio.conf`는 `options rapidio hdid=-1,2`로 kernel command-line 설정을 override합니다.

`softdep rapidio post: idt_gen2 idtcps tsi57x`는 rapidio core module 직후 switch driver를 load합니다. 대안인 `softdep rio-scan pre: idt_gen2 idtcps tsi57x`는 rio-scan enumerator module 직전에 load합니다.

두 `softdep` command 중 하나는 제거하거나 comment 처리해 필요한 module loading sequence 하나만 유지해야 합니다.

Switch module dependency
설정효과
`options rapidio hdid=-1,2`Host destination ID override
`softdep rapidio post: ...`Core 직후 switch driver load
`softdep rio-scan pre: ...`Enumerator 직전 switch driver load
두 softdep 중 하나 선택명확한 module order 유지

4.6 Using Loadable RapidIO Switch Drivers
-----------------------------------------

In the case when RapidIO switch drivers are built as loadable modules a user
must ensure that they are loaded before the enumeration/discovery starts.
This process can be automated by specifying pre- or post- dependencies in the
RapidIO-specific modprobe configuration file as shown in the example below.

File /etc/modprobe.d/rapidio.conf::

  # Configure RapidIO subsystem modules

  # Set enumerator host destination ID (overrides kernel command line option)
  options rapidio hdid=-1,2

  # Load RapidIO switch drivers immediately after rapidio core module was loaded
  softdep rapidio post: idt_gen2 idtcps tsi57x

  # OR :

  # Load RapidIO switch drivers just before rio-scan enumerator module is loaded
  softdep rio-scan pre: idt_gen2 idtcps tsi57x

  --------------------------

NOTE:
  In the example above, one of "softdep" commands must be removed or
  commented out to keep required module loading sequence.

참고 자료

349-362

참고 자료에는 RapidIO Trade Association의 Interconnect Specifications와 Technology Comparisons, LWN의 Linux RapidIO 지원 기사, Matt Porter의 Ottawa Linux Symposium 2005 논문이 포함됩니다.

References
번호자료
[1]RapidIO Interconnect Specifications
[2]RapidIO Technology Comparisons
[3]RapidIO support for Linux, LWN
[4]Matt Porter, RapidIO for Linux, OLS 2005

5. References
=============

[1] RapidIO Trade Association. RapidIO Interconnect Specifications.
    http://www.rapidio.org.

[2] Rapidio TA. Technology Comparisons.
    http://www.rapidio.org/education/technology_comparisons/

[3] RapidIO support for Linux.
    https://lwn.net/Articles/139118/

[4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
    https://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf