Neterion Xframe I/II PCI-X 10GbE Driver

드라이버 개요, 식별과 기능

이 문서는 `GPL-2.0` 라이선스를 따릅니다.

Neterion(구 S2io) Xframe I/II PCI-X 10GbE 드라이버

이 Linux 드라이버는 Xframe I PCI-X 1.0과 Xframe II PCI-X 2.0 어댑터를 지원합니다. jumbo frame, MSI/MSI-X, checksum offload, TSO, UFO 등을 IPv4와 IPv6 모두에서 제공합니다.

어댑터와 인터페이스 식별

• 어댑터를 시스템에 장착합니다.
• `s2io.ko`를 빌드해 로드합니다.
• 최근 커널 로그를 확인합니다.

# insmod s2io.ko
# dmesg | tail -40

로그는 다음과 비슷합니다.

eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
eth4: Device is on 64 bit 133MHz PCIX(M1) bus

메시지에서 Xframe I/II 유형, revision, driver version, interface name과 interrupt type(INTA, MSI, MSI-X)을 확인합니다. Xframe II는 PCI/PCI-X bus width와 frequency도 표시합니다. `ethtool -p <ethX>`를 실행하면 대응 어댑터의 LED가 여러 번 깜박입니다.

.. SPDX-License-Identifier: GPL-2.0

=========================================================
Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver
=========================================================

Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.

.. Contents
  - 1.  Introduction
  - 2.  Identifying the adapter/interface
  - 3.  Features supported
  - 4.  Command line parameters
  - 5.  Performance suggestions
  - 6.  Available Downloads


1. Introduction
===============
This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
Xframe II PCI-X 2.0 adapters. It supports several features
such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
See below for complete list of features.

All features are supported for both IPv4 and IPv6.

2. Identifying the adapter/interface
====================================

a. Insert the adapter(s) in your system.
b. Build and load driver::

        # insmod s2io.ko

c. View log messages::

        # dmesg | tail -40

You will see messages similar to::

        eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
        eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
        eth4: Device is on 64 bit 133MHz PCIX(M1) bus

The above messages identify the adapter type(Xframe I/II), adapter revision,
driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
as well.

To associate an interface with a physical adapter use "ethtool -p <ethX>".
The corresponding adapter's LED will blink multiple times.

3. Features supported
=====================
a. Jumbo frames. Xframe I/II supports MTU up to 9600 bytes,
   modifiable using ip command.

b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
   and receive, TSO.

c. Multi-buffer receive mode. Scattering of packet across multiple
   buffers. Currently driver supports 2-buffer mode which yields
   significant performance improvement on certain platforms(SGI Altix,
   IBM xSeries).

d. MSI/MSI-X. Can be enabled on platforms which support this feature
   resulting in noticeable performance improvement (up to 7% on certain
   platforms).

e. Statistics. Comprehensive MAC-level and software statistics displayed
   using "ethtool -S" option.

f. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
   with multiple steering options.

명령줄 매개변수

명령줄 매개변수

4. Command line parameters
==========================

a. tx_fifo_num
        Number of transmit queues

Valid range: 1-8

Default: 1

b. rx_ring_num
        Number of receive rings

Valid range: 1-8

Default: 1

c. tx_fifo_len
        Size of each transmit queue

Valid range: Total length of all queues should not exceed 8192

Default: 4096

d. rx_ring_sz
        Size of each receive ring(in 4K blocks)

Valid range: Limited by memory on system

Default: 30

e. intr_type
        Specifies interrupt type. Possible values 0(INTA), 2(MSI-X)

Valid values: 0, 2

Default: 2

성능 권장 사항과 지원

성능 권장 사항

일반적으로 MTU를 최대로 설정합니다. switch 구성은 9000, 직접 연결(back-to-back)은 9600을 사용하고 TCP window를 알맞게 조정합니다.

MTU 1500에서는 210 KiB가 좋은 결과를 보인 예가 있습니다.

# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"

MTU 9000에서는 TCP window 10 MB를 권장합니다.

# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"

송신 성능

• 기본적으로 PCI bus parameter의 BIOS 설정을 따릅니다. 필요하면 `setpci`로 max-split-transactions(MOST)와 MMRBC를 실험할 수 있습니다.
• 관측된 최적 MOST 값은 Opteron에서 2, Itanium에서 3이지만 하드웨어마다 다를 수 있습니다.
• MMRBC는 4K로 설정합니다.
• TX checksum offload가 켜져 있는지 ethtool로 확인합니다.
• `ethtool -K`로 TSO를 켭니다.

#setpci -d 17d5:* 62=1d
#setpci -d 17d5:* 62=3d
# ethtool -K <ethX> tso on

수신 성능

• 필요하면 PCI latency timer를 248(`f8`)로 설정합니다.
• 일부 SGI Altix와 IBM xSeries 플랫폼에서 큰 향상을 주는 2-buffer mode를 사용합니다.
• `ethtool -K ethX`로 RX checksum offload가 켜졌는지 확인합니다.
• 커널 구성의 S2IO 10GbE Xframe NIC에서 NAPI를 활성화해 CPU 사용량을 낮춥니다.

#setpci -d 17d5:* LATENCY_TIMER=f8

AMD Opteron과 8131 chipset 조합에서는 안전한 값으로 MMRBC=1, MOST=1을 권장합니다. 자세한 내용은 원문의 AMD8131 errata 링크를 참고하십시오.

추가 지원은 IBM, HP, SGI 등 사용 중인 10GbE Xframe NIC 공급업체에 문의하십시오.

5. Performance suggestions
==========================

General:

a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
b. Set TCP windows size to optimal value.

For instance, for MTU=1500 a value of 210K has been observed to result in
good performance::

        # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
        # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"

For MTU=9000, TCP window size of 10 MB is recommended::

        # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
        # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"

Transmit performance:

a. By default, the driver respects BIOS settings for PCI bus parameters.
   However, you may want to experiment with PCI bus parameters
   max-split-transactions(MOST) and MMRBC (use setpci command).

   A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.

   It could be different for your hardware.

   Set MMRBC to 4K**.

   For example you can set

   For opteron::

        #setpci -d 17d5:* 62=1d

   For Itanium::

        #setpci -d 17d5:* 62=3d

   For detailed description of the PCI registers, please see Xframe User Guide.

b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this
   parameter.

c. Turn on TSO(using "ethtool -K")::

        # ethtool -K <ethX> tso on

Receive performance:

a. By default, the driver respects BIOS settings for PCI bus parameters.
   However, you may want to set PCI latency timer to 248::

        #setpci -d 17d5:* LATENCY_TIMER=f8

   For detailed description of the PCI registers, please see Xframe User Guide.

b. Use 2-buffer mode. This results in large performance boost on
   certain platforms(eg. SGI Altix, IBM xSeries).

c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to
   set/verify this option.

d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network
   device support --->  Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to
   bring down CPU utilization.

.. note::

   For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are
   recommended as safe parameters.

For more information, please review the AMD8131 errata at
http://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/
26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf

6. Support
==========

For further support please contact either your 10GbE Xframe NIC vendor (IBM,
HP, SGI etc.)