요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
.. include:: <isonum.txt>
===============================================
``intel_pstate`` CPU Performance Scaling Driver
===============================================
:Copyright: |copy| 2017 Intel Corporation
:Author: Rafael J. Wysocki <[email protected]>
General Information
===================
``intel_pstate`` is a part of the
:doc:`CPU performance scaling subsystem <cpufreq>` in the Linux kernel
(``CPUFreq``). It is a scaling driver for the Sandy Bridge and later
generations of Intel processors. Note, however, that some of those processors
may not be supported. [To understand ``intel_pstate`` it is necessary to know
how ``CPUFreq`` works in general, so this is the time to read
Documentation/admin-guide/pm/cpufreq.rst if you have not done that yet.]
For the processors supported by ``intel_pstate``, the P-state concept is broader
than just an operating frequency or an operating performance point (see the
LinuxCon Europe 2015 presentation by Kristen Accardi [1]_ for more
information about that). For this reason, the representation of P-states used
by ``intel_pstate`` internally follows the hardware specification (for details
refer to Intel Software Developer’s Manual [2]_). However, the ``CPUFreq`` core
uses frequencies for identifying operating performance points of CPUs and
frequencies are involved in the user space interface exposed by it, so
``intel_pstate`` maps its internal representation of P-states to frequencies too
(fortunately, that mapping is unambiguous). At the same time, it would not be
practical for ``intel_pstate`` to supply the ``CPUFreq`` core with a table of
available frequencies due to the possible size of it, so the driver does not do
that. Some functionality of the core is limited by that.
Since the hardware P-state selection interface used by ``intel_pstate`` is
available at the logical CPU level, the driver always works with individual
CPUs. Consequently, if ``intel_pstate`` is in use, every ``CPUFreq`` policy
object corresponds to one logical CPU and ``CPUFreq`` policies are effectively
equivalent to CPUs. In particular, this means that they become "inactive" every
time the corresponding CPU is taken offline and need to be re-initialized when
it goes back online.
``intel_pstate`` is not modular, so it cannot be unloaded, which means that the
only way to pass early-configuration-time parameters to it is via the kernel
command line. However, its configuration can be adjusted via ``sysfs`` to a
great extent. In some configurations it even is possible to unregister it via
``sysfs`` which allows another ``CPUFreq`` scaling driver to be loaded and
registered (see `below <status_attr_>`_).
Operation Modes
===============
``intel_pstate`` can operate in two different modes, active or passive. In the
active mode, it uses its own internal performance scaling governor algorithm or
allows the hardware to do performance scaling by itself, while in the passive
mode it responds to requests made by a generic ``CPUFreq`` governor implementing
a certain performance scaling algorithm. Which of them will be in effect
depends on what kernel command line options are used and on the capabilities of
the processor.
Active Mode
-----------
This is the default operation mode of ``intel_pstate`` for processors with
hardware-managed P-states (HWP) support. If it works in this mode, the
``scaling_driver`` policy attribute in ``sysfs`` for all ``CPUFreq`` policies
contains the string "intel_pstate".
In this mode the driver bypasses the scaling governors layer of ``CPUFreq`` and
provides its own scaling algorithms for P-state selection. Those algorithms
can be applied to ``CPUFreq`` policies in the same way as generic scaling
governors (that is, through the ``scaling_governor`` policy attribute in
``sysfs``). [Note that different P-state selection algorithms may be chosen for
different policies, but that is not recommended.]
They are not generic scaling governors, but their names are the same as the
names of some of those governors. Moreover, confusingly enough, they generally
do not work in the same way as the generic governors they share the names with.
For example, the ``powersave`` P-state selection algorithm provided by
``intel_pstate`` is not a counterpart of the generic ``powersave`` governor
(roughly, it corresponds to the ``schedutil`` and ``ondemand`` governors).
There are two P-state selection algorithms provided by ``intel_pstate`` in the
active mode: ``powersave`` and ``performance``. The way they both operate
depends on whether or not the hardware-managed P-states (HWP) feature has been
enabled in the processor and possibly on the processor model.
Which of the P-state selection algorithms is used by default depends on the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option.
Namely, if that option is set, the ``performance`` algorithm will be used by
default, and the other one will be used by default if it is not set.
Active Mode With HWP
~~~~~~~~~~~~~~~~~~~~
If the processor supports the HWP feature, it will be enabled during the
processor initialization and cannot be disabled after that. It is possible
to avoid enabling it by passing the ``intel_pstate=no_hwp`` argument to the
kernel in the command line.
If the HWP feature has been enabled, ``intel_pstate`` relies on the processor to
select P-states by itself, but still it can give hints to the processor's
internal P-state selection logic. What those hints are depends on which P-state
selection algorithm has been applied to the given policy (or to the CPU it
corresponds to).
Even though the P-state selection is carried out by the processor automatically,
``intel_pstate`` registers utilization update callbacks with the CPU scheduler
in this mode. However, they are not used for running a P-state selection
algorithm, but for periodic updates of the current CPU frequency information to
be made available from the ``scaling_cur_freq`` policy attribute in ``sysfs``.
HWP + ``performance``
.....................
In this configuration ``intel_pstate`` will write 0 to the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob (otherwise), which means that the processor's
internal P-state selection logic is expected to focus entirely on performance.
This will override the EPP/EPB setting coming from the ``sysfs`` interface
(see `Energy vs Performance Hints`_ below). Moreover, any attempts to change
the EPP/EPB to a value different from 0 ("performance") via ``sysfs`` in this
configuration will be rejected.
Also, in this configuration the range of P-states available to the processor's
internal P-state selection logic is always restricted to the upper boundary
(that is, the maximum P-state that the driver is allowed to use).
HWP + ``powersave``
...................
In this configuration ``intel_pstate`` will set the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob (otherwise) to whatever value it was
previously set to via ``sysfs`` (or whatever default value it was
set to by the platform firmware). This usually causes the processor's
internal P-state selection logic to be less performance-focused.
Active Mode Without HWP
~~~~~~~~~~~~~~~~~~~~~~~
This operation mode is optional for processors that do not support the HWP
feature or when the ``intel_pstate=no_hwp`` argument is passed to the kernel in
the command line. The active mode is used in those cases if the
``intel_pstate=active`` argument is passed to the kernel in the command line.
In this mode ``intel_pstate`` may refuse to work with processors that are not
recognized by it. [Note that ``intel_pstate`` will never refuse to work with
any processor with the HWP feature enabled.]
In this mode ``intel_pstate`` registers utilization update callbacks with the
CPU scheduler in order to run a P-state selection algorithm, either
``powersave`` or ``performance``, depending on the ``scaling_governor`` policy
setting in ``sysfs``. The current CPU frequency information to be made
available from the ``scaling_cur_freq`` policy attribute in ``sysfs`` is
periodically updated by those utilization update callbacks too.
``performance``
...............
Without HWP, this P-state selection algorithm is always the same regardless of
the processor model and platform configuration.
It selects the maximum P-state it is allowed to use, subject to limits set via
``sysfs``, every time the driver configuration for the given CPU is updated
(e.g. via ``sysfs``).
This is the default P-state selection algorithm if the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option
is set.
``powersave``
.............
Without HWP, this P-state selection algorithm is similar to the algorithm
implemented by the generic ``schedutil`` scaling governor except that the
utilization metric used by it is based on numbers coming from feedback
registers of the CPU. It generally selects P-states proportional to the
current CPU utilization.
This algorithm is run by the driver's utilization update callback for the
given CPU when it is invoked by the CPU scheduler, but not more often than
every 10 ms. Like in the ``performance`` case, the hardware configuration
is not touched if the new P-state turns out to be the same as the current
one.
This is the default P-state selection algorithm if the
:c:macro:`CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE` kernel configuration option
is not set.
Passive Mode
------------
This is the default operation mode of ``intel_pstate`` for processors without
hardware-managed P-states (HWP) support. It is always used if the
``intel_pstate=passive`` argument is passed to the kernel in the command line
regardless of whether or not the given processor supports HWP. [Note that the
``intel_pstate=no_hwp`` setting causes the driver to start in the passive mode
if it is not combined with ``intel_pstate=active``.] Like in the active mode
without HWP support, in this mode ``intel_pstate`` may refuse to work with
processors that are not recognized by it if HWP is prevented from being enabled
through the kernel command line.
If the driver works in this mode, the ``scaling_driver`` policy attribute in
``sysfs`` for all ``CPUFreq`` policies contains the string "intel_cpufreq".
Then, the driver behaves like a regular ``CPUFreq`` scaling driver. That is,
it is invoked by generic scaling governors when necessary to talk to the
hardware in order to change the P-state of a CPU (in particular, the
``schedutil`` governor can invoke it directly from scheduler context).
While in this mode, ``intel_pstate`` can be used with all of the (generic)
scaling governors listed by the ``scaling_available_governors`` policy attribute
in ``sysfs`` (and the P-state selection algorithms described above are not
used). Then, it is responsible for the configuration of policy objects
corresponding to CPUs and provides the ``CPUFreq`` core (and the scaling
governors attached to the policy objects) with accurate information on the
maximum and minimum operating frequencies supported by the hardware (including
the so-called "turbo" frequency ranges). In other words, in the passive mode
the entire range of available P-states is exposed by ``intel_pstate`` to the
``CPUFreq`` core. However, in this mode the driver does not register
utilization update callbacks with the CPU scheduler and the ``scaling_cur_freq``
information comes from the ``CPUFreq`` core (and is the last frequency selected
by the current scaling governor for the given policy).
.. _turbo:
Turbo P-states Support
======================
In the majority of cases, the entire range of P-states available to
``intel_pstate`` can be divided into two sub-ranges that correspond to
different types of processor behavior, above and below a boundary that
will be referred to as the "turbo threshold" in what follows.
The P-states above the turbo threshold are referred to as "turbo P-states" and
the whole sub-range of P-states they belong to is referred to as the "turbo
range". These names are related to the Turbo Boost technology allowing a
multicore processor to opportunistically increase the P-state of one or more
cores if there is enough power to do that and if that is not going to cause the
thermal envelope of the processor package to be exceeded.
Specifically, if software sets the P-state of a CPU core within the turbo range
(that is, above the turbo threshold), the processor is permitted to take over
performance scaling control for that core and put it into turbo P-states of its
choice going forward. However, that permission is interpreted differently by
different processor generations. Namely, the Sandy Bridge generation of
processors will never use any P-states above the last one set by software for
the given core, even if it is within the turbo range, whereas all of the later
processor generations will take it as a license to use any P-states from the
turbo range, even above the one set by software. In other words, on those
processors setting any P-state from the turbo range will enable the processor
to put the given core into all turbo P-states up to and including the maximum
supported one as it sees fit.
One important property of turbo P-states is that they are not sustainable. More
precisely, there is no guarantee that any CPUs will be able to stay in any of
those states indefinitely, because the power distribution within the processor
package may change over time or the thermal envelope it was designed for might
be exceeded if a turbo P-state was used for too long.
In turn, the P-states below the turbo threshold generally are sustainable. In
fact, if one of them is set by software, the processor is not expected to change
it to a lower one unless in a thermal stress or a power limit violation
situation (a higher P-state may still be used if it is set for another CPU in
the same package at the same time, for example).
Some processors allow multiple cores to be in turbo P-states at the same time,
but the maximum P-state that can be set for them generally depends on the number
of cores running concurrently. The maximum turbo P-state that can be set for 3
cores at the same time usually is lower than the analogous maximum P-state for
2 cores, which in turn usually is lower than the maximum turbo P-state that can
be set for 1 core. The one-core maximum turbo P-state is thus the maximum
supported one overall.
The maximum supported turbo P-state, the turbo threshold (the maximum supported
non-turbo P-state) and the minimum supported P-state are specific to the
processor model and can be determined by reading the processor's model-specific
registers (MSRs). Moreover, some processors support the Configurable TDP
(Thermal Design Power) feature and, when that feature is enabled, the turbo
threshold effectively becomes a configurable value that can be set by the
platform firmware.
Unlike ``_PSS`` objects in the ACPI tables, ``intel_pstate`` always exposes
the entire range of available P-states, including the whole turbo range, to the
``CPUFreq`` core and (in the passive mode) to generic scaling governors. This
generally causes turbo P-states to be set more often when ``intel_pstate`` is
used relative to ACPI-based CPU performance scaling (see `below <acpi-cpufreq_>`_
for more information).
Moreover, since ``intel_pstate`` always knows what the real turbo threshold is
(even if the Configurable TDP feature is enabled in the processor), its
``no_turbo`` attribute in ``sysfs`` (described `below <no_turbo_attr_>`_) should
work as expected in all cases (that is, if set to disable turbo P-states, it
always should prevent ``intel_pstate`` from using them).
Processor Support
=================
To handle a given processor ``intel_pstate`` requires a number of different
pieces of information on it to be known, including:
* The minimum supported P-state.
* The maximum supported `non-turbo P-state <turbo_>`_.
* Whether or not turbo P-states are supported at all.
* The maximum supported `one-core turbo P-state <turbo_>`_ (if turbo P-states
are supported).
* The scaling formula to translate the driver's internal representation
of P-states into frequencies and the other way around.
Generally, ways to obtain that information are specific to the processor model
or family. Although it often is possible to obtain all of it from the processor
itself (using model-specific registers), there are cases in which hardware
manuals need to be consulted to get to it too.
For this reason, there is a list of supported processors in ``intel_pstate`` and
the driver initialization will fail if the detected processor is not in that
list, unless it supports the HWP feature. [The interface to obtain all of the
information listed above is the same for all of the processors supporting the
HWP feature, which is why ``intel_pstate`` works with all of them.]
Support for Hybrid Processors
=============================
Some processors supported by ``intel_pstate`` contain two or more types of CPU
cores differing by the maximum turbo P-state, performance vs power characteristics,
cache sizes, and possibly other properties. They are commonly referred to as
hybrid processors. To support them, ``intel_pstate`` requires HWP to be enabled
and it assumes the HWP performance units to be the same for all CPUs in the
system, so a given HWP performance level always represents approximately the
same physical performance regardless of the core (CPU) type.
Hybrid Processors with SMT
--------------------------
On systems where SMT (Simultaneous Multithreading), also referred to as
HyperThreading (HT) in the context of Intel processors, is enabled on at least
one core, ``intel_pstate`` assigns performance-based priorities to CPUs. Namely,
the priority of a given CPU reflects its highest HWP performance level which
causes the CPU scheduler to generally prefer more performant CPUs, so the less
performant CPUs are used when the other ones are fully loaded. SMT siblings
(that is, logical CPUs sharing one physical core) are given the same priority.
The scheduler can pull tasks from lower-priority cores and place them on any
sibling. Since the scheduler spreads tasks among physical cores, tasks will be
placed on the SMT siblings of physical cores only after all physical cores are
busy.
This approach maximizes performance in the majority of cases, but unfortunately
it also leads to excessive energy usage in some important scenarios, like video
playback, which is not generally desirable. While there is no other viable
choice with SMT enabled because the effective capacity and utilization of SMT
siblings are hard to determine, hybrid processors without SMT can be handled in
more energy-efficient ways.
.. _CAS:
Capacity-Aware Scheduling Support
---------------------------------
The capacity-aware scheduling (CAS) support in the CPU scheduler is enabled by
``intel_pstate`` by default on hybrid processors without SMT. CAS generally
causes the scheduler to put tasks on a CPU so long as there is a sufficient
amount of spare capacity on it, and if the utilization of a given task is too
high for it, the task will need to go somewhere else.
Since CAS takes CPU capacities into account, it does not require CPU
prioritization and it allows tasks to be distributed more symmetrically among
the more performant and less performant CPUs. Once placed on a CPU with enough
capacity to accommodate it, a task may just continue to run there regardless of
whether or not the other CPUs are fully loaded, so on average CAS reduces the
utilization of the more performant CPUs which causes the energy usage to be more
balanced because the more performant CPUs are generally less energy-efficient
than the less performant ones.
In order to use CAS, the scheduler needs to know the capacity of each CPU in
the system and it needs to be able to compute scale-invariant utilization of
CPUs, so ``intel_pstate`` provides it with the requisite information.
First of all, the capacity of each CPU is represented by the ratio of its highest
HWP performance level, multiplied by 1024, to the highest HWP performance level
of the most performant CPU in the system, which works because the HWP performance
units are the same for all CPUs. Second, the frequency-invariance computations,
carried out by the scheduler to always express CPU utilization in the same units
regardless of the frequency it is currently running at, are adjusted to take the
CPU capacity into account. All of this happens when ``intel_pstate`` has
registered itself with the ``CPUFreq`` core and it has figured out that it is
running on a hybrid processor without SMT.
Energy-Aware Scheduling Support
-------------------------------
If ``CONFIG_ENERGY_MODEL`` has been set during kernel configuration and
``intel_pstate`` runs on a hybrid processor without SMT, in addition to enabling
`CAS <CAS_>`_ it registers an Energy Model for the processor. This allows the
Energy-Aware Scheduling (EAS) support to be enabled in the CPU scheduler if
``schedutil`` is used as the ``CPUFreq`` governor which requires ``intel_pstate``
to operate in the `passive mode <Passive Mode_>`_.
The Energy Model registered by ``intel_pstate`` is artificial (that is, it is
based on abstract cost values and it does not include any real power numbers)
and it is relatively simple to avoid unnecessary computations in the scheduler.
There is a performance domain in it for every CPU in the system and the cost
values for these performance domains have been chosen so that running a task on
a less performant (small) CPU appears to be always cheaper than running that
task on a more performant (big) CPU. However, for two CPUs of the same type,
the cost difference depends on their current utilization, and the CPU whose
current utilization is higher generally appears to be a more expensive
destination for a given task. This helps to balance the load among CPUs of the
same type.
Since EAS works on top of CAS, high-utilization tasks are always migrated to
CPUs with enough capacity to accommodate them, but thanks to EAS, low-utilization
tasks tend to be placed on the CPUs that look less expensive to the scheduler.
Effectively, this causes the less performant and less loaded CPUs to be
preferred as long as they have enough spare capacity to run the given task
which generally leads to reduced energy usage.
The Energy Model created by ``intel_pstate`` can be inspected by looking at
the ``energy_model`` directory in ``debugfs`` (typlically mounted on
``/sys/kernel/debug/``).
User Space Interface in ``sysfs``
=================================
Global Attributes
-----------------
``intel_pstate`` exposes several global attributes (files) in ``sysfs`` to
control its functionality at the system level. They are located in the
``/sys/devices/system/cpu/intel_pstate/`` directory and affect all CPUs.
Some of them are not present if the ``intel_pstate=per_cpu_perf_limits``
argument is passed to the kernel in the command line.
``max_perf_pct``
Maximum P-state the driver is allowed to set in percent of the
maximum supported performance level (the highest supported `turbo
P-state <turbo_>`_).
This attribute will not be exposed if the
``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel
command line.
``min_perf_pct``
Minimum P-state the driver is allowed to set in percent of the
maximum supported performance level (the highest supported `turbo
P-state <turbo_>`_).
This attribute will not be exposed if the
``intel_pstate=per_cpu_perf_limits`` argument is present in the kernel
command line.
``num_pstates``
Number of P-states supported by the processor (between 0 and 255
inclusive) including both turbo and non-turbo P-states (see
`Turbo P-states Support`_).
This attribute is present only if the value exposed by it is the same
for all of the CPUs in the system.
The value of this attribute is not affected by the ``no_turbo``
setting described `below <no_turbo_attr_>`_.
This attribute is read-only.
``turbo_pct``
Ratio of the `turbo range <turbo_>`_ size to the size of the entire
range of supported P-states, in percent.
This attribute is present only if the value exposed by it is the same
for all of the CPUs in the system.
This attribute is read-only.
.. _no_turbo_attr:
``no_turbo``
If set (equal to 1), the driver is not allowed to set any turbo P-states
(see `Turbo P-states Support`_). If unset (equal to 0, which is the
default), turbo P-states can be set by the driver.
[Note that ``intel_pstate`` does not support the general ``boost``
attribute (supported by some other scaling drivers) which is replaced
by this one.]
This attribute does not affect the maximum supported frequency value
supplied to the ``CPUFreq`` core and exposed via the policy interface,
but it affects the maximum possible value of per-policy P-state limits
(see `Interpretation of Policy Attributes`_ below for details).
``hwp_dynamic_boost``
This attribute is only present if ``intel_pstate`` works in the
`active mode with the HWP feature enabled <Active Mode With HWP_>`_ in
the processor. If set (equal to 1), it causes the minimum P-state limit
to be increased dynamically for a short time whenever a task previously
waiting on I/O is selected to run on a given logical CPU (the purpose
of this mechanism is to improve performance).
This setting has no effect on logical CPUs whose minimum P-state limit
is directly set to the highest non-turbo P-state or above it.
.. _status_attr:
``status``
Operation mode of the driver: "active", "passive" or "off".
"active"
The driver is functional and in the `active mode
<Active Mode_>`_.
"passive"
The driver is functional and in the `passive mode
<Passive Mode_>`_.
"off"
The driver is not functional (it is not registered as a scaling
driver with the ``CPUFreq`` core).
This attribute can be written to in order to change the driver's
operation mode or to unregister it. The string written to it must be
one of the possible values of it and, if successful, the write will
cause the driver to switch over to the operation mode represented by
that string - or to be unregistered in the "off" case. [Actually,
switching over from the active mode to the passive mode or the other
way around causes the driver to be unregistered and registered again
with a different set of callbacks, so all of its settings (the global
as well as the per-policy ones) are then reset to their default
values, possibly depending on the target operation mode.]
``energy_efficiency``
This attribute is only present on platforms with CPUs matching the Kaby
Lake or Coffee Lake desktop CPU model. By default, energy-efficiency
optimizations are disabled on these CPU models if HWP is enabled.
Enabling energy-efficiency optimizations may limit maximum operating
frequency with or without the HWP feature. With HWP enabled, the
optimizations are done only in the turbo frequency range. Without it,
they are done in the entire available frequency range. Setting this
attribute to "1" enables the energy-efficiency optimizations and setting
to "0" disables them.
Interpretation of Policy Attributes
-----------------------------------
The interpretation of some ``CPUFreq`` policy attributes described in
Documentation/admin-guide/pm/cpufreq.rst is special with ``intel_pstate``
as the current scaling driver and it generally depends on the driver's
`operation mode <Operation Modes_>`_.
First of all, the values of the ``cpuinfo_max_freq``, ``cpuinfo_min_freq`` and
``scaling_cur_freq`` attributes are produced by applying a processor-specific
multiplier to the internal P-state representation used by ``intel_pstate``.
Also, the values of the ``scaling_max_freq`` and ``scaling_min_freq``
attributes are capped by the frequency corresponding to the maximum P-state that
the driver is allowed to set.
If the ``no_turbo`` `global attribute <no_turbo_attr_>`_ is set, the driver is
not allowed to use turbo P-states, so the maximum value of ``scaling_max_freq``
and ``scaling_min_freq`` is limited to the maximum non-turbo P-state frequency.
Accordingly, setting ``no_turbo`` causes ``scaling_max_freq`` and
``scaling_min_freq`` to go down to that value if they were above it before.
However, the old values of ``scaling_max_freq`` and ``scaling_min_freq`` will be
restored after unsetting ``no_turbo``, unless these attributes have been written
to after ``no_turbo`` was set.
If ``no_turbo`` is not set, the maximum possible value of ``scaling_max_freq``
and ``scaling_min_freq`` corresponds to the maximum supported turbo P-state,
which also is the value of ``cpuinfo_max_freq`` in either case.
Next, the following policy attributes have special meaning if
``intel_pstate`` works in the `active mode <Active Mode_>`_:
``scaling_available_governors``
List of P-state selection algorithms provided by ``intel_pstate``.
``scaling_governor``
P-state selection algorithm provided by ``intel_pstate`` currently in
use with the given policy.
``scaling_cur_freq``
Frequency of the average P-state of the CPU represented by the given
policy for the time interval between the last two invocations of the
driver's utilization update callback by the CPU scheduler for that CPU.
One more policy attribute is present if the HWP feature is enabled in the
processor:
``base_frequency``
Shows the base frequency of the CPU. Any frequency above this will be
in the turbo frequency range.
The meaning of these attributes in the `passive mode <Passive Mode_>`_ is the
same as for other scaling drivers.
Additionally, the value of the ``scaling_driver`` attribute for ``intel_pstate``
depends on the operation mode of the driver. Namely, it is either
"intel_pstate" (in the `active mode <Active Mode_>`_) or "intel_cpufreq" (in the
`passive mode <Passive Mode_>`_).
Coordination of P-State Limits
------------------------------
``intel_pstate`` allows P-state limits to be set in two ways: with the help of
the ``max_perf_pct`` and ``min_perf_pct`` `global attributes
<Global Attributes_>`_ or via the ``scaling_max_freq`` and ``scaling_min_freq``
``CPUFreq`` policy attributes. The coordination between those limits is based
on the following rules, regardless of the current operation mode of the driver:
1. All CPUs are affected by the global limits (that is, none of them can be
requested to run faster than the global maximum and none of them can be
requested to run slower than the global minimum).
2. Each individual CPU is affected by its own per-policy limits (that is, it
cannot be requested to run faster than its own per-policy maximum and it
cannot be requested to run slower than its own per-policy minimum). The
effective performance depends on whether the platform supports per core
P-states, hyper-threading is enabled and on current performance requests
from other CPUs. When platform doesn't support per core P-states, the
effective performance can be more than the policy limits set on a CPU, if
other CPUs are requesting higher performance at that moment. Even with per
core P-states support, when hyper-threading is enabled, if the sibling CPU
is requesting higher performance, the other siblings will get higher
performance than their policy limits.
3. The global and per-policy limits can be set independently.
In the `active mode with the HWP feature enabled <Active Mode With HWP_>`_, the
resulting effective values are written into hardware registers whenever the
limits change in order to request its internal P-state selection logic to always
set P-states within these limits. Otherwise, the limits are taken into account
by scaling governors (in the `passive mode <Passive Mode_>`_) and by the driver
every time before setting a new P-state for a CPU.
Additionally, if the ``intel_pstate=per_cpu_perf_limits`` command line argument
is passed to the kernel, ``max_perf_pct`` and ``min_perf_pct`` are not exposed
at all and the only way to set the limits is by using the policy attributes.
Energy vs Performance Hints
---------------------------
If the hardware-managed P-states (HWP) is enabled in the processor, additional
attributes, intended to allow user space to help ``intel_pstate`` to adjust the
processor's internal P-state selection logic by focusing it on performance or on
energy-efficiency, or somewhere between the two extremes, are present in every
``CPUFreq`` policy directory in ``sysfs``. They are :
``energy_performance_preference``
Current value of the energy vs performance hint for the given policy
(or the CPU represented by it).
The hint can be changed by writing to this attribute.
``energy_performance_available_preferences``
List of strings that can be written to the
``energy_performance_preference`` attribute.
They represent different energy vs performance hints and should be
self-explanatory, except that ``default`` represents whatever hint
value was set by the platform firmware.
Strings written to the ``energy_performance_preference`` attribute are
internally translated to integer values written to the processor's
Energy-Performance Preference (EPP) knob (if supported) or its
Energy-Performance Bias (EPB) knob. It is also possible to write a positive
integer value between 0 to 255, if the EPP feature is present. If the EPP
feature is not present, writing integer value to this attribute is not
supported. In this case, user can use the
"/sys/devices/system/cpu/cpu*/power/energy_perf_bias" interface.
[Note that tasks may by migrated from one CPU to another by the scheduler's
load-balancing algorithm and if different energy vs performance hints are
set for those CPUs, that may lead to undesirable outcomes. To avoid such
issues it is better to set the same energy vs performance hint for all CPUs
or to pin every task potentially sensitive to them to a specific CPU.]
.. _acpi-cpufreq:
``intel_pstate`` vs ``acpi-cpufreq``
====================================
On the majority of systems supported by ``intel_pstate``, the ACPI tables
provided by the platform firmware contain ``_PSS`` objects returning information
that can be used for CPU performance scaling (refer to the ACPI specification
[3]_ for details on the ``_PSS`` objects and the format of the information
returned by them).
The information returned by the ACPI ``_PSS`` objects is used by the
``acpi-cpufreq`` scaling driver. On systems supported by ``intel_pstate``
the ``acpi-cpufreq`` driver uses the same hardware CPU performance scaling
interface, but the set of P-states it can use is limited by the ``_PSS``
output.
On those systems each ``_PSS`` object returns a list of P-states supported by
the corresponding CPU which basically is a subset of the P-states range that can
be used by ``intel_pstate`` on the same system, with one exception: the whole
`turbo range <turbo_>`_ is represented by one item in it (the topmost one). By
convention, the frequency returned by ``_PSS`` for that item is greater by 1 MHz
than the frequency of the highest non-turbo P-state listed by it, but the
corresponding P-state representation (following the hardware specification)
returned for it matches the maximum supported turbo P-state (or is the
special value 255 meaning essentially "go as high as you can get").
The list of P-states returned by ``_PSS`` is reflected by the table of
available frequencies supplied by ``acpi-cpufreq`` to the ``CPUFreq`` core and
scaling governors and the minimum and maximum supported frequencies reported by
it come from that list as well. In particular, given the special representation
of the turbo range described above, this means that the maximum supported
frequency reported by ``acpi-cpufreq`` is higher by 1 MHz than the frequency
of the highest supported non-turbo P-state listed by ``_PSS`` which, of course,
affects decisions made by the scaling governors, except for ``powersave`` and
``performance``.
For example, if a given governor attempts to select a frequency proportional to
estimated CPU load and maps the load of 100% to the maximum supported frequency
(possibly multiplied by a constant), then it will tend to choose P-states below
the turbo threshold if ``acpi-cpufreq`` is used as the scaling driver, because
in that case the turbo range corresponds to a small fraction of the frequency
band it can use (1 MHz vs 1 GHz or more). In consequence, it will only go to
the turbo range for the highest loads and the other loads above 50% that might
benefit from running at turbo frequencies will be given non-turbo P-states
instead.
One more issue related to that may appear on systems supporting the
`Configurable TDP feature <turbo_>`_ allowing the platform firmware to set the
turbo threshold. Namely, if that is not coordinated with the lists of P-states
returned by ``_PSS`` properly, there may be more than one item corresponding to
a turbo P-state in those lists and there may be a problem with avoiding the
turbo range (if desirable or necessary). Usually, to avoid using turbo
P-states overall, ``acpi-cpufreq`` simply avoids using the topmost state listed
by ``_PSS``, but that is not sufficient when there are other turbo P-states in
the list returned by it.
Apart from the above, ``acpi-cpufreq`` works like ``intel_pstate`` in the
`passive mode <Passive Mode_>`_, except that the number of P-states it can set
is limited to the ones listed by the ACPI ``_PSS`` objects.
Kernel Command Line Options for ``intel_pstate``
================================================
Several kernel command line options can be used to pass early-configuration-time
parameters to ``intel_pstate`` in order to enforce specific behavior of it. All
of them have to be prepended with the ``intel_pstate=`` prefix.
``disable``
Do not register ``intel_pstate`` as the scaling driver even if the
processor is supported by it.
``active``
Register ``intel_pstate`` in the `active mode <Active Mode_>`_ to start
with.
``passive``
Register ``intel_pstate`` in the `passive mode <Passive Mode_>`_ to
start with.
``force``
Register ``intel_pstate`` as the scaling driver instead of
``acpi-cpufreq`` even if the latter is preferred on the given system.
This may prevent some platform features (such as thermal controls and
power capping) that rely on the availability of ACPI P-states
information from functioning as expected, so it should be used with
caution.
This option does not work with processors that are not supported by
``intel_pstate`` and on platforms where the ``pcc-cpufreq`` scaling
driver is used instead of ``acpi-cpufreq``.
``no_hwp``
Do not enable the hardware-managed P-states (HWP) feature even if it is
supported by the processor.
``hwp_only``
Register ``intel_pstate`` as the scaling driver only if the
hardware-managed P-states (HWP) feature is supported by the processor.
``support_acpi_ppc``
Take ACPI ``_PPC`` performance limits into account.
If the preferred power management profile in the FADT (Fixed ACPI
Description Table) is set to "Enterprise Server" or "Performance
Server", the ACPI ``_PPC`` limits are taken into account by default
and this option has no effect.
``per_cpu_perf_limits``
Use per-logical-CPU P-State limits (see `Coordination of P-state
Limits`_ for details).
``no_cas``
Do not enable `capacity-aware scheduling <CAS_>`_ which is enabled by
default on hybrid systems without SMT.
Diagnostics and Tuning
======================
Trace Events
------------
There are two static trace events that can be used for ``intel_pstate``
diagnostics. One of them is the ``cpu_frequency`` trace event generally used
by ``CPUFreq``, and the other one is the ``pstate_sample`` trace event specific
to ``intel_pstate``. Both of them are triggered by ``intel_pstate`` only if
it works in the `active mode <Active Mode_>`_.
The following sequence of shell commands can be used to enable them and see
their output (if the kernel is generally configured to support event tracing)::
# cd /sys/kernel/tracing/
# echo 1 > events/power/pstate_sample/enable
# echo 1 > events/power/cpu_frequency/enable
# cat trace
gnome-terminal--4510 [001] ..s. 1177.680733: pstate_sample: core_busy=107 scaled=94 from=26 to=26 mperf=1143818 aperf=1230607 tsc=29838618 freq=2474476
cat-5235 [002] ..s. 1177.681723: cpu_frequency: state=2900000 cpu_id=2
If ``intel_pstate`` works in the `passive mode <Passive Mode_>`_, the
``cpu_frequency`` trace event will be triggered either by the ``schedutil``
scaling governor (for the policies it is attached to), or by the ``CPUFreq``
core (for the policies with other scaling governors).
``ftrace``
----------
The ``ftrace`` interface can be used for low-level diagnostics of
``intel_pstate``. For example, to check how often the function to set a
P-state is called, the ``ftrace`` filter can be set to
:c:func:`intel_pstate_set_pstate`::
# cd /sys/kernel/tracing/
# cat available_filter_functions | grep -i pstate
intel_pstate_set_pstate
intel_pstate_cpu_init
...
# echo intel_pstate_set_pstate > set_ftrace_filter
# echo function > current_tracer
# cat trace | head -15
# tracer: function
#
# entries-in-buffer/entries-written: 80/80 #P:4
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
Xorg-3129 [000] ..s. 2537.644844: intel_pstate_set_pstate <-intel_pstate_timer_func
gnome-terminal--4510 [002] ..s. 2537.649844: intel_pstate_set_pstate <-intel_pstate_timer_func
gnome-shell-3409 [001] ..s. 2537.650850: intel_pstate_set_pstate <-intel_pstate_timer_func
<idle>-0 [000] ..s. 2537.654843: intel_pstate_set_pstate <-intel_pstate_timer_func
References
==========
.. [1] Kristen Accardi, *Balancing Power and Performance in the Linux Kernel*,
https://events.static.linuxfound.org/sites/events/files/slides/LinuxConEurope_2015.pdf
.. [2] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3: System Programming Guide*,
https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-system-programming-manual-325384.html
.. [3] *Advanced Configuration and Power Interface Specification*,
https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
일반 정보
1-53이 문서는 `SPDX-License-Identifier: GPL-2.0`을 따르며, 저작권은 2017 Intel Corporation에 있습니다. 작성자는 Rafael J. Wysocki `<[email protected]>`입니다.
`intel_pstate`는 Linux kernel의 CPU performance scaling subsystem인 `CPUFreq`의 일부입니다. Sandy Bridge 이후 세대 Intel processor를 위한 scaling driver이지만 그중 일부 processor는 지원되지 않을 수 있습니다. `intel_pstate`를 이해하려면 `CPUFreq`의 일반 동작을 알아야 하므로, 아직 읽지 않았다면 `Documentation/admin-guide/pm/cpufreq.rst`를 먼저 읽는 것이 좋습니다.
`intel_pstate`가 지원하는 processor에서 P-state 개념은 operating frequency나 operating performance point보다 넓습니다. 자세한 배경은 Kristen Accardi의 LinuxCon Europe 2015 발표 [1]을 참조하십시오. 이 때문에 `intel_pstate`가 내부에서 사용하는 P-state representation은 hardware specification을 따르며, 세부 사항은 Intel Software Developer’s Manual [2]에 있습니다. 반면 `CPUFreq` core와 user-space interface는 frequency로 operating performance point를 식별하므로 `intel_pstate`도 내부 P-state representation을 frequency에 모호함 없이 mapping합니다. 가능한 크기가 너무 커서 available frequency table을 `CPUFreq` core에 제공하는 것은 실용적이지 않으므로 driver는 그 table을 제공하지 않으며, 이에 따라 core 기능 일부가 제한됩니다.
`intel_pstate`가 사용하는 hardware P-state selection interface는 logical CPU 수준에서 제공되므로 driver는 항상 개별 CPU 단위로 동작합니다. 따라서 모든 `CPUFreq` policy object가 logical CPU 하나에 대응하고 policy는 사실상 CPU와 같습니다. 대응 CPU가 offline이 될 때마다 policy도 inactive가 되며 CPU가 online으로 돌아오면 다시 초기화해야 합니다.
`intel_pstate`는 modular driver가 아니어서 unload할 수 있으므로 초기 설정 parameter는 kernel command line으로만 전달할 수 있습니다. 다만 설정 대부분은 `sysfs`로 조정할 수 있고, 일부 configuration에서는 `sysfs`로 driver 등록을 해제하여 다른 `CPUFreq` scaling driver를 load하고 등록할 수도 있습니다.
Operation mode와 active mode
54-96`intel_pstate`는 active와 passive 두 mode로 동작합니다. Active mode에서는 자체 performance scaling governor algorithm을 사용하거나 hardware가 직접 performance scaling을 수행하게 합니다. Passive mode에서는 특정 scaling algorithm을 구현한 generic `CPUFreq` governor의 요청에 응답합니다. 실제 mode는 kernel command line option과 processor capability에 따라 정해집니다.
Active mode는 hardware-managed P-states(HWP)를 지원하는 processor의 기본 mode입니다. 이 mode에서는 모든 `CPUFreq` policy의 `sysfs` `scaling_driver` attribute가 `intel_pstate` 문자열을 담습니다.
Driver는 `CPUFreq`의 scaling governor layer를 우회하고 자체 P-state selection algorithm을 제공합니다. 이 algorithm도 generic scaling governor처럼 `sysfs`의 `scaling_governor` policy attribute를 통해 policy에 적용할 수 있습니다. Policy마다 다른 algorithm을 선택할 수 있지만 권장하지 않습니다.
이 algorithm들은 generic scaling governor가 아니지만 일부 generic governor와 같은 이름을 사용하며 동작 방식도 대체로 다릅니다. 예를 들어 `intel_pstate`의 `powersave` algorithm은 generic `powersave` governor의 대응물이 아니고, 대략 `schedutil`과 `ondemand` governor에 대응합니다.
Active mode의 P-state selection algorithm은 `powersave`와 `performance` 두 가지입니다. 둘의 동작은 processor에서 HWP가 활성화되었는지와 processor model에 따라 달라질 수 있습니다. 기본 algorithm은 `CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE`에 따라 정해져, option이 설정되면 `performance`, 아니면 `powersave`를 사용합니다.
HWP를 사용하는 active mode
97-143Processor가 HWP를 지원하면 processor 초기화 중 활성화되며 이후 비활성화할 수 없습니다. Kernel command line에 `intel_pstate=no_hwp`를 전달하면 활성화를 피할 수 있습니다.
HWP가 활성화되면 `intel_pstate`는 processor가 직접 P-state를 선택하게 하지만 내부 selection logic에는 hint를 줄 수 있습니다. Hint 내용은 해당 policy 또는 대응 CPU에 적용한 P-state selection algorithm에 따라 달라집니다.
Processor가 자동으로 P-state를 선택하더라도 `intel_pstate`는 CPU scheduler에 utilization update callback을 등록합니다. 이 callback은 P-state selection algorithm을 실행하지 않고, `sysfs`의 `scaling_cur_freq` policy attribute로 제공할 현재 CPU frequency 정보를 주기적으로 갱신합니다.
HWP와 `performance` 조합에서는 processor의 EPP(Energy-Performance Preference) knob가 있으면 0을 쓰고, 없으면 EPB(Energy-Performance Bias) knob에 0을 씁니다. 이는 processor 내부 P-state selection logic이 성능에만 집중해야 한다는 뜻입니다. 이 값은 `sysfs`에서 온 EPP/EPB 설정을 override하며, `sysfs`로 0, 즉 `performance`가 아닌 값을 설정하려는 시도는 거부됩니다. 또한 processor가 사용할 수 있는 P-state range는 driver가 허용한 maximum P-state인 upper boundary로 항상 제한됩니다.
HWP와 `powersave` 조합에서는 `intel_pstate`가 EPP 또는 EPB knob를 이전에 `sysfs`로 설정한 값, 또는 platform firmware가 설정한 기본값으로 맞춥니다. 보통 processor 내부 P-state selection logic의 성능 집중도가 낮아집니다.
HWP 없는 active mode
144-194HWP를 지원하지 않거나 kernel command line에 `intel_pstate=no_hwp`를 전달한 processor에서 이 mode는 선택 사항입니다. 이 경우 `intel_pstate=active`를 전달하면 active mode를 사용합니다. 이 mode에서는 driver가 인식하지 못하는 processor에서 동작을 거부할 수 있지만, HWP가 활성화된 processor에서는 결코 거부하지 않습니다.
`intel_pstate`는 CPU scheduler에 utilization update callback을 등록하고 `sysfs`의 `scaling_governor` 설정에 따라 `powersave` 또는 `performance` algorithm을 실행합니다. 이 callback은 `scaling_cur_freq`가 보여 줄 현재 frequency 정보도 주기적으로 갱신합니다.
HWP가 없을 때 `performance` algorithm은 processor model이나 platform configuration과 관계없이 동일합니다. CPU별 driver configuration이 `sysfs` 등으로 갱신될 때마다 `sysfs` limit 안에서 허용된 maximum P-state를 선택합니다. `CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE`가 설정되면 이것이 기본 algorithm입니다.
HWP가 없을 때 `powersave` algorithm은 generic `schedutil` governor와 비슷하지만 CPU feedback register의 수치를 기반으로 한 utilization metric을 사용합니다. 대체로 현재 CPU utilization에 비례하는 P-state를 선택합니다.
CPU scheduler가 해당 CPU의 utilization update callback을 호출할 때 이 algorithm을 실행하되 10 ms보다 자주 실행하지 않습니다. 새 P-state가 현재와 같으면 `performance`와 마찬가지로 hardware configuration을 건드리지 않습니다. `CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE`가 설정되지 않으면 이것이 기본 algorithm입니다.
Passive mode
195-229Passive mode는 HWP를 지원하지 않는 processor의 기본 `intel_pstate` mode입니다. Processor의 HWP 지원 여부와 관계없이 `intel_pstate=passive`를 전달하면 항상 이 mode를 사용합니다. `intel_pstate=no_hwp`만 설정하고 `intel_pstate=active`를 함께 지정하지 않아도 driver는 passive mode로 시작합니다. Kernel command line으로 HWP 활성화를 막은 상태에서 processor를 인식하지 못하면 active no-HWP mode처럼 동작을 거부할 수 있습니다.
이 mode에서는 모든 `CPUFreq` policy의 `scaling_driver`가 `intel_cpufreq`입니다. Driver는 일반 `CPUFreq` scaling driver처럼 generic scaling governor의 요청을 받아 CPU P-state를 바꾸며, `schedutil`은 scheduler context에서 직접 호출할 수 있습니다.
`scaling_available_governors`가 나열하는 모든 generic scaling governor와 함께 사용할 수 있고 active mode의 자체 algorithm은 사용하지 않습니다. Driver는 CPU별 policy object를 구성하고 turbo range를 포함한 hardware의 정확한 minimum·maximum operating frequency를 `CPUFreq` core와 governor에 제공합니다. 즉 전체 available P-state range를 노출합니다. CPU scheduler에 utilization update callback은 등록하지 않으며, `scaling_cur_freq`는 현재 governor가 마지막으로 선택한 frequency를 `CPUFreq` core에서 가져옵니다.
Turbo P-state 지원
230-300대부분의 경우 `intel_pstate`가 사용할 수 있는 P-state 전체 범위는 processor 동작이 달라지는 경계인 turbo threshold를 기준으로 위와 아래 두 sub-range로 나뉩니다.
Turbo threshold 위의 상태를 turbo P-state라 하고 이들이 속한 전체 sub-range를 turbo range라 합니다. 이름은 power 여유가 있고 processor package의 thermal envelope를 넘지 않을 때 multicore processor가 하나 이상의 core P-state를 기회적으로 높이는 Turbo Boost technology와 관련됩니다.
Software가 CPU core를 turbo range의 P-state로 설정하면 processor는 그 core의 performance scaling control을 넘겨받아 이후 원하는 turbo P-state를 선택할 수 있습니다. Sandy Bridge는 software가 마지막으로 설정한 상태보다 높은 P-state를 사용하지 않지만, 이후 세대는 software 설정보다 높더라도 turbo range의 어느 상태든 사용할 허가로 해석합니다. 따라서 이후 processor에서 turbo P-state 하나를 설정하면 최대 지원 상태까지 모든 turbo P-state를 processor 판단에 따라 사용할 수 있습니다.
Turbo P-state는 지속 가능하다고 보장되지 않습니다. Package 내부 power distribution은 시간에 따라 바뀔 수 있고, turbo P-state를 오래 사용하면 설계 thermal envelope를 넘을 수 있기 때문입니다.
Turbo threshold 아래 P-state는 일반적으로 지속 가능합니다. Software가 이 가운데 하나를 설정하면 thermal stress나 power limit violation이 없는 한 processor가 더 낮은 상태로 바꾸지 않을 것으로 기대합니다. 다만 같은 package의 다른 CPU에 더 높은 상태가 동시에 설정되는 등의 경우 더 높은 P-state를 사용할 수 있습니다.
일부 processor는 여러 core가 동시에 turbo P-state에 있을 수 있지만 maximum P-state는 동시 실행 core 수에 따라 달라집니다. 보통 3-core maximum은 2-core maximum보다 낮고 2-core maximum은 1-core maximum보다 낮으므로, one-core maximum turbo P-state가 전체 maximum supported 상태입니다.
Maximum turbo P-state, maximum non-turbo P-state인 turbo threshold, minimum supported P-state는 processor model별 MSR(model-specific register)에서 확인할 수 있습니다. Configurable TDP(Thermal Design Power)를 지원하고 활성화한 processor에서는 platform firmware가 turbo threshold를 설정할 수 있습니다.
ACPI `_PSS` object와 달리 `intel_pstate`는 전체 turbo range를 포함한 available P-state 범위를 `CPUFreq` core와 passive mode의 generic governor에 항상 노출합니다. 그래서 ACPI 기반 CPU performance scaling보다 turbo P-state가 더 자주 설정되는 경향이 있습니다.
`intel_pstate`는 Configurable TDP가 활성화되어도 실제 turbo threshold를 알고 있으므로 `sysfs`의 `no_turbo` attribute는 모든 경우에 기대대로 동작해야 합니다. 즉 turbo를 비활성화하면 driver가 turbo P-state를 사용하지 못하게 해야 합니다.
Processor 지원
301-331Processor를 다루려면 `intel_pstate`가 minimum supported P-state, maximum supported non-turbo P-state, turbo P-state 지원 여부, 지원한다면 maximum one-core turbo P-state, 그리고 내부 P-state representation과 frequency를 양방향으로 변환하는 scaling formula를 알아야 합니다.
이 정보를 얻는 방법은 보통 processor model이나 family별로 다릅니다. Processor의 MSR에서 모두 얻을 수 있는 경우가 많지만 hardware manual을 확인해야 하는 경우도 있습니다.
이 때문에 `intel_pstate`에는 supported processor 목록이 있으며, 발견된 processor가 목록에 없으면 HWP를 지원하지 않는 한 driver 초기화가 실패합니다. HWP 정보 interface는 모든 HWP processor에서 같으므로 `intel_pstate`는 HWP를 지원하는 모든 processor에서 동작합니다.
Hybrid processor와 SMT
332-364일부 supported processor에는 maximum turbo P-state, performance 대비 power 특성, cache 크기 등이 다른 둘 이상의 CPU core type이 있습니다. 이를 hybrid processor라 합니다. 지원하려면 HWP가 활성화되어야 하며, `intel_pstate`는 system의 모든 CPU에서 HWP performance unit이 같아서 core type과 무관하게 같은 HWP level이 대략 같은 physical performance를 나타낸다고 가정합니다.
하나 이상의 core에서 SMT(Simultaneous Multithreading), Intel 용어로 HyperThreading(HT)이 활성화된 system에서는 `intel_pstate`가 CPU에 performance-based priority를 부여합니다. Priority는 CPU의 highest HWP performance level을 반영하므로 scheduler는 더 높은 성능 CPU를 선호하고, 이들이 모두 load된 뒤 낮은 성능 CPU를 사용합니다. Physical core 하나를 공유하는 SMT siblings에는 같은 priority를 줍니다. Scheduler는 낮은 priority core에서 task를 빼 어느 sibling에든 둘 수 있으며, task를 physical core에 분산하므로 모든 physical core가 busy가 된 뒤에야 SMT sibling에 task를 배치합니다.
이 접근은 대부분의 경우 성능을 최대화하지만 video playback 같은 중요한 scenario에서 energy를 과도하게 사용할 수 있습니다. SMT sibling의 effective capacity와 utilization을 판단하기 어려워 SMT가 켜졌을 때는 다른 실용적 선택이 없지만, SMT가 없는 hybrid processor는 더 energy-efficient하게 처리할 수 있습니다.
Capacity-Aware Scheduling 지원
365-398`intel_pstate`는 SMT가 없는 hybrid processor에서 CPU scheduler의 CAS(capacity-aware scheduling)를 기본으로 활성화합니다. CAS는 spare capacity가 충분한 동안 task를 해당 CPU에 두고, task utilization이 너무 높아지면 다른 CPU로 옮깁니다.
CAS는 CPU capacity를 고려하므로 CPU prioritization이 필요 없고 높은 성능 CPU와 낮은 성능 CPU 사이에 task를 더 대칭적으로 분산할 수 있습니다. 충분한 capacity가 있는 CPU에 놓인 task는 다른 CPU가 fully loaded인지와 관계없이 계속 실행될 수 있습니다. 평균적으로 높은 성능 CPU의 utilization을 낮추며, 이런 CPU가 대체로 energy-efficient하지 않으므로 energy usage도 더 균형을 이룹니다.
CAS를 사용하려면 scheduler가 모든 CPU의 capacity를 알고 scale-invariant utilization을 계산할 수 있어야 하므로 `intel_pstate`가 필요한 정보를 제공합니다.
각 CPU capacity는 그 CPU의 highest HWP performance level에 1024를 곱한 값을 system에서 가장 높은 성능 CPU의 highest HWP level로 나눈 비율입니다. HWP unit이 모든 CPU에서 같으므로 이 방식이 성립합니다. Scheduler가 현재 frequency와 무관하게 같은 단위로 utilization을 표현하는 frequency-invariance 계산도 CPU capacity를 고려하도록 조정합니다. 이 설정은 `intel_pstate`가 `CPUFreq` core에 등록한 뒤 SMT 없는 hybrid processor임을 알아냈을 때 수행됩니다.
Energy-Aware Scheduling 지원
399-432Kernel configuration에 `CONFIG_ENERGY_MODEL`이 설정되고 `intel_pstate`가 SMT 없는 hybrid processor에서 실행되면 CAS와 함께 processor Energy Model을 등록합니다. `schedutil`을 `CPUFreq` governor로 사용할 때 CPU scheduler의 EAS(Energy-Aware Scheduling)를 활성화할 수 있으며, 이를 위해 `intel_pstate`는 passive mode로 동작해야 합니다.
`intel_pstate`가 등록하는 Energy Model은 실제 power 수치가 아닌 abstract cost value에 기반한 인공 model이며 scheduler의 불필요한 계산을 줄이도록 단순합니다. CPU마다 performance domain 하나가 있고, 낮은 성능의 small CPU에서 task를 실행하는 비용이 높은 성능의 big CPU보다 항상 싸게 보이도록 cost를 정합니다. 같은 type의 두 CPU는 현재 utilization이 높은 쪽이 더 비싼 destination으로 보이므로 같은 type 안에서 load balancing을 돕습니다.
EAS는 CAS 위에서 동작하므로 high-utilization task는 수용할 capacity가 충분한 CPU로 항상 이동합니다. Low-utilization task는 scheduler에 더 싸게 보이는 CPU에 놓이는 경향이 있어, 충분한 spare capacity가 있는 동안 낮은 성능이며 load가 적은 CPU를 선호하고 보통 energy usage를 줄입니다.
`intel_pstate`가 만든 Energy Model은 일반적으로 `/sys/kernel/debug/`에 mount되는 `debugfs`의 `energy_model` directory에서 확인할 수 있습니다.
`sysfs` global attribute
433-550`intel_pstate`는 system 수준 기능을 제어하는 global attribute를 `/sys/devices/system/cpu/intel_pstate/`에 노출하며 모든 CPU에 영향을 줍니다. `intel_pstate=per_cpu_perf_limits`를 kernel command line에 전달하면 일부 attribute는 나타나지 않습니다.
`max_perf_pct`는 driver가 설정할 수 있는 maximum P-state를 maximum supported performance level, 즉 highest supported turbo P-state의 백분율로 나타냅니다. `intel_pstate=per_cpu_perf_limits`가 있으면 노출되지 않습니다.
`min_perf_pct`는 driver가 설정할 수 있는 minimum P-state를 maximum supported performance level의 백분율로 나타냅니다. 이 attribute도 `intel_pstate=per_cpu_perf_limits`가 있으면 노출되지 않습니다.
`num_pstates`는 turbo와 non-turbo를 모두 포함하여 processor가 지원하는 P-state 개수이며 0부터 255까지입니다. 이 값이 system의 모든 CPU에서 같을 때만 나타납니다. `no_turbo` 설정의 영향을 받지 않는 read-only attribute입니다.
`turbo_pct`는 전체 supported P-state range 크기에 대한 turbo range 크기의 백분율입니다. 모든 CPU에서 값이 같을 때만 나타나는 read-only attribute입니다.
`no_turbo`가 1이면 driver가 turbo P-state를 설정할 수 없고, 기본값인 0이면 설정할 수 있습니다. `intel_pstate`는 다른 일부 driver의 일반 `boost` attribute를 지원하지 않고 `no_turbo`로 대신합니다. 이 설정은 `CPUFreq` core와 policy interface에 제공하는 maximum supported frequency 자체는 바꾸지 않지만 CPU별 P-state limit의 가능한 maximum에 영향을 줍니다.
`hwp_dynamic_boost`는 HWP가 활성화된 active mode에서만 있습니다. 1로 설정하면 이전에 I/O를 기다리던 task가 logical CPU에서 실행되도록 선택될 때 성능 개선을 위해 minimum P-state limit을 잠시 동적으로 높입니다. Minimum limit이 highest non-turbo P-state 이상으로 직접 설정된 CPU에는 영향이 없습니다.
`status`는 driver operation mode인 `active`, `passive`, `off`를 나타냅니다. `active`와 `passive`는 driver가 해당 mode로 기능 중임을, `off`는 `CPUFreq` core의 scaling driver로 등록되지 않아 기능하지 않음을 뜻합니다.
`status`에 가능한 문자열을 쓰면 mode를 바꾸거나 `off`의 경우 등록을 해제합니다. Active와 passive 사이 전환은 실제로 driver를 등록 해제한 뒤 다른 callback set으로 다시 등록하므로 global 및 policy별 설정이 target mode에 따른 기본값으로 reset됩니다.
`energy_efficiency`는 Kaby Lake 또는 Coffee Lake desktop CPU model에 맞는 platform에서만 있습니다. HWP가 켜진 이 model에서는 energy-efficiency optimization이 기본으로 꺼져 있습니다. 이를 켜면 HWP 유무와 관계없이 maximum operating frequency를 제한할 수 있습니다. HWP가 있으면 turbo range에서만, 없으면 전체 available frequency range에서 최적화합니다. `1`은 활성화하고 `0`은 비활성화합니다.
Policy attribute 해석
551-608`intel_pstate`가 현재 scaling driver일 때 `Documentation/admin-guide/pm/cpufreq.rst`에 설명된 일부 `CPUFreq` policy attribute의 해석은 특별하며 대체로 operation mode에 따라 달라집니다.
`cpuinfo_max_freq`, `cpuinfo_min_freq`, `scaling_cur_freq` 값은 `intel_pstate` 내부 P-state representation에 processor-specific multiplier를 적용해 만듭니다. `scaling_max_freq`와 `scaling_min_freq`는 driver가 설정할 수 있는 maximum P-state에 대응하는 frequency로 상한이 정해집니다.
`no_turbo`가 설정되면 turbo P-state를 사용할 수 없으므로 `scaling_max_freq`와 `scaling_min_freq`의 maximum은 maximum non-turbo P-state frequency로 제한됩니다. 기존 값이 더 높으면 이 값으로 내려갑니다. 이후 `no_turbo`를 해제하면 원래 값이 복원되지만, 설정된 동안 해당 attribute에 새 값을 썼다면 복원하지 않습니다.
`no_turbo`가 설정되지 않으면 `scaling_max_freq`와 `scaling_min_freq`의 가능한 maximum은 maximum supported turbo P-state에 대응하며, 어느 경우든 이것이 `cpuinfo_max_freq` 값입니다.
Active mode에서 `scaling_available_governors`는 `intel_pstate`가 제공하는 P-state selection algorithm 목록이고, `scaling_governor`는 현재 policy에 쓰는 algorithm입니다. `scaling_cur_freq`는 해당 CPU에서 scheduler가 driver utilization update callback을 마지막 두 번 호출한 사이의 average P-state frequency입니다. HWP가 켜지면 `base_frequency`도 있으며 이보다 높은 frequency가 turbo range입니다.
Passive mode에서 이 attribute들의 의미는 다른 scaling driver와 같습니다. `scaling_driver` 값은 active mode에서 `intel_pstate`, passive mode에서 `intel_cpufreq`입니다.
P-state limit 조정
609-647`intel_pstate`는 global attribute `max_perf_pct`·`min_perf_pct` 또는 `CPUFreq` policy attribute `scaling_max_freq`·`scaling_min_freq`로 P-state limit을 설정합니다. 다음 규칙은 mode와 무관하게 적용됩니다.
첫째, global limit은 모든 CPU에 적용되어 어느 CPU도 global maximum보다 빠르거나 global minimum보다 느리게 실행하도록 요청할 수 없습니다. 둘째, 각 CPU에는 자체 policy별 limit도 적용됩니다. Effective performance는 per-core P-state 지원, hyper-threading 활성화, 다른 CPU의 현재 요청에 따라 달라집니다. Per-core P-state를 지원하지 않으면 다른 CPU가 더 높은 성능을 요청할 때 한 CPU의 실제 성능이 policy limit을 넘을 수 있습니다. 지원하더라도 hyper-threading이 켜지고 sibling CPU가 더 높은 성능을 요청하면 다른 siblings도 자기 policy limit보다 높은 성능을 얻습니다. 셋째, global limit과 policy별 limit은 독립적으로 설정할 수 있습니다.
HWP가 활성화된 active mode에서는 limit이 바뀔 때 resulting effective value를 hardware register에 써서 내부 P-state selection logic이 항상 범위 안의 상태를 고르게 합니다. 그 밖의 경우 passive mode의 scaling governor와 driver가 새 P-state를 설정하기 전에 limit을 고려합니다.
Kernel에 `intel_pstate=per_cpu_perf_limits`를 전달하면 `max_perf_pct`와 `min_perf_pct`를 전혀 노출하지 않으며 policy attribute만으로 limit을 설정합니다.
Energy 대비 performance hint
648-685Processor에서 HWP가 활성화되면 user space가 processor 내부 P-state selection logic을 performance, energy-efficiency 또는 그 사이에 맞추도록 돕는 추가 attribute가 모든 `sysfs` `CPUFreq` policy directory에 나타납니다.
`energy_performance_preference`는 policy 또는 대응 CPU의 현재 energy 대 performance hint이며 이 attribute에 써서 바꿀 수 있습니다. `energy_performance_available_preferences`는 앞 attribute에 쓸 수 있는 문자열 목록입니다. 각 문자열은 서로 다른 hint를 나타내며, `default`는 platform firmware가 설정한 값을 뜻합니다.
`energy_performance_preference`에 쓴 문자열은 내부적으로 EPP knob가 있으면 EPP integer로, 아니면 EPB knob 값으로 변환합니다. EPP가 있으면 0부터 255 사이의 양의 integer도 쓸 수 있습니다. EPP가 없으면 integer 쓰기를 지원하지 않으며 `/sys/devices/system/cpu/cpu*/power/energy_perf_bias` interface를 사용해야 합니다.
Scheduler load balancing이 task를 CPU 사이에 migrate할 수 있으므로 CPU마다 다른 hint를 설정하면 원치 않는 결과가 생길 수 있습니다. 모든 CPU에 같은 hint를 설정하거나 hint에 민감한 task를 특정 CPU에 pin하는 편이 좋습니다.
`intel_pstate`와 `acpi-cpufreq` 비교
686-747대부분의 `intel_pstate` 지원 system에서 platform firmware의 ACPI table에는 CPU performance scaling에 쓸 정보를 반환하는 `_PSS` object가 있습니다. 자세한 format은 ACPI specification [3]을 참조하십시오.
`acpi-cpufreq` scaling driver는 `_PSS` 정보를 사용합니다. 같은 hardware CPU performance scaling interface를 쓰지만 사용할 수 있는 P-state set은 `_PSS` output으로 제한됩니다.
각 `_PSS`는 대응 CPU의 supported P-state 목록을 반환하며, 이는 대체로 `intel_pstate`가 사용할 수 있는 range의 subset입니다. 예외적으로 turbo range 전체를 topmost item 하나로 나타냅니다. 관례상 이 item의 frequency는 highest non-turbo P-state보다 1 MHz 높지만 hardware specification을 따르는 P-state representation은 maximum turbo P-state와 일치하거나 사실상 가능한 만큼 높이라는 special value 255입니다.
`acpi-cpufreq`가 `CPUFreq` core와 governor에 주는 available frequency table, minimum, maximum도 `_PSS` 목록에서 옵니다. Turbo range의 특별한 표현 때문에 reported maximum frequency는 highest non-turbo P-state보다 1 MHz 높으며, 이는 `powersave`와 `performance`를 제외한 governor 결정에 영향을 줍니다.
예를 들어 governor가 estimated CPU load에 비례하는 frequency를 선택하고 100% load를 maximum supported frequency에 대응시키면, `acpi-cpufreq`에서는 turbo range가 전체 band의 매우 작은 부분인 1 MHz 대 1 GHz 이상으로 보입니다. 따라서 가장 높은 load에서만 turbo range에 들어가고, turbo가 유리할 수 있는 50% 초과의 다른 load에는 non-turbo P-state를 줄 가능성이 큽니다.
Configurable TDP로 firmware가 turbo threshold를 설정할 수 있는 system에서 `_PSS` 목록과 제대로 조정되지 않으면 turbo P-state에 대응하는 item이 여러 개일 수 있어 turbo range 회피가 어려워집니다. `acpi-cpufreq`는 보통 `_PSS`의 topmost state를 피하지만 목록의 다른 item도 turbo라면 충분하지 않습니다.
그 밖에는 `acpi-cpufreq`가 passive mode의 `intel_pstate`처럼 동작하지만 설정할 수 있는 P-state 수가 ACPI `_PSS`에 나열된 상태로 제한됩니다.
`intel_pstate` kernel command line option
748-803초기 설정 때 특정 동작을 강제하는 모든 option에는 `intel_pstate=` prefix를 붙입니다.
`disable`은 processor가 지원되어도 `intel_pstate`를 scaling driver로 등록하지 않습니다. `active`와 `passive`는 각각 해당 mode로 등록하여 시작합니다.
`force`는 system이 `acpi-cpufreq`를 선호해도 대신 `intel_pstate`를 등록합니다. ACPI P-state 정보에 의존하는 thermal control이나 power capping 같은 platform 기능이 기대대로 동작하지 않을 수 있어 주의해야 합니다. `intel_pstate`가 지원하지 않는 processor와 `acpi-cpufreq` 대신 `pcc-cpufreq`를 쓰는 platform에서는 동작하지 않습니다.
`no_hwp`는 processor가 지원해도 HWP를 활성화하지 않습니다. `hwp_only`는 processor가 HWP를 지원할 때만 `intel_pstate`를 scaling driver로 등록합니다.
`support_acpi_ppc`는 ACPI `_PPC` performance limit을 고려합니다. FADT(Fixed ACPI Description Table)의 preferred power management profile이 `Enterprise Server` 또는 `Performance Server`이면 `_PPC` limit을 기본으로 고려하므로 이 option은 효과가 없습니다.
`per_cpu_perf_limits`는 logical CPU별 P-state limit을 사용합니다. `no_cas`는 SMT 없는 hybrid system에서 기본 활성화되는 capacity-aware scheduling을 활성화하지 않습니다.
진단과 trace event
804-830`intel_pstate` 진단에는 `CPUFreq`가 일반적으로 사용하는 `cpu_frequency`와 `intel_pstate` 전용 `pstate_sample` static trace event가 있습니다. 둘 다 driver가 active mode일 때만 `intel_pstate`가 trigger합니다. Kernel이 event tracing을 지원한다면 다음 shell command로 활성화하고 출력을 볼 수 있습니다.
# cd /sys/kernel/tracing/
# echo 1 > events/power/pstate_sample/enable
# echo 1 > events/power/cpu_frequency/enable
# cat trace
gnome-terminal--4510 [001] ..s. 1177.680733: pstate_sample: core_busy=107 scaled=94 from=26 to=26 mperf=1143818 aperf=1230607 tsc=29838618 freq=2474476
cat-5235 [002] ..s. 1177.681723: cpu_frequency: state=2900000 cpu_id=2Passive mode에서는 `schedutil`이 연결된 policy의 `cpu_frequency` event를 그 governor가 trigger하고, 다른 governor를 쓰는 policy에서는 `CPUFreq` core가 trigger합니다.
`ftrace` 진단
831-863`ftrace` interface는 `intel_pstate`의 low-level 진단에 사용할 수 있습니다. 예를 들어 P-state 설정 function 호출 빈도를 확인하려면 `ftrace` filter를 `intel_pstate_set_pstate`에 맞추고 다음 command와 trace output을 사용합니다.
# cd /sys/kernel/tracing/
# cat available_filter_functions | grep -i pstate
intel_pstate_set_pstate
intel_pstate_cpu_init
...
# echo intel_pstate_set_pstate > set_ftrace_filter
# echo function > current_tracer
# cat trace | head -15
# tracer: function
#
# entries-in-buffer/entries-written: 80/80 #P:4
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
Xorg-3129 [000] ..s. 2537.644844: intel_pstate_set_pstate <-intel_pstate_timer_func
gnome-terminal--4510 [002] ..s. 2537.649844: intel_pstate_set_pstate <-intel_pstate_timer_func
gnome-shell-3409 [001] ..s. 2537.650850: intel_pstate_set_pstate <-intel_pstate_timer_func
<idle>-0 [000] ..s. 2537.654843: intel_pstate_set_pstate <-intel_pstate_timer_func원문의 ASCII header가 표시하는 trace record field를 읽기 쉬운 표로 재구성했습니다.
참고문헌
864-874[1] Kristen Accardi, *Balancing Power and Performance in the Linux Kernel*: `https://events.static.linuxfound.org/sites/events/files/slides/LinuxConEurope_2015.pdf`
[2] *Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3: System Programming Guide*: `https://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-software-developer-system-programming-manual-325384.html`
[3] *Advanced Configuration and Power Interface Specification*: `https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf`
요약과 해설
intel_pstate.rst:1-874`intel_pstate`는 logical CPU별 policy를 운용하며, HWP 지원 여부와 mode에 따라 processor 자체 logic, driver algorithm, generic governor 가운데 P-state 선택 주체가 달라집니다.
운영 시에는 turbo와 global·policy별 limit의 결합, hybrid CPU의 scheduler capacity model, EPP/EPB hint가 실제 성능과 energy 사용에 함께 영향을 준다는 점을 특히 확인해야 합니다.