linux-imx

        NO_HZ: Reducing Scheduling-Clock Ticks

​

​

This document describes Kconfig options and boot parameters that can

reduce the number of scheduling-clock interrupts, thereby improving energy

efficiency and reducing OS jitter.  Reducing OS jitter is important for

some types of computationally intensive high-performance computing (HPC)

applications and for real-time applications.

​

There are three main ways of managing scheduling-clock interrupts

(also known as "scheduling-clock ticks" or simply "ticks"):

​

1.  Never omit scheduling-clock ticks (CONFIG_HZ_PERIODIC=y or

    CONFIG_NO_HZ=n for older kernels).  You normally will -not-

    want to choose this option.

​

2.  Omit scheduling-clock ticks on idle CPUs (CONFIG_NO_HZ_IDLE=y or

    CONFIG_NO_HZ=y for older kernels).  This is the most common

    approach, and should be the default.

​

3.  Omit scheduling-clock ticks on CPUs that are either idle or that

    have only one runnable task (CONFIG_NO_HZ_FULL=y).  Unless you

    are running realtime applications or certain types of HPC

    workloads, you will normally -not- want this option.

​

These three cases are described in the following three sections, followed

by a third section on RCU-specific considerations, a fourth section

discussing testing, and a fifth and final section listing known issues.

​

​

NEVER OMIT SCHEDULING-CLOCK TICKS

​

Very old versions of Linux from the 1990s and the very early 2000s

are incapable of omitting scheduling-clock ticks.  It turns out that

there are some situations where this old-school approach is still the

right approach, for example, in heavy workloads with lots of tasks

that use short bursts of CPU, where there are very frequent idle

periods, but where these idle periods are also quite short (tens or

hundreds of microseconds).  For these types of workloads, scheduling

clock interrupts will normally be delivered any way because there

will frequently be multiple runnable tasks per CPU.  In these cases,

attempting to turn off the scheduling clock interrupt will have no effect

other than increasing the overhead of switching to and from idle and

transitioning between user and kernel execution.

​

This mode of operation can be selected using CONFIG_HZ_PERIODIC=y (or

CONFIG_NO_HZ=n for older kernels).

​

However, if you are instead running a light workload with long idle

periods, failing to omit scheduling-clock interrupts will result in

excessive power consumption.  This is especially bad on battery-powered

devices, where it results in extremely short battery lifetimes.  If you

are running light workloads, you should therefore read the following

section.

​

In addition, if you are running either a real-time workload or an HPC

workload with short iterations, the scheduling-clock interrupts can

degrade your applications performance.  If this describes your workload,

you should read the following two sections.

​

​

OMIT SCHEDULING-CLOCK TICKS FOR IDLE CPUs

​

If a CPU is idle, there is little point in sending it a scheduling-clock

interrupt.  After all, the primary purpose of a scheduling-clock interrupt

is to force a busy CPU to shift its attention among multiple duties,

and an idle CPU has no duties to shift its attention among.

​

The CONFIG_NO_HZ_IDLE=y Kconfig option causes the kernel to avoid sending

scheduling-clock interrupts to idle CPUs, which is critically important

both to battery-powered devices and to highly virtualized mainframes.

A battery-powered device running a CONFIG_HZ_PERIODIC=y kernel would

drain its battery very quickly, easily 2-3 times as fast as would the

same device running a CONFIG_NO_HZ_IDLE=y kernel.  A mainframe running

1,500 OS instances might find that half of its CPU time was consumed by

unnecessary scheduling-clock interrupts.  In these situations, there

is strong motivation to avoid sending scheduling-clock interrupts to

idle CPUs.  That said, dyntick-idle mode is not free:

​

1.  It increases the number of instructions executed on the path

    to and from the idle loop.

​

2.  On many architectures, dyntick-idle mode also increases the

    number of expensive clock-reprogramming operations.

​

Therefore, systems with aggressive real-time response constraints often

run CONFIG_HZ_PERIODIC=y kernels (or CONFIG_NO_HZ=n for older kernels)

in order to avoid degrading from-idle transition latencies.

​

An idle CPU that is not receiving scheduling-clock interrupts is said to

be "dyntick-idle", "in dyntick-idle mode", "in nohz mode", or "running

tickless".  The remainder of this document will use "dyntick-idle mode".