Sunday, October 18, 2015

Using ‘Stress / Stress-ng’ Tools to Impose High CPU Load and Stress Test on Linux

As a System Administrator, you may want to examine and monitor the status of your Linux systems when they are under stress of high load. This can be a good way for System Administrators and Programmers to:
  1. fine tune activities on a system.
  2. monitor operating system kernel interfaces.
  3. test your Linux hardware components such as CPU, memory, disk devices and many others to observe their performance under stress.
  4. measure different power consuming loads on a system.
In this guide, we shall look at two important tools, stress and stress-ng for stress testing under your Linux systems.

1. stress – is a workload generator tool designed to subject your system to a configurable measure of CPU, memory, I/O and disk stress.

2. stress-ng – is an updated version of the stress workload generator tool which tests your system for following features:
  • CPU compute
  • drive stress
  • I/O syncs
  • Pipe I/O
  • cache thrashing
  • VM stress
  • socket stressing
  • process creation and termination
  • context switching properties
Though these tools are good for examining your system, they should not just be used by any system user.

Important: It is highly recommended that you use these tools with root user privileges, because they can stress your Linux machine so fast and to avoid certain system errors on poorly designed hardware.
How to Install ‘stress’ Tool in Linux

To install stress tool on Debian and its derivatives such Ubuntu and Mint, run the following command.
$ sudo apt-get install stress

To install stress on a RHEL/CentOS and Fedora Linux, you need to turn on EPEL repository and then type the following yum command to install the same:
# yum install stress

The general syntax for using stress is:
$ sudo stress option argument

Some options that you can use with stress.
  • To spawn N workers spinning on sqrt() function, use the –cpu N option as follows.
  • To spawn N workers spinning on sync() function, use the –io N option as follows.
  • To spawn N workers spinning on malloc()/free() functions, use the –vm N option.
  • To allocate memory per vm worker, use the –vm-bytes N option.
  • Instead of freeing and reallocating memory resources, you can redirty memory by using the –vm-keep option.
  • Set sleep to N seconds before freeing memory by using the –vm-hang N option.
  • To spawn N workers spinning on write()/unlink() functions, use the –hdd N option.
  • You can set a timeout after N seconds by using the –timeout N option.
  • Set a wait factor of N microseconds before any work starts by using the –backoff N option as follows.
  • To show more detailed information when running stress, use the -v option.
  • Use –help to view help for using stress or view the manpage.

How Do I use stress on Linux systems?


1. To examine effect of the command every time you run it, first run the uptime command and note down the load average.
Next, run the stress command to spawn 8 workers spinning on sqrt() with a timeout of 20 seconds. After running stress, again run the uptime command and compare the load average.
nilaxan@localhost ~ $ uptime
nilaxan@localhost ~ $ sudo stress --cpu  8 --timeout 20
nilaxan@localhost ~ $ uptime

Sample Output

nilaxan@localhost ~ $ uptime    
 17:20:00 up  7:51,  2 users,  load average: 1.91, 2.16, 1.93  [<-- Watch Load Average]
nilaxan@localhost ~ $ sudo stress --cpu 8 --timeout 20
stress: info: [17246] dispatching hogs: 8 cpu, 0 io, 0 vm, 0 hdd
stress: info: [17246] successful run completed in 21s
nilaxan@localhost ~ $ uptime
 17:20:24 up  7:51,  2 users,  load average: 5.14, 2.88, 2.17  [<-- Watch Load Average]

2. To spwan 8 workers spinning on sqrt() with a timeout of 30 seconds, showing detailed information about the operation, run this command:
nilaxan@localhost ~ $ uptime
nilaxan@localhost ~ $ sudo stress --cpu 8 -v --timeout 30s
nilaxan@localhost ~ $ uptime

Sample Output

nilaxan@localhost ~ $ uptime
 17:27:25 up  7:58,  2 users,  load average: 1.40, 1.90, 1.98  [<-- Watch Load Average]
nilaxan@localhost ~ $ sudo stress --cpu 8 -v --timeout 30s
stress: info: [17353] dispatching hogs: 8 cpu, 0 io, 0 vm, 0 hdd
stress: dbug: [17353] using backoff sleep of 24000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 8 [17354] forked
stress: dbug: [17353] using backoff sleep of 21000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 7 [17355] forked
stress: dbug: [17353] using backoff sleep of 18000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 6 [17356] forked
stress: dbug: [17353] using backoff sleep of 15000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 5 [17357] forked
stress: dbug: [17353] using backoff sleep of 12000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 4 [17358] forked
stress: dbug: [17353] using backoff sleep of 9000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 3 [17359] forked
stress: dbug: [17353] using backoff sleep of 6000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 2 [17360] forked
stress: dbug: [17353] using backoff sleep of 3000us
stress: dbug: [17353] setting timeout to 30s
stress: dbug: [17353] --> hogcpu worker 1 [17361] forked
stress: dbug: [17353] 
nilaxan@localhost ~ $ uptime
 17:27:59 up  7:59,  2 users,  load average: 5.41, 2.82, 2.28  [<-- Watch Load Average]

3. To spwan one worker of malloc() and free() functions with a timeout of 60 seconds, run the following command.
nilaxan@localhost ~ $ uptime
nilaxan@localhost ~ $ sudo stress --vm 1 --timeout 60s 
nilaxan@localhost ~ $ uptime

Sample Output

nilaxan@localhost ~ $ uptime
 17:34:07 up  8:05,  2 users,  load average: 1.54, 2.04, 2.11  [<-- Watch Load Average]
nilaxan@localhost ~ $ sudo stress --vm 1 --timeout 60s 
stress: info: [17420] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: info: [17420] successful run completed in 60s
nilaxan@localhost ~ $ uptime
 17:35:20 up  8:06,  2 users,  load average: 2.45, 2.24, 2.17  [<-- Watch Load Average]

4. To spwan 4 workers spinning on sqrt(), 2 workers 
spwaning on sync(), 2 workers on malloc()/free(), with a time out of 20 
seconds and allocate a memory of 256MB per vm worker, run this command 
below.
nilaxan@localhost ~ $ uptime
nilaxan@localhost ~ $ sudo stress --cpu 4 --io 3 --vm 2 --vm-bytes 256M --timeout 20s 
nilaxan@localhost ~ $ uptime

Sample Output

nilaxan@localhost ~ $ uptime
 17:40:33 up  8:12,  2 users,  load average: 1.68, 1.84, 2.02  [<-- Watch Load Average]
nilaxan@localhost ~ $ sudo stress --cpu 4 --io 3 --vm 2 --vm-bytes 256M --timeout 20s
stress: info: [17501] dispatching hogs: 4 cpu, 3 io, 2 vm, 0 hdd
stress: info: [17501] successful run completed in 20s
nilaxan@localhost ~ $ uptime
 17:40:58 up  8:12,  2 users,  load average: 4.63, 2.54, 2.24  [<-- Watch Load Average]

How to Install ‘stress-ng’ Tool in Linux

To install stress-ng, run the following command.
$ sudo apt-get install stress-ng             [on Debian based systems]
# yum install stress-ng                      [on RedHat based systems]
The general syntax for using `stress-ng` is :
$ sudo stress-ng option argument

Some of the options that you can use with stress-ng:
  • To start N instances of each stress test, use the –all N option as follows.
  • To start N processes to exercises the CPU by sequentially working through all the different CPU stress testing methods, use the –cpu N option as follows.
  • To use a given CPU stress testing method, use –cpu-method option. There are many methods available that you can use, to view the manpage to see all the methods to use.
  • To stop CPU stress process after N bogo operations, use the –cpu-ops N option.
  • To start N I/O stress testing processes, use the –io N option.
  • To stop io stress processes after N bogo operations, use the –io-ops N option.
  • To start N vm stress testing processes, use the –vm N option.
  • To specify amount of memory per vm process, use –vm-bytes N option.
  • To stop vm stress processes after N bogo operations, use –vm-ops N options
  • Use the –hdd N option to start N harddisk exercising processes.
  • To stop hdd stress processes after N bogo operations, use –hdd-ops N option.
  • You can set a timeout after N seconds by using the –timeout N option.
  • To generate a summary report after bogo operations, you can use –metrics or –metrics-brief options. The –metrics-brief displays non zero metrics.
  • You can also start N processes that will create and remove directories using mkdir and rmdir by using the –dir N option.
  • To stop directory operations processes use –dir-ops N options.
  • To start N CPU consuming processes that will exercise the present nice levels, include the
  • –nice N option. When using this option, every iteration will fork off a child process that runs through all the different nice levels running a busy loop for 0.1 seconds per level and then exits.
  • To stop nice loops, use the –nice-ops N option as follows.
  • To start N processes that change the file mode bits via chmod(2) and fchmod(2) on the same file, use the –chmod N option. Remember the greater the value for N then the more contention on the file. The stressor will work through all the combination of mode bits that you specify with chmod(2).
  • You can stop chmod operations by the –chmod-ops N option.
  • You can use the -v option to display more information about ongoing operations.
  • Use -h to view help for stress-ng.

How Do I use ‘stress-ng’ in Linux systems?

1. To run 8 CPU stressors with a timeout of 60 seconds and a summary at the end of operation 

nilaxan@localhost:~$ uptime
nilaxan@localhost:~$ sudo stress-ng --cpu 8 --timeout 60 --metrics-brief
nilaxan@localhost:~$ uptime

Sample Output

nilaxan@localhost:~$ uptime
 18:15:29 up 12 min,  1 user,  load average: 0.00, 0.01, 0.03  [<-- Watch Load Average]
nilaxan@localhost:~$ sudo stress-ng --cpu 8 --timeout 60 --metrics-brief
stress-ng: info: [1247] dispatching hogs: 8 cpu
stress-ng: info: [1247] successful run completed in 60.42s
stress-ng: info: [1247] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info: [1247]       (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info: [1247] cpu   11835     60.32     59.75      0.05       196.20       197.91
nilaxan@localhost:~$ uptime
 18:16:47 up 13 min,  1 user,  load average: 4.75, 1.47, 0.54  [<-- Watch Load Average]

2. To run 4 FFT CPU stressors with a timeout of 2 minutes.
nilaxan@localhost:~$ uptime
nilaxan@localhost:~$ sudo stress-ng --cpu 4 --cpu-method fft --timeout 2m
nilaxan@localhost:~$ uptime

Sample Output

nilaxan@localhost:~$ uptime
 18:25:26 up 22 min,  1 user,  load average: 0.00, 0.26, 0.31  [<-- Watch Load Average]
nilaxan@localhost:~$ sudo stress-ng --cpu 4 --cpu-method fft --timeout 2m
stress-ng: info: [1281] dispatching hogs: 4 cpu
stress-ng: info: [1281] successful run completed in 120.01s
nilaxan@localhost:~$ uptime
 18:27:31 up 24 min,  1 user,  load average: 3.21, 1.49, 0.76  [<-- Watch Load Average]

3. To run 5 hdd stressors and stop after 100000 bogo operations, run this command.
nilaxan@localhost:~$ uptime
nilaxan@localhost:~$ sudo stress-ng --hdd 5 --hdd-ops 100000
nilaxan@localhost:~$ uptime

Sample Output

nilaxan@localhost:~$ uptime
 18:29:32 up 26 min,  1 user,  load average: 0.43, 1.00, 0.67  [<-- Watch Load Average] 
nilaxan@localhost:~$ sudo stress-ng --hdd 5 --hdd-ops 100000
stress-ng: info: [1290] defaulting to a 86400 second run per stressor
stress-ng: info: [1290] dispatching hogs: 5 hdd
stress-ng: info: [1290] successful run completed in 136.16s
nilaxan@localhost:~$ uptime
 18:31:56 up 29 min,  1 user,  load average: 4.24, 2.49, 1.28  [<-- Watch Load Average]

4. To run 8 CPU stressors, 4 I/O stressors and 1 
virtual memory stressor using 1GB of virtual memory for one minute, run 
this command below.
nilaxan@localhost:~$ uptime
nilaxan@localhost:~$ sudo stress-ng --cpu 4 --io 4 --vm 1 --vm-bytes 1G --timeout 60s --metrics-brief
nilaxan@localhost:~$ uptime

Sample Output

nilaxan@localhost:~$ uptime
 18:34:18 up 31 min,  1 user,  load average: 0.41, 1.56, 1.10  [<-- Watch Load Average]
nilaxan@localhost:~$ sudo stress-ng --cpu 4 --io 4 --vm 1 --vm-bytes 1G --timeout 60s --metrics-brief
stress-ng: info: [1304] dispatching hogs: 4 cpu, 4 iosync, 1 vm
stress-ng: info: [1304] successful run completed in 60.12s
stress-ng: info: [1304] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info: [1304]       (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info: [1304] cpu    1501     60.07      2.67     10.39        24.99       114.93
stress-ng: info: [1304] iosync  381463     60.01      0.00     12.90      6357.10     29570.78
nilaxan@localhost:~$ uptime
 18:35:36 up 32 min,  1 user,  load average: 4.66, 2.80, 1.59  [<-- Watch Load Average]

Summary


As recommended, these tools should be used with superuser privileges as they have certain effects on the system. These tools are good for general System Administration in Linux. I hope this guide was useful and if you have any additional ideas on how to test the health status of your system using these tools or any other tools.