83

It's a question which has been asked many times, however there is no well supported answer I could find.

Many people suggest the use of top command, but if you run top once (because you have a script for example collecting Cpu usage every 1 second) it will always give the same Cpu usage result (example 1, example 2).

A more accurate way to calculate CPU usage, is by reading the values from /proc/stat, but most of the answers use only the first 4 fields from /proc/stat to calculate it (one example here).

/proc/stat/ has 10 fields per CPU core as of Linux kernel 2.6.33!

I also found this Accurately Calculating CPU Utilization in Linux using /proc/stat question which is pointing out the same issue, -that most other questions only take into consideration 4 out of the many fields- but still the answer given here starts with "I think" (not certain), and except that, it is only concerned about the first 7 fields (out of 10 in /proc/stat/)

This perl script uses all of the fields to calculate the CPU usage, which again I do not think is correct after some further investigation.

After taking a quick look into the kernel code here, it looks like, for example, guest_nice and guest fields are always increasing together with nice and user (so they should not be included in the cpu usage calculation, since they are included in nice and user fields already)

/* * Account guest cpu time to a process. * @p: the process that the cpu time gets accounted to * @cputime: the cpu time spent in virtual machine since the last update * @cputime_scaled: cputime scaled by cpu frequency */ static void account_guest_time(struct task_struct *p, cputime_t cputime, cputime_t cputime_scaled) { u64 *cpustat = kcpustat_this_cpu->cpustat; /* Add guest time to process. */ p->utime += cputime; p->utimescaled += cputime_scaled; account_group_user_time(p, cputime); p->gtime += cputime; /* Add guest time to cpustat. */ if (task_nice(p) > 0) { cpustat[CPUTIME_NICE] += (__force u64) cputime; cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime; } else { cpustat[CPUTIME_USER] += (__force u64) cputime; cpustat[CPUTIME_GUEST] += (__force u64) cputime; } } 

So to sum up, what is an accurate way to calculate the CPU usage in Linux and which fields should be considered in the calculations and how (which fields are attributed to the idle time and which fields to non-idle time)?

4
  • The right way to collect CPU usage information every second is to run top -b continuously. Commented Apr 29, 2014 at 15:38
  • I would like to collect data by using a 3rd party script and CPU is only one of the metrics need to be collected. So I would like to calculate CPU usage since the previous run (interval might vary) of this 3rd party script. top -b runs continuously, so it has to run in a separate thread and save the collected data in a different output. Commented Apr 29, 2014 at 16:36
  • What CPU usage are looking for? A single process? System-wide? Should it be expressed as a percentage, seconds, ...? Commented Apr 29, 2014 at 22:13
  • Percentage of usage from the previous measurement! Commented Apr 29, 2014 at 22:44

6 Answers 6

114

According the htop source code, my assumptions looks like they are valid:

(see static inline double LinuxProcessList_scanCPUTime(LinuxProcessList* this) function at LinuxProcessList.c)

// Guest time is already accounted in usertime usertime = usertime - guest; # As you see here, it subtracts guest from user time nicetime = nicetime - guestnice; # and guest_nice from nice time // Fields existing on kernels >= 2.6 // (and RHEL's patched kernel 2.4...) unsigned long long int idlealltime = idletime + ioWait; # ioWait is added in the idleTime unsigned long long int systemalltime = systemtime + irq + softIrq; unsigned long long int virtalltime = guest + guestnice; unsigned long long int totaltime = usertime + nicetime + systemalltime + idlealltime + steal + virtalltime; 

And so, from fields listed in the first line of /proc/stat: (see section 1.8 at documentation)

 user nice system idle iowait irq softirq steal guest guest_nice cpu 74608 2520 24433 1117073 6176 4054 0 0 0 0 

Algorithmically, we can calculate the CPU usage percentage like:

PrevIdle = previdle + previowait Idle = idle + iowait PrevNonIdle = prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal NonIdle = user + nice + system + irq + softirq + steal PrevTotal = PrevIdle + PrevNonIdle Total = Idle + NonIdle # differentiate: actual value minus the previous one totald = Total - PrevTotal idled = Idle - PrevIdle CPU_Percentage = (totald - idled)/totald 
Sign up to request clarification or add additional context in comments.

6 Comments

I'd like to share a little example program that I created based on this answer: github.com/scaidermern/top-processes. Feel free to make us of it, it is licensed under CC0.
I reckon iowait is an uninterruptible state, so it would be more practical to count it to "business" rather than idle.
guest and guest_nice should be added to NonIdle as well according to this post: unix.stackexchange.com/a/303224
@KErlandsson according to this post, guest and guest_nice are already added into the user and nice time. As mention as comment in the Vangelus response Guest time is already accounted in usertime
Is not work, my cpu usage is 80% but the results says 1.0 to 8.0.
|
21

The following is a bash script which is based on Vangelis's answer. It produces output like this:

total 49.1803 cpu0 14.2857 cpu1 100 cpu2 28.5714 cpu3 100 cpu4 30 cpu5 25 

Create a file called get_cpu_usage.sh

Run it using the following command: bash get_cpu_usage.sh 0.2

The argument is the number of seconds to measure. In this case it's 200 milliseconds.

The contents are:

#!/bin/sh sleepDurationSeconds=$1 previousDate=$(date +%s%N | cut -b1-13) previousStats=$(cat /proc/stat) sleep $sleepDurationSeconds currentDate=$(date +%s%N | cut -b1-13) currentStats=$(cat /proc/stat) cpus=$(echo "$currentStats" | grep -P 'cpu' | awk -F " " '{print $1}') for cpu in $cpus do currentLine=$(echo "$currentStats" | grep "$cpu ") user=$(echo "$currentLine" | awk -F " " '{print $2}') nice=$(echo "$currentLine" | awk -F " " '{print $3}') system=$(echo "$currentLine" | awk -F " " '{print $4}') idle=$(echo "$currentLine" | awk -F " " '{print $5}') iowait=$(echo "$currentLine" | awk -F " " '{print $6}') irq=$(echo "$currentLine" | awk -F " " '{print $7}') softirq=$(echo "$currentLine" | awk -F " " '{print $8}') steal=$(echo "$currentLine" | awk -F " " '{print $9}') guest=$(echo "$currentLine" | awk -F " " '{print $10}') guest_nice=$(echo "$currentLine" | awk -F " " '{print $11}') previousLine=$(echo "$previousStats" | grep "$cpu ") prevuser=$(echo "$previousLine" | awk -F " " '{print $2}') prevnice=$(echo "$previousLine" | awk -F " " '{print $3}') prevsystem=$(echo "$previousLine" | awk -F " " '{print $4}') previdle=$(echo "$previousLine" | awk -F " " '{print $5}') previowait=$(echo "$previousLine" | awk -F " " '{print $6}') previrq=$(echo "$previousLine" | awk -F " " '{print $7}') prevsoftirq=$(echo "$previousLine" | awk -F " " '{print $8}') prevsteal=$(echo "$previousLine" | awk -F " " '{print $9}') prevguest=$(echo "$previousLine" | awk -F " " '{print $10}') prevguest_nice=$(echo "$previousLine" | awk -F " " '{print $11}') PrevIdle=$((previdle + previowait)) Idle=$((idle + iowait)) PrevNonIdle=$((prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal)) NonIdle=$((user + nice + system + irq + softirq + steal)) PrevTotal=$((PrevIdle + PrevNonIdle)) Total=$((Idle + NonIdle)) totald=$((Total - PrevTotal)) idled=$((Idle - PrevIdle)) CPU_Percentage=$(awk "BEGIN {print ($totald - $idled)/$totald*100}") if [[ "$cpu" == "cpu" ]]; then echo "total "$CPU_Percentage else echo $cpu" "$CPU_Percentage fi done 

Comments

7

Hey i was also researching for the topic and found this thread really helpful. I used Vangelis Tasoulas formula to write a small python script for this. Attached is my Python code for the issue. It loads the cpu usage per cpu_id every second. Maybe its helps others as well. Also comments/suggestions are welcome :-)

#!/usr/bin/python # -*- coding: utf-8 -*- ''' Created on 04.12.2014 @author: plagtag ''' from time import sleep import sys class GetCpuLoad(object): ''' classdocs ''' def __init__(self, percentage=True, sleeptime = 1): ''' @parent class: GetCpuLoad @date: 04.12.2014 @author: plagtag @info: @param: @return: CPU load in percentage ''' self.percentage = percentage self.cpustat = '/proc/stat' self.sep = ' ' self.sleeptime = sleeptime def getcputime(self): ''' http://stackoverflow.com/questions/23367857/accurate-calculation-of-cpu-usage-given-in-percentage-in-linux read in cpu information from file The meanings of the columns are as follows, from left to right: 0cpuid: number of cpu 1user: normal processes executing in user mode 2nice: niced processes executing in user mode 3system: processes executing in kernel mode 4idle: twiddling thumbs 5iowait: waiting for I/O to complete 6irq: servicing interrupts 7softirq: servicing softirqs #the formulas from htop user nice system idle iowait irq softirq steal guest guest_nice cpu 74608 2520 24433 1117073 6176 4054 0 0 0 0 Idle=idle+iowait NonIdle=user+nice+system+irq+softirq+steal Total=Idle+NonIdle # first line of file for all cpus CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal) ''' cpu_infos = {} #collect here the information with open(self.cpustat,'r') as f_stat: lines = [line.split(self.sep) for content in f_stat.readlines() for line in content.split('\n') if line.startswith('cpu')] #compute for every cpu for cpu_line in lines: if '' in cpu_line: cpu_line.remove('')#remove empty elements cpu_line = [cpu_line[0]]+[float(i) for i in cpu_line[1:]]#type casting cpu_id,user,nice,system,idle,iowait,irq,softrig,steal,guest,guest_nice = cpu_line Idle=idle+iowait NonIdle=user+nice+system+irq+softrig+steal Total=Idle+NonIdle #update dictionionary cpu_infos.update({cpu_id:{'total':Total,'idle':Idle}}) return cpu_infos def getcpuload(self): ''' CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal) ''' start = self.getcputime() #wait a second sleep(self.sleeptime) stop = self.getcputime() cpu_load = {} for cpu in start: Total = stop[cpu]['total'] PrevTotal = start[cpu]['total'] Idle = stop[cpu]['idle'] PrevIdle = start[cpu]['idle'] CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal)*100 cpu_load.update({cpu: CPU_Percentage}) return cpu_load if __name__=='__main__': x = GetCpuLoad() while True: try: data = x.getcpuload() print data except KeyboardInterrupt: sys.exit("Finished") 

1 Comment

unfortunately, this doesn't appear to be accurate, see my question here: stackoverflow.com/questions/60579935/…
2

idnt.net has a good description for how to use the /proc/stat cpu data, include a bash-script for extracting cpu and description of the lines. I just wanted to link it here, since I found it valuable.

Comments

1

I was also looking for the same. Here is my ruby program based on the Vangelis Tasoulas's answer:

#!/usr/bin/env ruby $VERBOSE = true prev_file = IO.readlines(::File.join('', 'proc', 'stat')).select { |line| line.start_with?('cpu') } Kernel.sleep(0.05) file = IO.readlines(::File.join('', 'proc', 'stat')).select { |line| line.start_with?('cpu') } file.size.times do |i| data, prev_data = file[i].split.map(&:to_f), prev_file[i].split.map(&:to_f) %w(user nice sys idle iowait irq softirq steal).each_with_index do |el, index| eval "@#{el}, @prev_#{el} = #{data[index + 1]}, #{prev_data[index + 1]}" end previdle, idle = @prev_idle + @prev_iowait, @idle + @iowait totald = idle + (@user + @nice + @sys + @irq + @softirq + @steal) - (previdle + (@prev_user + @prev_nice + @prev_sys + @prev_irq + @prev_softirq + @prev_steal)) puts "CPU #{i}: #{((totald - (idle - previdle)) / totald * 100).round(2)} %" end 

Comments

0

The following is a bash script builds on Fidel's answer and arberg's link.

I wanted to lower the usage of cat awk grep and date calls and spend less cpu usage trying to figure out the cpu usage.

Output:

total: 4% cpu0: 10% cpu1: 5% cpu2: 1% cpu3: 1% 

create a bash script using this:

#!/bin/bash # Paramiter one used to set time in sec between reads sleepDurationSeconds=$1 # read cpu stats to arrays readarray -t previousStats < <( awk '/^cpu /{flag=1}/^intr/{flag=0}flag' /proc/stat ) sleep $sleepDurationSeconds readarray -t currentStats < <( awk '/^cpu /{flag=1}/^intr/{flag=0}flag' /proc/stat ) # loop through the arrays for i in "${!previousStats[@]}"; do # Break up arrays 1 line sting into an array element for each item in string previousStat_elemant_array=(${previousStats[i]}) currentStat_elemant_array=(${currentStats[i]}) # Get all columns from user to steal previousStat_colums="${previousStat_elemant_array[@]:1:7}" currentStat_colums="${currentStat_elemant_array[@]:1:7}" # Replace the column seperator (space) with + previous_cpu_sum=$((${previousStat_colums// /+})) current_cpu_sum=$((${currentStat_colums// /+})) # Get the delta between two reads cpu_delta=$((current_cpu_sum - previous_cpu_sum)) # Get the idle time Delta cpu_idle=$((currentStat_elemant_array[4]- previousStat_elemant_array[4])) # Calc time spent working cpu_used=$((cpu_delta - cpu_idle)) # Calc percentage cpu_usage=$((100 * cpu_used / cpu_delta)) # Get cpu used for calc cpu percentage used cpu_used_for_calc="${currentStat_elemant_array[0]}" if [[ "$cpu_used_for_calc" == "cpu" ]]; then echo "total: "$cpu_usage"%" else echo $cpu_used_for_calc": "$cpu_usage"%" fi done 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.