There are many utilities for examing the state of a running system. The /proc file system and top are two that provide a lot of informtation. This page describes both, lists other tools, and shows some examples of how they can be used to obtain information about different parts of the system.

Contents:

• /proc and /sys file systems and lsmem, lscpu, lgpu, ...
• top and htop
• Utilities for finding information about differt parts of the system, and some Examples

/proc is a pseudo file system that provides access to process, and also to system state. /sys is a pseudo file system that provides information about system state, including devices, buses, and drivers.

Through files in /proc and /sys you can obtain an enourmous amount of information about the system (see some of the examples later on this page). Reading to these file are a way to obtain information about system and process state. Some files can be written to as well, and writing to a file triggers a change in something about the system or a particular process' state/policy/etc.

Each process running in the system has a subdirectory in /proc with lots of pseudo files that are interfaces to its state and status. These subdirectories are named by the process' pid numbers (run ps -A to see all pids in the system). By reading from files in a process' /proc/pid/ subdirectory, you can obtain all kinds of infromation about it. Some of these files can also be written to. Debuggers, for example, might write to files in this directory to set breakpoints.

Here is an example of reading a /proc file to get some information about a particular proces:

cd /proc/1234
ls                       # lists a lot of pseudo files assoced with pid 1234
cat /proc/1234/status    # cat out of these files
                         #   status: list lots of info about process 1234

Here are some other examples of getting system info from /proc:

cat /proc/meminfo    # list information about how memory is currently used
cat /proc/swaps      # see swap partitions and their use
cat /proc/stat       # see lots of statistics about cpus use

lscpu, lsmem, lshw, ...

Linux provides some lsX utilities that retrieve some information from /proc and /sys and summarize it an easier to read form that cat'ing out /proc files. lscpu and lsmem are two examples to list information about the CPU and memory architecture on a machine. For example, to find the number of physical cores on a machine I'd look at these values from lscpu:

lscpu 
...
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              8
Socket(s):                       2

This tells me that there are a total of 16 physical cores on this machine (2 Sockets (or chips), and 8 cores per socket (chip)), and that each core is dual-hyperthreaded to make the 16 core machine appear to the OS as 32 cpus.

Look at the man page for lscpu, lsmem, and others to see what they display and their command line options.

more information:

To find out more information about /proc and what system state you can examine (and change):

• run procinfo lists some of the data available through /proc:

  procinfo
  

• Look at the man page:

   man 5 proc 

  (or on-line proc man page)

• just ls, cd and cat out /proc and /sys files directly to figure out what they are/do.

  For example here is how to list process 1's executable name:

  cat /proc/1/comm
  

• if you have access to linux source, in the Documentation subdirectory is a lot of information about particular files in /proc. For example if I have the 4.4.0 source, I can find it here:

  linux-4.4.0/Documentation/
  

• some documentation about the /proc file system
• a paper about the /sys file system
• /proc/stat explained (from linuxhowtos.org)
• Description of the /sys and /proc interfaces (from kernel user_space_howto)

top and htop are Unix utilities that lists a lot of information about processes and threads running in the system and how they are using resources like memory and CPU. If you run top with no command line options, then top will display per-process statistics. If you run top with -H, top will display statistics for individual threads:

top -H

htop has slightly different display of these data. You can sort the data on different fields using the F6 function key.

top is very configurable to allow you to change what data it displays.

Here is some more information about using top and htop including an example of how to configure top to display different data. The example it shows may be particularly useful for multi-threaded processes.

Also see the man page for top for more information.

Below are a few examples of utilities for examining some system state (any utilities that are only usable as root, are shown being run as 'sudo command' below). Note: some of these commands are in /sbin. If /sbin is not in your path, then list the full path name to the command (/sbin/command_name) or add /sbin to your path. See man pages for more command line options.

User info

 who         # prints out all users logged into machine
 whoami      # in case you forget who you are
 finger name # get some basic information about accounts with name or username

 id          # list of current user's identifying info (uid, gid, groups, ...)

 env         # list all your environment variables

Kernel

 uname -a         # print kernel version
 cat /proc/stat   # print lots of system statistics
 /proc/sys/       # /proc directory contain lots of kernel variables 
                  # (some can be written to change kernel configuation) 
 procinfo         # print out a bunch of system stats from /proc

 getconf -a       # list all the system configuration variable values

Hardware Config

 lscpu    # list information about cpu architecture
 lsmem    # list information about mem architecture
 lsdev    # list installed hardware
 lspci    # list all pci devices
 lshw     # list detailed config of hardware
 cat /proc/cpuinfo    # information about processor(s)

 # detailed information about cpus is in /sys/devices/system/cpu/ subdirs
 # for example to see information about the L1 data cache:
 cat /sys/devices/system/cpu/cpu0/cache/index0/size
 cat /sys/devices/system/cpu/cpu0/cache/index0/type
 cat /sys/devices/system/cpu/cpu0/cache/index0/level
 ...

 getconf  # will list some cache size info too

CPU and on-chip caches

 cat /proc/cpuinfo    # lists information about each core

 # list information about on-chip caches assocated with each core (cpuX)
 # each level of cache's info in a diffrent indexNUM subdirectory:
 cat /sys/devices/system/cpu/cpuX/cache/indexNUM/type|size|...

 # (ex) list info about core 8's index0 cache (its L1 data cache):
 cat /sys/devices/system/cpu/cpu8/cache/index0/size
 cat /sys/devices/system/cpu/cpu8/cache/index0/type
 cat /sys/devices/system/cpu/cpu8/cache/index0/level
 cat /sys/devices/system/cpu/cpu8/cache/index0/ways_of_associativity
 cat /sys/devices/system/cpu/cpu8/cache/index0/number_of_sets
 cat /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list

Memory

 free                # information about free memory in the system
 cat /proc/meminfo   # information about memory usage in the system
 cat /proc/slabinfo  # information about kernel caches
 cat /proc/swaps     # informatin about swap partitions
 top                 # real-time update of running system: memory use, processes
 getconf PAGESIZE    # see the system page size
 vmstat              # list virtual memory statistics

Linux Memory Management has some example of how to obtain and interpret information about memory

Running Processes

 top                  # information about processes running on system 
 ps -A                # list all processes running on the system
 cat /proc/loadavg    # get information about system load
 uptime               # load info
 xload                # realtime load info

Disk and other devices

 du                   # prints out disk usage information 
 df                   # displays disk usage summary for each partition
 sfdisk -l            # list the partion table for disk devices
 cat /proc/scsi/scsi  # see scsi devices known by kernel 
 cat /proc/ide        # see ide devices  known by kernel
 iostat               # see R and W accesses to different devices
 sudo fdisk /dev/sda  # choose option p to print information about the device, q to quit
                      # (you must be root to run fdisk.  be very careful not to choose
                      # options that change the partition tables for this device)

See the man pages for (du, df, iostat) for more information on these. Also, here is some other information on using iostat from admin magazine.

File System

 file filename      # list information about the type of a particular file 
 stat filename      # list filesystem information about a particular file 

 du                 # prints out disk usage information  
 df                 # list space on all mounted filesytems 
 ls -il             # lists the inode numbers of files
 lsof               # list open files
 cat /proc/sys/fs/  # contains files with file system stats

# /sys/fs/ contains info about filesystems mounted on specific device partions
# /sys/fs/FILESYSNAME/DEVICEPARTITON/  
# for example:
 cat /sys/fs/ext4/sdb7/session_write_kbytes  # num KB written since mounted


 sudo debugfs       # ext2 filesystem debugger  (need sudo access)

information about the files in /sys/fs/ext4 from kernel.org

Network

 cat /proc/net/dev    # network device status information 
 cat /sys/class/net/<interface>    # get information aobut specific NW interface
   cat /sys/class/net/eth0/speed   # e.g., get speed of ethernet (in MBi/sec) 
 ss                   # socket statistics tool  ss -i dumps a lot of info
 ifconfig -a          # display configure info for all NW interfaces on system
 dig                  # dns lookup 
 nslookup             # another dns lookup tool
 arp -v 
 netstat              # information about nw connections, routing tables, etc.
 tcpdump              # dump tcp traffic (need root access to run this)
 wireshark            # tool to interactively dump and analyize NW traffic 

See the man page for (netstat, procinfo, ifconfig, arp, dig, wireshark) for more information on these