Observe and troubleshoot system performance on an Ubuntu server

To observe and troubleshoot system performance on an Ubuntu server, there are several commands we can run:

htop or top

These are real-time system monitors that show a detailed list of running processes, along with information on CPU, memory usage, and more. htop is an enhanced version of top with a better user interface and more features.

vmstat

vmstat command reports virtual memory statistics and can give an overview of system processes, memory, paging, block IO, traps, and CPU activity.

vmstat -s

iostat

iostat is useful for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates.

iostat

mpstat

mpstat command is used to display CPU usage statistics.

mpstat -P ALL

free

free shows the amount of free and used memory in the system.

free -h

sar

sar System Activity Reporter can collect, report, or save system activity information.

sar -u 1 3

dstat

dstat is versatile tool for generating system resource statistics.

dstat

netstat

netstat shows network statistics. This can help us figure out network bottlenecks.

netstat -tulnp

nmon

nmon is high-performance system monitor tool for Linux that shows CPU, memory, network, disk, and other information.

nmon

lsof

lsof lists open files and the corresponding processes. Useful for finding out what files are being used by which processes.

lsof

mysqldumpslow

mysqldumpslow parse MySQL slow query log files and summarize their contents to identify slow-running SQL queries.

mysqldumpslow /path/to/slowquery.log

mysqltuner

A script that evaluates your MySQL installation and provides suggestions for performance improvements.

mysqltuner

perf

A performance analyzing tool in Linux, useful for detailed performance analysis.

perf stat -B command_to_run

To use some of these commands, you may need to install the relevant packages on your Ubuntu server using apt. For example:

sudo apt install sysstat iotop htop dstat

These commands can help you pinpoint the cause of the high CPU usage by MySQL.

However, interpreting the output of these commands can be complex and might require a good understanding of Linux system internals.