Runtime debugging tools for Linux

Here’s a useful presentation on Linux debugging tools — tools that don’t require source code, additional prints or logging.

strace has a new flag that I didn’t know about: -y, which prints the paths that are associated with file descriptors.
opensnoop lets you see the details of open() calls across the entire system, or for an individual process, or for paths containing certain characters, or it can print the file paths that couldn’t be opened.
pgrep shows the stack trace of a running process, which can be useful to get an idea of what a program spends most of its time doing.
dstat shows system resource stats. It is a replacement for vmstat, iostat and ifstat.
htop — a more beautiful ‘top’, and easier to use. I still mostly use ‘top’ because it is installed by default. Other great tools I use include ‘powertop’ and ‘iotop’.
ngrep — an alternative to tcpdump, but allows the use of regexes to match plain-text data in packets.
tcpdump — useful when troubleshooting network connections between servers.
wireshark — a more UI-friendly tool than tcpdump, with dissectors for most protocols