1Demonstrations of opensnoop, the Linux eBPF/bcc version. 2 3 4opensnoop traces the open() syscall system-wide, and prints various details. 5Example output: 6 7# ./opensnoop 8PID COMM FD ERR PATH 917326 <...> 7 0 /sys/kernel/debug/tracing/trace_pipe 101576 snmpd 9 0 /proc/net/dev 111576 snmpd 11 0 /proc/net/if_inet6 121576 snmpd 11 0 /proc/sys/net/ipv4/neigh/eth0/retrans_time_ms 131576 snmpd 11 0 /proc/sys/net/ipv6/neigh/eth0/retrans_time_ms 141576 snmpd 11 0 /proc/sys/net/ipv6/conf/eth0/forwarding 151576 snmpd 11 0 /proc/sys/net/ipv6/neigh/eth0/base_reachable_time_ms 161576 snmpd 11 0 /proc/sys/net/ipv4/neigh/lo/retrans_time_ms 171576 snmpd 11 0 /proc/sys/net/ipv6/neigh/lo/retrans_time_ms 181576 snmpd 11 0 /proc/sys/net/ipv6/conf/lo/forwarding 191576 snmpd 11 0 /proc/sys/net/ipv6/neigh/lo/base_reachable_time_ms 201576 snmpd 9 0 /proc/diskstats 211576 snmpd 9 0 /proc/stat 221576 snmpd 9 0 /proc/vmstat 231956 supervise 9 0 supervise/status.new 241956 supervise 9 0 supervise/status.new 2517358 run 3 0 /etc/ld.so.cache 2617358 run 3 0 /lib/x86_64-linux-gnu/libtinfo.so.5 2717358 run 3 0 /lib/x86_64-linux-gnu/libdl.so.2 2817358 run 3 0 /lib/x86_64-linux-gnu/libc.so.6 2917358 run -1 6 /dev/tty 3017358 run 3 0 /proc/meminfo 3117358 run 3 0 /etc/nsswitch.conf 3217358 run 3 0 /etc/ld.so.cache 3317358 run 3 0 /lib/x86_64-linux-gnu/libnss_compat.so.2 3417358 run 3 0 /lib/x86_64-linux-gnu/libnsl.so.1 3517358 run 3 0 /etc/ld.so.cache 3617358 run 3 0 /lib/x86_64-linux-gnu/libnss_nis.so.2 3717358 run 3 0 /lib/x86_64-linux-gnu/libnss_files.so.2 3817358 run 3 0 /etc/passwd 3917358 run 3 0 ./run 40^C 41 42While tracing, the snmpd process opened various /proc files (reading metrics), 43and a "run" process read various libraries and config files (looks like it 44was starting up: a new process). 45 46opensnoop can be useful for discovering configuration and log files, if used 47during application startup. 48 49 50The -p option can be used to filter on a PID, which is filtered in-kernel. Here 51I've used it with -T to print timestamps: 52 53 ./opensnoop -Tp 1956 54TIME(s) PID COMM FD ERR PATH 550.000000000 1956 supervise 9 0 supervise/status.new 560.000289999 1956 supervise 9 0 supervise/status.new 571.023068000 1956 supervise 9 0 supervise/status.new 581.023381997 1956 supervise 9 0 supervise/status.new 592.046030000 1956 supervise 9 0 supervise/status.new 602.046363000 1956 supervise 9 0 supervise/status.new 613.068203997 1956 supervise 9 0 supervise/status.new 623.068544999 1956 supervise 9 0 supervise/status.new 63 64This shows the supervise process is opening the status.new file twice every 65second. 66 67 68The -U option include UID on output: 69 70# ./opensnoop -U 71UID PID COMM FD ERR PATH 720 27063 vminfo 5 0 /var/run/utmp 73103 628 dbus-daemon -1 2 /usr/local/share/dbus-1/system-services 74103 628 dbus-daemon 18 0 /usr/share/dbus-1/system-services 75103 628 dbus-daemon -1 2 /lib/dbus-1/system-services 76 77 78The -u option filtering UID: 79 80# ./opensnoop -Uu 1000 81UID PID COMM FD ERR PATH 821000 30240 ls 3 0 /etc/ld.so.cache 831000 30240 ls 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 841000 30240 ls 3 0 /lib/x86_64-linux-gnu/libc.so.6 851000 30240 ls 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 861000 30240 ls 3 0 /lib/x86_64-linux-gnu/libdl.so.2 871000 30240 ls 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 88 89The -x option only prints failed opens: 90 91# ./opensnoop -x 92PID COMM FD ERR PATH 9318372 run -1 6 /dev/tty 9418373 run -1 6 /dev/tty 9518373 multilog -1 13 lock 9618372 multilog -1 13 lock 9718384 df -1 2 /usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo 9818384 df -1 2 /usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo 9918384 df -1 2 /usr/share/locale/en_US/LC_MESSAGES/coreutils.mo 10018384 df -1 2 /usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo 10118384 df -1 2 /usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo 10218384 df -1 2 /usr/share/locale/en/LC_MESSAGES/coreutils.mo 10318385 run -1 6 /dev/tty 10418386 run -1 6 /dev/tty 105 106This caught a df command failing to open a coreutils.mo file, and trying from 107different directories. 108 109The ERR column is the system error number. Error number 2 is ENOENT: no such 110file or directory. 111 112 113A maximum tracing duration can be set with the -d option. For example, to trace 114for 2 seconds: 115 116# ./opensnoop -d 2 117PID COMM FD ERR PATH 1182191 indicator-multi 11 0 /sys/block 1192191 indicator-multi 11 0 /sys/block 1202191 indicator-multi 11 0 /sys/block 1212191 indicator-multi 11 0 /sys/block 1222191 indicator-multi 11 0 /sys/block 123 124 125The -n option can be used to filter on process name using partial matches: 126 127# ./opensnoop -n ed 128 129PID COMM FD ERR PATH 1302679 sed 3 0 /etc/ld.so.cache 1312679 sed 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 1322679 sed 3 0 /lib/x86_64-linux-gnu/libc.so.6 1332679 sed 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 1342679 sed 3 0 /lib/x86_64-linux-gnu/libdl.so.2 1352679 sed 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 1362679 sed 3 0 /proc/filesystems 1372679 sed 3 0 /usr/lib/locale/locale-archive 1382679 sed -1 2 1392679 sed 3 0 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 1402679 sed 3 0 /dev/null 1412680 sed 3 0 /etc/ld.so.cache 1422680 sed 3 0 /lib/x86_64-linux-gnu/libselinux.so.1 1432680 sed 3 0 /lib/x86_64-linux-gnu/libc.so.6 1442680 sed 3 0 /lib/x86_64-linux-gnu/libpcre.so.3 1452680 sed 3 0 /lib/x86_64-linux-gnu/libdl.so.2 1462680 sed 3 0 /lib/x86_64-linux-gnu/libpthread.so.0 1472680 sed 3 0 /proc/filesystems 1482680 sed 3 0 /usr/lib/locale/locale-archive 1492680 sed -1 2 150^C 151 152This caught the 'sed' command because it partially matches 'ed' that's passed 153to the '-n' option. 154 155 156The -e option prints out extra columns; for example, the following output 157contains the flags passed to open(2), in octal: 158 159# ./opensnoop -e 160PID COMM FD ERR FLAGS PATH 16128512 sshd 10 0 00101101 /proc/self/oom_score_adj 16228512 sshd 3 0 02100000 /etc/ld.so.cache 16328512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libwrap.so.0 16428512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libaudit.so.1 16528512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libpam.so.0 16628512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libselinux.so.1 16728512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libsystemd.so.0 16828512 sshd 3 0 02100000 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2 16928512 sshd 3 0 02100000 /lib/x86_64-linux-gnu/libutil.so.1 170 171 172The -f option filters based on flags to the open(2) call, for example: 173 174# ./opensnoop -e -f O_WRONLY -f O_RDWR 175PID COMM FD ERR FLAGS PATH 17628084 clear_console 3 0 00100002 /dev/tty 17728084 clear_console -1 13 00100002 /dev/tty0 17828084 clear_console -1 13 00100001 /dev/tty0 17928084 clear_console -1 13 00100002 /dev/console 18028084 clear_console -1 13 00100001 /dev/console 18128051 sshd 8 0 02100002 /var/run/utmp 18228051 sshd 7 0 00100001 /var/log/wtmp 183 184 185The --cgroupmap option filters based on a cgroup set. It is meant to be used 186with an externally created map. 187 188# ./opensnoop --cgroupmap /sys/fs/bpf/test01 189 190For more details, see docs/special_filtering.md 191 192 193USAGE message: 194 195# ./opensnoop -h 196usage: opensnoop.py [-h] [-T] [-U] [-x] [-p PID] [-t TID] 197 [--cgroupmap CGROUPMAP] [--mntnsmap MNTNSMAP] [-u UID] 198 [-d DURATION] [-n NAME] [-e] [-f FLAG_FILTER] [-F] 199 [-b BUFFER_PAGES] 200 201Trace open() syscalls 202 203optional arguments: 204 -h, --help show this help message and exit 205 -T, --timestamp include timestamp on output 206 -U, --print-uid print UID column 207 -x, --failed only show failed opens 208 -p PID, --pid PID trace this PID only 209 -t TID, --tid TID trace this TID only 210 --cgroupmap CGROUPMAP 211 trace cgroups in this BPF map only 212 --mntnsmap MNTNSMAP trace mount namespaces in this BPF map only 213 -u UID, --uid UID trace this UID only 214 -d DURATION, --duration DURATION 215 total duration of trace in seconds 216 -n NAME, --name NAME only print process names containing this name 217 -e, --extended_fields 218 show extended fields 219 -f FLAG_FILTER, --flag_filter FLAG_FILTER 220 filter on flags argument (e.g., O_WRONLY) 221 -F, --full-path show full path for an open file with relative path 222 -b BUFFER_PAGES, --buffer-pages BUFFER_PAGES 223 size of the perf ring buffer (must be a power of two 224 number of pages and defaults to 64) 225 226examples: 227 ./opensnoop # trace all open() syscalls 228 ./opensnoop -T # include timestamps 229 ./opensnoop -U # include UID 230 ./opensnoop -x # only show failed opens 231 ./opensnoop -p 181 # only trace PID 181 232 ./opensnoop -t 123 # only trace TID 123 233 ./opensnoop -u 1000 # only trace UID 1000 234 ./opensnoop -d 10 # trace for 10 seconds only 235 ./opensnoop -n main # only print process names containing "main" 236 ./opensnoop -e # show extended fields 237 ./opensnoop -f O_WRONLY -f O_RDWR # only print calls for writing 238 ./opensnoop -F # show full path for an open file with relative path 239 ./opensnoop --cgroupmap mappath # only trace cgroups in this BPF map 240 ./opensnoop --mntnsmap mappath # only trace mount namespaces in the map 241