Setting up monitoring using rrdtool (and rrdcollect)

This material is work-in-progress ... Do not follow instructions here until this notice is removed. (Last edited by Mhavela on 21 Mar 2012.)

Todo: Create/Modify/Verify rrdcollect and move it from 'testing' to 'main'

Install programs

apk add rrdtool rrdcollect

Create rrd-databases

As we will use rrdcollect to collect our data for us, we will create all databases that the default config for rrdcollect tries to use.

stat.rrd

rrdtool create /var/lib/rrdtool/stat.rrd \
--step 60 \
DS:cpu_user:COUNTER:120:0:U \
DS:cpu_nice:COUNTER:120:0:U \
DS:cpu_system:COUNTER:120:0:U \
DS:cpu_idle:COUNTER:120:0:U \
DS:cpu_iowait:COUNTER:120:0:U \
DS:cpu_irq:COUNTER:120:0:U \
DS:cpu_softirq:COUNTER:120:0:U \
DS:ctxt:COUNTER:120:0:U \
DS:page_in:COUNTER:120:0:U \
DS:page_out:COUNTER:120:0:U \
DS:processes:COUNTER:120:0:U \
DS:swap_in:COUNTER:120:0:U \
DS:swap_out:COUNTER:120:0:U \
RRA:AVERAGE:0.5:1:360 \
RRA:AVERAGE:0.5:10:1008 \
RRA:MAX:0.5:10:1008

memory.rrd

rrdtool create /var/lib/rrdtool/memory.rrd \
--step 60 \
DS:mem_used:GAUGE:120:0:U \
DS:mem_free:GAUGE:120:0:U \
DS:mem_shared:GAUGE:120:0:U \
DS:mem_buffers:GAUGE:120:0:U \
DS:mem_cached:GAUGE:120:0:U \
DS:swap_used:GAUGE:120:0:U \
RRA:AVERAGE:0.5:1:360 \
RRA:AVERAGE:0.5:10:1008 \
RRA:MAX:0.5:10:1008

eth0.rrd

rrdtool create /var/lib/rrdtool/eth0.rrd \
--step 60 \
DS:bytes_in:COUNTER:120:0:U \
DS:pkts_in:COUNTER:120:0:U \
DS:bytes_out:COUNTER:120:0:U \
DS:pkts_out:COUNTER:120:0:U \
RRA:AVERAGE:0.5:1:360 \
RRA:AVERAGE:0.5:10:1008 \
RRA:MAX:0.5:10:1008

(The default rrdcollect.conf tries to fetch data for eth1. With small modifications of the above command can be used to create such 'eth1.rrd')

Note: If you chose to change the "--step 60" which specifies the base interval in seconds with which data will be fed into the RRD.
Make sure to change the 'step' value in /etc/rrdcollect/rrdcollect.conf to reflect your changes above.

Tip: In the above examples the first RRA in each .rrd is more precise (1min interval), but it holds data for shorter time. (1x360x60) equals 21600s/6h)
The second RRA evaluates 10 min interval and holds data for longer period. (10x1008x60 equals 604800s/168h/7d)

Gather information and put it in the RRD

rc-service rrdcollect start

Create graphs based on the rrd's

In the below examples you will notice that the .png file is exported to "/var/www/localhost/htdocs/".
You would need to either create /var/www/localhost/htdocs/ or change the path for the images.

Stat

rrdtool graph /var/www/localhost/htdocs/stat.png --start -1800  \
-a PNG -t "Stat" --vertical-label "bits/s" \
-w 1260 -h 400 -r \
DEF:cpu_user=/var/lib/rrdtool/stat.rrd:cpu_user:AVERAGE \
DEF:cpu_nice=/var/lib/rrdtool/stat.rrd:cpu_nice:AVERAGE \
DEF:cpu_system=/var/lib/rrdtool/stat.rrd:cpu_system:AVERAGE \
DEF:cpu_idle=/var/lib/rrdtool/stat.rrd:cpu_idle:AVERAGE \
DEF:cpu_iowait=/var/lib/rrdtool/stat.rrd:cpu_iowait:AVERAGE \
DEF:cpu_irq=/var/lib/rrdtool/stat.rrd:cpu_irq:AVERAGE \
DEF:cpu_softirq=/var/lib/rrdtool/stat.rrd:cpu_softirq:AVERAGE \
DEF:ctxt=/var/lib/rrdtool/stat.rrd:ctxt:AVERAGE \
DEF:page_in=/var/lib/rrdtool/stat.rrd:page_in:AVERAGE \
DEF:page_out=/var/lib/rrdtool/stat.rrd:page_out:AVERAGE \
DEF:processes=/var/lib/rrdtool/stat.rrd:processes:AVERAGE \
DEF:swap_in=/var/lib/rrdtool/stat.rrd:swap_in:AVERAGE \
DEF:swap_out=/var/lib/rrdtool/stat.rrd:swap_out:AVERAGE \
AREA:cpu_user#D7CC00:cpu_user \
AREA:cpu_nice#D7CC00:cpu_nice \
LINE2:cpu_system#D73600:cpu_system \
LINE2:cpu_idle#D73600:cpu_idle \
LINE2:ctxt#0101D6:ctxt \
LINE2:page_in#0101D6:page_in \
LINE2:page_out#D73600:page_out \
LINE2:processes#D73600:processes \
LINE2:swap_in#D73600:swap_in \
LINE2:swap_out#D73600:swap_out 

Memory

rrdtool graph /var/www/localhost/htdocs/memory.png --start -1800  \
-a PNG -t "Memory" --vertical-label "" \
-w 1260 -h 400 -r \
DEF:mem_used=/var/lib/rrdtool/memory.rrd:mem_used:AVERAGE \
DEF:mem_free=/var/lib/rrdtool/memory.rrd:mem_free:AVERAGE \
DEF:mem_shared=/var/lib/rrdtool/memory.rrd:mem_shared:AVERAGE \
DEF:mem_buffers=/var/lib/rrdtool/memory.rrd:mem_buffers:AVERAGE \
DEF:mem_cached=/var/lib/rrdtool/memory.rrd:mem_cached:AVERAGE \
DEF:swap_used=/var/lib/rrdtool/memory.rrd:swap_used:AVERAGE \
AREA:mem_used#D7CC00:mem_used \
AREA:mem_free#D7CC00:mem_free \
LINE2:mem_shared#D73600:mem_shared \
LINE2:mem_buffers#D73600:mem_buffers \
LINE2:mem_cached#0101D6:mem_cached \
LINE2:swap_used#0101D6:swap_used

eth0

rrdtool graph /var/www/localhost/htdocs/eth0.png --start -1h  \
-a PNG -t "eth0" --vertical-label "bits/s" \
-w 1260 -h 400 -r \
DEF:bytes_in=/var/lib/rrdtool/eth0.rrd:bytes_in:AVERAGE \
DEF:bytes_out=/var/lib/rrdtool/eth0.rrd:bytes_out:AVERAGE \
CDEF:bits_in=bytes_in,8,\* \
CDEF:bits_out=bytes_out,-8,\* \
AREA:bits_in#339933:bits_in \
AREA:bits_out#aa3333:bits_out \
HRULE:0#000000

rrdtool graph /var/www/localhost/htdocs/eth0pkt.png --start -1800  \
-a PNG -t "eth0" --vertical-label "packets" \
-w 1260 -h 400 -r \
DEF:pkts_in=/var/lib/rrdtool/eth0.rrd:pkts_in:AVERAGE \
DEF:pkts_out=/var/lib/rrdtool/eth0.rrd:pkts_out:AVERAGE \
CDEF:pkts_out_negative=pkts_out,-1,\* \
LINE2:pkts_in#006600:pkts_in \
LINE2:pkts_out_negative#D73600:pkts_out \
HRULE:0#000000