Cacti: traffic analysis and monitoring network

From Alpine Linux
Revision as of 15:12, 18 March 2020 by Mckaygerhard (talk | contribs) (→‎Cacti Installation: fix warning message about the splitting of package)

Cacti is a complete network monitoring and data analising solution using RRDTool's data storage and graphing functionality. It is the most widely used monitoring tool by ISPs to see graphically the network.

Dedicated host preconfiguration

Cacti have very special and fixed requirements from the host, so for production systems must be installed on a dedicated host machine.

Hostname setup

hostname monitor

echo 'hostname="monitor"' > /etc/conf.d/hostname 

echo "monitor" > /etc/hostname

cat > /etc/hosts << EOF
127.0.0.1 monitor.venenux.net monitor localhost.localdomain localhost
151.101.128.249 dl-cdn.alpinelinux.org
::1 localhost localhost.localdomain
EOF

We added as plus the ip address of cdn alpine linux to avoid more packeds from a DNS server.

Repositories and packages

Unfortunatelly some commands are more complex, we must take in consideration that common commands are just busybox minimalist versions, so we must change it to normal ones: 

cat > /etc/apk/repositories << EOF
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community
EOF

apk update

apk add attr dialog binutils findutils readline lsof less nano curl

export PAGER=less

cat > /etc/apk/repositories << EOF
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community
http://uk.alpinelinux.org/alpine/edge/main
http://uk.alpinelinux.org/alpine/edge/community
EOF

apk update

apk add utmps

cat > /etc/apk/repositories << EOF
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community
EOF

apk update

Requirements

  • A web server like lighttpd
  • The PHP scripting support
  • A database engine like mariadb
  • To retreive data the net-snmp tools
  • To graphics the data the rrtool package
Warning: These complex configurations will be necessary, as Cacti is demanding in its requirements once installed to right functionality.


The web server: Lighttpd installation and configuration

Cacti runs as a web program, so we need the web server configured, due apache2 are so famous we only will document the lighttpd, becose for more used options there's already so much info: 

apk add lighttpd gamin

mkdir -p /var/www/localhost/htdocs
sed -i -r 's#\#.*server.port.*=.*#server.port          = 80#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*server.stat-cache-engine.*=.*# server.stat-cache-engine = "fam"#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#\#.*server.event-handler = "linux-sysepoll".*#server.event-handler = "linux-sysepoll"#g' /etc/lighttpd/lighttpd.conf

mkdir -p /var/www/localhost/htdocs/serverinfo
sed -i -r 's#\#.*mod_status.*,.*#    "mod_status",#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*status.status-url.*=.*#status.status-url  = "/serverinfo/server-status"#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*status.config-url.*=.*#status.config-url  = "/serverinfo/server-config"#g' /etc/lighttpd/lighttpd.conf

mkdir -p /var/www/localhost/cgi-bin
sed -i -r 's#\#.*mod_alias.*,.*#    "mod_alias",#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*include "mod_cgi.conf".*#   include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf

mkdir -p /var/lib/lighttpd
chown -R lighttpd:lighttpd /var/www/localhost/
chown -R lighttpd:lighttpd /var/lib/lighttpd
chown -R lighttpd:lighttpd /var/log/lighttpd

rc-update add lighttpd default

rc-service lighttpd restart

checkset="";checkset=$(grep 'noatime' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && echo listo || sed -i -r 's#server settings.*#server settings"\nserver.use-noatime = "enable"\n#g' /etc/lighttpd/lighttpd.conf

checkset="";checkset=$(grep 'network-backend' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && echo listo || sed -i -r 's#server settings.*#server settings"\nserver.network-backend = "linux-sendfile"\n#g' /etc/lighttpd/lighttpd.conf

checkset="";checkset=$(grep 'max-fds' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && echo listo || sed -i -r 's#server settings.*#server settings\nserver.max-fds = 2048\n#g' /etc/lighttpd/lighttpd.conf

rc-service lighttpd restart
Note: Next steps are purelly optional, just made to able to use only https to all the traffic between the host monitor of cacti and the rest of monitoring devices!
apk add openssl

mkdir -p /etc/ssl/certs/

openssl req -x509 -days 1460 -nodes -newkey rsa:4096 \
   -subj "/C=VE/ST=Bolivar/L=Upata/O=VenenuX/OU=Systemas:hozYmartillo/CN=localhost" \
   -keyout /etc/ssl/certs/localhost.pem -out /etc/ssl/certs/localhost.pem

chmod 755 /etc/ssl/certs/localhost.pem

cat > /etc/lighttpd/mod_ssl.conf << EOF
server.modules += ("mod_openssl")
\$SERVER["socket"] == "0.0.0.0:443" {
    ssl.engine  = "enable"
    ssl.pemfile = "/etc/ssl/certs/localhost.pem"
    ssl.cipher-list = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM"
    ssl.honor-cipher-order = "enable"
}
\$HTTP["scheme"] == "http" {
    \$HTTP["host"] =~ ".*" {
        url.redirect = (".*" => "https://%0\$0")
    }
}
EOF
sed -i -r 's#\#.*mod_redirect.*,.*#    "mod_redirect",#g' /etc/lighttpd/lighttpd.conf
itawxrc="";itawxrc=$(grep 'include "mod_ssl.conf' /etc/lighttpd/lighttpd.conf);[[ "$itawxrc" != "" ]] && echo listo || sed -i -r 's#.*include "mime-types.conf".*#include "mime-types.conf"\ninclude "mod_ssl.conf"#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#ssl.pemfile.*=.*#ssl.pemfile   = "/etc/ssl/certs/localhost.pem"#g' /etc/lighttpd/lighttpd.conf
rc-service lighttpd restart

The PHP: installation and configurations

Next requirement are the PHP scripting lang, becose Cacti are build with PHP, and has support for LDAP also.

Note: Cacti supports PHP5 and PHP7, in next section we will only cover PHP7 becose are the only availabe at recent Alpine versions, but if you use any older Alpine host for testing, you can use this command to detect what to install export phpmax=$(debver=$(cat /etc/alpine-release|cut -d '.' -f1);[ $debver -ge 6 ] && echo 7|| echo 5), the shel var phpmax indicates based on Alpine version if 5 or 7 php will be used in command lines as: apk add php$phpmax.

The commit that mess all are https://git.alpinelinux.org/aports/commit/community/cacti/cacti.php-fpm.conf?id=4272e802a1be191657becb739e6a248c1d0411a7 where a specific pool for php-fpm are made for cacti.. this its a mess due are not documented.. so we must avoid and ignored that due are undocumented and property configured the general pool: 


apk add php7-fpm php7-bcmath php7-bz2 php7-ctype php7-curl php7-dom \
 php7-enchant php7-exif php7-gd php7-gettext php7-gmp php7-iconv \
 php7-imap php7-intl php7-json php7-mbstring php7-opcache php7-openssl \
 php7-phar php7-posix php7-pspell php7-recode php7-session php7-simplexml \
 php7-sockets php7-sysvmsg php7-sysvsem php7-sysvshm php7-tidy php7-tokenizer \
 php7-xml php7-xmlreader php7-xmlrpc php7-xmlwriter php7-xsl php7-zip php7-sqlite3 \
 php7-gd php7-gmp php7-ldap php7-openssl php7-pdo_mysql php7-posix php7-sockets php7-xml

apk add php7-pdo php7-pdo_mysql php7-mysqli php7-pdo_sqlite php7-sqlite3 \
 php7-odbc php7-pdo_odbc php7-dba

The following configurations are for high or huge loads on a 2G RAM server, for more information about configuring PHP on Alpine linux see Production LAMP system: Lighttpd + PHP + MySQL wiki page. 

sed -i -r 's|.*cgi.fix_pathinfo=.*|cgi.fix_pathinfo=1|g' /etc/php*/php.ini
sed -i -r 's#.*safe_mode =.*#safe_mode = Off#g' /etc/php*/php.ini
sed -i -r 's#.*expose_php =.*#expose_php = Off#g' /etc/php*/php.ini
sed -i -r 's#memory_limit =.*#memory_limit = 512M#g' /etc/php*/php.ini
sed -i -r 's#upload_max_filesize =.*#upload_max_filesize = 56M#g' /etc/php*/php.ini
sed -i -r 's#post_max_size =.*#post_max_size = 128M#g' /etc/php*/php.ini
sed -i -r 's#^file_uploads =.*#file_uploads = On#g' /etc/php*/php.ini
sed -i -r 's#^max_file_uploads =.*#max_file_uploads = 12#g' /etc/php*/php.ini
sed -i -r 's#^allow_url_fopen = .*#allow_url_fopen = On#g' /etc/php*/php.ini
sed -i -r 's#^.default_charset =.*#default_charset = "UTF-8"#g' /etc/php*/php.ini
sed -i -r 's#^.max_execution_time =.*#max_execution_time = 90#g' /etc/php*/php.ini
sed -i -r 's#^max_input_time =.*#max_input_time = 90#g' /etc/php*/php.ini
sed -i -r 's#.*date.timezone =.*#date.timezone = America/Panama#g' /etc/php*/php.ini

sed -i -r 's|.*events.mechanism =.*|events.mechanism = epoll|g' /etc/php*/php-fpm.conf
sed -i -r 's|.*emergency_restart_threshold =.*|emergency_restart_threshold = 12|g' /etc/php*/php-fpm.conf
sed -i -r 's|.*emergency_restart_interval =.*|emergency_restart_interval = 1m|g' /etc/php*/php-fpm.conf
sed -i -r 's|.*process_control_timeout =.*|process_control_timeout = 8s|g' /etc/php*/php-fpm.conf

sed -i -r 's|^.*pm.max_requests =.*|pm.max_requests = 10000|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm.max_children =.*|pm.max_children = 12|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm.start_servers =.*|pm.start_servers = 4|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm.min_spare_servers =.*|pm.min_spare_servers = 4|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm.max_spare_servers =.*|pm.max_spare_servers = 8|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm.process_idle_timeout =.*|pm.process_idle_timeout = 8s|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*pm =.*|pm = ondemand|g' /etc/php*/php-fpm.d/www.conf

mkdir -p /var/run/php-fpm7/

chown lighttpd:root /var/run/php-fpm7

sed -i -r 's|^.*listen =.*|listen = /run/php-fpm7/php7-fpm.sock|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^pid =.*|pid = /run/php-fpm7/php7-fpm.pid|g' /etc/php*/php-fpm.conf
sed -i -r 's#^user =.*#user = lighttpd#g' /etc/php*/php.ini
sed -i -r 's#^group =.*#group = lighttpd#g' /etc/php*/php.ini
sed -i -r 's|^.*listen.owner =.*|listen.owner = lighttpd|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*listen.group =.*|listen.group = lighttpd|g' /etc/php*/php-fpm.d/www.conf
sed -i -r 's|^.*listen.mode =.*|listen.mode = 0660|g' /etc/php*/php-fpm.d/www.conf

rc-update add php-fpm7 default

service php-fpm7 restart

After have php ready, lest integrate into the current preinstalled web server, we already choose lighttpd so: 

mkdir -p /var/www/localhost/cgi-bin

sed -i -r 's#\#.*mod_alias.*,.*#    "mod_alias",#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*include "mod_cgi.conf".*#   include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*include "mod_fastcgi.conf".*#\#   include "mod_fastcgi.conf"#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#.*include "mod_fastcgi_fpm.conf".*#   include "mod_fastcgi_fpm.conf"#g' /etc/lighttpd/lighttpd.conf

cat > /etc/lighttpd/mod_fastcgi_fpm.conf << EOF
server.modules += ( "mod_fastcgi" )
index-file.names += ( "index.php" )
fastcgi.server = (
    ".php" => (
      "localhost" => (
        "socket"                => "/var/run/php-fpm7/php7-fpm.sock",
        "broken-scriptfilename" => "enable"
      ))
)
EOF

sed -i -r 's|^.*listen =.*|listen = /var/run/php-fpm7/php7-fpm.sock|g' /etc/php*/php-fpm.d/www.conf

sed -i -r 'php-fpm7 restart

rc-service lighttpd restart

echo "<?php echo phpinfo(); ?>" > /var/www/localhost/htdocs/info.php

To test PHP are woring correctly, browse the web server with http://ipaddress/info.php of course change "ipaddrs" with the ip of the web server.

The Database: MariaDB installation and configuration

Warning: Cacti also can run with PostgreSQL, inclusivelly are a better choice for high production and huge data systems, but we documented here mysql only due postgresql need more complex tunning parameters


Note: Also can install adminer to manage the database using web browsing, see Adminer in production LAMP systems that can manage any kind of database graphically
apk add mysql mysql-client tzdata

mysql_install_db --user=mysql --datadir=/var/lib/mysql

rc-service mariadb start

mysql_tzinfo_to_sql /usr/share/zoneinfo/ | mysql -u root mysql

sed -i "s|.*max_allowed_packet\s*=.*|max_allowed_packet = 100M|g" /etc/my.cnf.d/mariadb-server.cnf

sed -i "s|.*bind-address\s*=.*|bind-address=127.0.0.1|g" /etc/mysql/my.cnf
sed -i "s|.*bind-address\s*=.*|bind-address=127.0.0.1|g" /etc/my.cnf.d/mariadb-server.cnf

cat > /etc/my.cnf.d/mariadb-server-default-charset.cnf << EOF
[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4
EOF

cat > /etc/my.cnf.d/mariadb-server-default-highload.cnf << EOF
[mysqld]
collation_server = utf8mb4_unicode_ci
character_set_server = utf8mb4
max_heap_table_size = 32M
tmp_table_size      = 32M
join_buffer_size    = 62M
innodb_file_format  = Barracuda
innodb_large_prefix = 1
innodb_buffer_pool_size = 512M
innodb_flush_log_at_timeout = 3
innodb_read_io_threads  = 32
innodb_buffer_pool_instances = 1
innodb_io_capacity     = 5000
innodb_io_capacity_max = 10000
EOF

rc-service mariadb restart

rc-update add mariadb default

After those commands runs the mysql_secure_installation script and answer as follows:

  1. Enter current password for root (enter for none): must be provided due we already set previously. correct respond are OK, successfully used password, moving on...
  2. Switch to unix_socket authentication [Y/n] this are not the case and must be disabled, so answer NO, and response will be ... skipping.
  3. Change the root password? [Y/n] Just press "n" only if you provided a good password, otherwise just change it!
  4. Remove anonymous users? [Y/n] In any case, production system must remove it, so answer Y and proper respond mus be ... Success!.
  5. Disallow root login remotely? [Y/n] For sure answer Y and proper respond mus be ... Success!.
  6. Remove test database and access to it? [Y/n] Should be removed, so answer Y and proper respond mus be ... Success!.
  7. Reload privilege tables now? [Y/n] Aanswer Y and proper respond mus be ... Success!.

After reponse all the questions.. restart the service with rs-service mariadb restart

The tools: net-snmp and rrtool

WIP 

apk add net-snmp net-snmp-tools rrtool

In this we only neet the commands for cacti, but for more info about the topic see:

Cacti Installation

As of Alpine 3.12, Cacti still are in edge branch, so we first pre-install the depends packages and later only from edge the cacti alone.

Installing Cacti Packages

Warning: As of Alpine 3.5, Cacti was a only one package "cacti", since alpine 3.6, cacti package are split in more packages. The commit that mess all are where a specific pool for php-fpm are made for cacti.. this its a mess due are not documented.. so we must avoid and ignored that due are undocumented and property configured the general pool.


cat > /etc/apk/repositories << EOF
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main
http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community
EOF

apk update

apk add bash busybox coreutils net-snmp-tools perl rrdtool ttf-dejavu php7-snmp

apk add cacti cacti-setup cacti-php7 cacti-lang

Cacti pre Configuration

Cacti are run under cacti user, we temporally set all world write permissions to the files and later fix the permissions access. 

cat > /etc/lighttpd/mod_cacti.conf << EOF
alias.url += (
     "/cacti/"	    =>    "/usr/share/webapps/cacti/"
)
\$HTTP["url"] =~ "^/cacti/" {
    dir-listing.activate = "disable"
}
EOF

sed -i -r 's#\#.*mod_alias.*,.*#    "mod_alias",#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#\#.*mod_accesslog.*,.*#    "mod_accesslog",#g' /etc/lighttpd/lighttpd.conf
sed -i -r 's#\#.*mod_setenv.*,.*#    "mod_setenv",#g' /etc/lighttpd/lighttpd.conf

sed -i -r 's#.*include "mod_cgi.conf".*#   include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf

checkssl="";checkssl=$(grep 'include "mod_cacti.conf' /etc/lighttpd/lighttpd.conf);[[ "$checkssl" != "" ]] && echo listo || sed -i -r 's#.*include "mod_cgi.conf".*#include "mod_cgi.conf"\ninclude "mod_cacti.conf"#g' /etc/lighttpd/lighttpd.conf

rc-service lighttpd restart

Cacti runs under cacti user, but web server user (apache2, lighttpd) also needs access, so enable group access and as we said, temporally write world permissions:

  1. grant temporally access to the web server
  2. Create the cacti database and populate it
  3. Grant Cacti MySQL user access (give it a more secure password):
  4. Quit from Mysql command prompt:
  5. Import the initial Cacti MySQL database
  6. set the user, pass, and db name to the cacti config file
  7. grant temporally world permission over log and lib directories of cacti
chown -R cacti:lighttpd /usr/share/webapps/cacti/;chown -R cacti:lighttpd /var/lib/cacti/

mysql -u root -p -e "CREATE DATABASE cacti;"

mysql -u root -p -e "GRANT ALL ON cacti.* TO 'cactiuser'@'localhost' IDENTIFIED BY 'cactipassword';FLUSH PRIVILEGES;

mysql -u root -p -e "GRANT GRANT OPTION ON cacti.* TO 'cactiuser'@'localhost';FLUSH PRIVILEGES;"

mysql -u root -p -e "GRANT SELECT ON mysql.time_zone_name TO 'cactiuser'@'localhost';"

mysql --user=cactiuser -p cactipassword cacti < /usr/share/webapps/cacti/cacti.sql

sed -i -r 's#\$database_default.*=.*;#\$database_default  = 'cacti';#g' /etc/cacti/config.php
sed -i -r 's#\$database_username.*=.*;#\$database_username  = 'cactiuser';#g' /etc/cacti/config.php
sed -i -r 's#\$database_password.*=.*;#\$database_password  = 'cactipassword';#g' /etc/cacti/config.php

chmod 777 /var/log/cacti
chmod 666 /var/log/cacti/*.log

Cacti web setup install

Login using: 

Password= admin user= admin

Next will be prompted to change password: 

change password.

In the web page click:

-> Next

Then select new install in case is not selected:

-> New install, Next
Warning: take note since cacti 1.2.8 and still at cacti 1.2.10, installer have several errors for Cacti_Stats.xml.gz template, so lasted "check" have no description you must uncheck last as pointed here: https://github.com/Cacti/cacti/issues/3313#issuecomment-594114681


Then finish

-> Finish

Add to crontab:

Cacti post Configuration

chmod 775 /var/log/cacti
chmod 664 /var/log/cacti/*.log

cd /etc/crontabs
vi root

copy to the end of the file: 

*/5 * * * * lighttpd php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

In case you are using other web server have to modify the "lighttpd" user. 

*/5 * * * * "web server user" php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

Add your devices and you're ready to start monitoring!

cacti plugins

If you used lasted cacti must use lasted develop version of each plugins specially if use php7 in webserver: 

cd /usr/share/webapps/cacti/plugins/

wget https://github.com/Cacti/plugin_monitor/archive/develop.tar.gz -O cacti-1.2.11-plugin-monitor.tar.gz
wget https://github.com/Cacti/plugin_gexport/archive/develop.tar.gz -O cacti-1.2.11-plugin-gexport.tar.gz
wget https://github.com/Cacti/plugin_routerconfigs/archive/develop.tar.gz -O cacti-1.2.11-plugin-routerconfigs.tar.g
wget https://github.com/Cacti/plugin_reportit/archive/develop.tar.gz cacti-1.2.11-plugin-reportit.tar.gz
wget https://github.com/Cacti/plugin_reportit/archive/develop.tar.gz -O cacti-1.2.11-plugin-reportit.tar.gz

After extract all, change and fix permissions: 


cd /usr/share/webapps/cacti/plugins/

chown -R cacti:lighttpd /usr/share/webapps/cacti/

chown -R cacti:lighttpd /var/lib/cacti/

An go to cacti configuration and plugins in the web interface to enable and install all of them.

See Also

Retrieved from "https://wiki.alpinelinux.org/w/index.php?title=Cacti:_traffic_analysis_and_monitoring_network&oldid=17081"