Cacti: traffic analysis and monitoring network
Cacti is a complete network monitoring and data analyzing solution using RRDTool's data storage and graphing functionality. It is the most widely used monitoring tool by ISPs to see the network graphically.
Dedicated host pre configuration
Cacti has very special and fixed requirements from the host. For production systems, it must be installed on a dedicated host machine.
Hostname setup
hostname monitor echo 'hostname="monitor"' > /etc/conf.d/hostname echo "monitor" > /etc/hostname cat > /etc/hosts << EOF 127.0.0.1 monitor.venenux.net monitor localhost.localdomain localhost 151.101.128.249 dl-cdn.alpinelinux.org ::1 localhost localhost.localdomain EOF
We added the ip address of cdn Alpine Linux to reduce DNS server traffic.
Repositories and packages
Unfortunately, some commands are more complex. We must take into consideration that common commands are just busybox minimalist versions, so we must make changes to ensure we are using the real ones. The following commands install bash as well as some other system tools as shown below. If they aren't present, Catci will fail randomly:
cat > /etc/apk/repositories << EOF http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community EOF apk update apk add bash attr dialog binutils findutils readline lsof less nano curl export PAGER=less cat > /etc/apk/repositories << EOF http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community http://uk.alpinelinux.org/alpine/edge/main http://uk.alpinelinux.org/alpine/edge/community EOF apk update apk add utmps cat > /etc/apk/repositories << EOF http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community EOF apk update
Take into consideration that if one package is retrieved, it must be the only package! i.e. not all the packages. First install all the dependencies and related files from the normal repository and later install only the required extras from edge! For that, you must visit https://pkgs.alpinelinux.org/packages and search for all the requirements of the involved package from edge. like utmps
.
Requirements
Before installing Cacti, many other packages must be installed and configured.
- A web server e.g. lighttpd
- The PHP scripting language
- A database engine e.g. mariadb
- To retreive data, net-snmp tools
- To graph the data, the rrdtool package
The web server: Lighttpd installation and configuration
Cacti runs as a web program, so we need the web server configured, Because Apache2 is so well known, we will document only lighttpd. There is a lot of info available about configuration options:
- setup port, cache engine, event handler of lighttpd, with htdocs
- setup web server status page
- setup alias mod for aliasing, cgi handler, and create the directories with the correct permissions
- setup the webserver start script and start the webserver to test it
- add network back-end handler. file descriptor setup and improve file cache retrieval time
- setup https and generate self signed certificate
apk add lighttpd gamin mkdir -p /var/www/localhost/htdocs sed -i -r 's#\#.*server.port.*=.*#server.port = 80#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*server.stat-cache-engine.*=.*# server.stat-cache-engine = "fam"#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#\#.*server.event-handler = "linux-sysepoll".*#server.event-handler = "linux-sysepoll"#g' /etc/lighttpd/lighttpd.conf mkdir -p /var/www/localhost/htdocs/serverinfo sed -i -r 's#\#.*mod_status.*,.*# "mod_status",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*status.status-url.*=.*#status.status-url = "/serverinfo/server-status"#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*status.config-url.*=.*#status.config-url = "/serverinfo/server-config"#g' /etc/lighttpd/lighttpd.conf mkdir -p /var/www/localhost/cgi-bin sed -i -r 's#\#.*mod_alias.*,.*# "mod_alias",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*include "mod_cgi.conf".*# include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf mkdir -p /var/lib/lighttpd chown -R lighttpd:lighttpd /var/www/localhost/ chown -R lighttpd:lighttpd /var/lib/lighttpd chown -R lighttpd:lighttpd /var/log/lighttpd rc-update add lighttpd default rc-service lighttpd restart checkset="";checkset=$(grep 'noatime' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && \ echo listo || sed -i -r 's#server settings.*#server settings"\nserver.use-noatime = "enable"\n#g' /etc/lighttpd/lighttpd.conf checkset="";checkset=$(grep 'network-backend' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && \ echo listo || sed -i -r 's#server settings.*#server settings"\nserver.network-backend = "linux-sendfile"\n#g' /etc/lighttpd/lighttpd.conf checkset="";checkset=$(grep 'max-fds' /etc/lighttpd/lighttpd.conf);[[ "$checkset" != "" ]] && \ echo listo || sed -i -r 's#server settings.*#server settings\nserver.max-fds = 2048\n#g' /etc/lighttpd/lighttpd.conf rc-service lighttpd restart
apk add openssl mkdir -p /etc/ssl/certs/ openssl req -x509 -days 1460 -nodes -newkey rsa:4096 \ -subj "/C=VE/ST=Bolivar/L=Upata/O=VenenuX/OU=Systemas:hozYmartillo/CN=localhost" \ -keyout /etc/ssl/certs/localhost.pem -out /etc/ssl/certs/localhost.pem chmod 755 /etc/ssl/certs/localhost.pem cat > /etc/lighttpd/mod_ssl.conf << EOF server.modules += ("mod_openssl") \$SERVER["socket"] == "0.0.0.0:443" { ssl.engine = "enable" ssl.pemfile = "/etc/ssl/certs/localhost.pem" ssl.cipher-list = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM" ssl.honor-cipher-order = "enable" } \$HTTP["scheme"] == "http" { \$HTTP["host"] =~ ".*" { url.redirect = (".*" => "https://%0\$0") } } EOF sed -i -r 's#\#.*mod_redirect.*,.*# "mod_redirect",#g' /etc/lighttpd/lighttpd.conf itawxrc="";itawxrc=$(grep 'include "mod_ssl.conf' /etc/lighttpd/lighttpd.conf);[[ "$itawxrc" != "" ]] && echo listo || \ sed -i -r 's#.*include "mime-types.conf".*#include "mime-types.conf"\ninclude "mod_ssl.conf"#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#ssl.pemfile.*=.*#ssl.pemfile = "/etc/ssl/certs/localhost.pem"#g' /etc/lighttpd/lighttpd.conf rc-service lighttpd restart
The PHP: installation and configurations
Next requirement is the PHP scripting Language. Because Cacti are build with PHP, it has support for LDAP.
export phpmax=$(debver=$(cat /etc/alpine-release|cut -d '.' -f1);[ $debver -ge 6 ] && echo 7|| echo 5)
, the shel var phpmax
indicates based on Alpine version if 5 or 7 php will be used in command lines as: apk add php$phpmax
.The commit that mess is at, is: https://git.alpinelinux.org/aports/commit/community/cacti/cacti.php-fpm.conf?id=4272e802a1be191657becb739e6a248c1d0411a7 where a specific pool for php-fpm is made for cacti. It's a mess because it's not documented. We must avoid and ignore it if we are to properly configure the general pool:
apk add php7-fpm php7-bcmath php7-bz2 php7-ctype php7-curl php7-dom \ php7-enchant php7-exif php7-gd php7-gettext php7-gmp php7-iconv \ php7-imap php7-intl php7-json php7-mbstring php7-opcache php7-openssl \ php7-phar php7-posix php7-pspell php7-recode php7-session php7-simplexml \ php7-sockets php7-sysvmsg php7-sysvsem php7-sysvshm php7-tidy php7-tokenizer \ php7-xml php7-xmlreader php7-xmlrpc php7-xmlwriter php7-xsl php7-zip php7-sqlite3 \ php7-gd php7-gmp php7-ldap php7-openssl php7-pdo_mysql php7-posix php7-sockets php7-xml apk add php7-pdo php7-pdo_mysql php7-mysqli php7-pdo_sqlite php7-sqlite3 \ php7-odbc php7-pdo_odbc php7-dba
The following configurations are for high or huge loads on a 2G RAM server, for more information about configuring PHP on Alpine Linux see Production LAMP system: Lighttpd + PHP + MySQL wiki page.
sed -i -r 's|.*cgi.fix_pathinfo=.*|cgi.fix_pathinfo=1|g' /etc/php*/php.ini sed -i -r 's#.*safe_mode =.*#safe_mode = Off#g' /etc/php*/php.ini sed -i -r 's#.*expose_php =.*#expose_php = Off#g' /etc/php*/php.ini sed -i -r 's#memory_limit =.*#memory_limit = 512M#g' /etc/php*/php.ini sed -i -r 's#upload_max_filesize =.*#upload_max_filesize = 56M#g' /etc/php*/php.ini sed -i -r 's#post_max_size =.*#post_max_size = 128M#g' /etc/php*/php.ini sed -i -r 's#^file_uploads =.*#file_uploads = On#g' /etc/php*/php.ini sed -i -r 's#^max_file_uploads =.*#max_file_uploads = 12#g' /etc/php*/php.ini sed -i -r 's#^allow_url_fopen = .*#allow_url_fopen = On#g' /etc/php*/php.ini sed -i -r 's#^.default_charset =.*#default_charset = "UTF-8"#g' /etc/php*/php.ini sed -i -r 's#^.max_execution_time =.*#max_execution_time = 90#g' /etc/php*/php.ini sed -i -r 's#^max_input_time =.*#max_input_time = 90#g' /etc/php*/php.ini sed -i -r 's#.*date.timezone =.*#date.timezone = America/Panama#g' /etc/php*/php.ini sed -i -r 's|.*events.mechanism =.*|events.mechanism = epoll|g' /etc/php*/php-fpm.conf sed -i -r 's|.*emergency_restart_threshold =.*|emergency_restart_threshold = 12|g' /etc/php*/php-fpm.conf sed -i -r 's|.*emergency_restart_interval =.*|emergency_restart_interval = 1m|g' /etc/php*/php-fpm.conf sed -i -r 's|.*process_control_timeout =.*|process_control_timeout = 8s|g' /etc/php*/php-fpm.conf sed -i -r 's|^.*pm.max_requests =.*|pm.max_requests = 10000|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm.max_children =.*|pm.max_children = 12|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm.start_servers =.*|pm.start_servers = 4|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm.min_spare_servers =.*|pm.min_spare_servers = 4|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm.max_spare_servers =.*|pm.max_spare_servers = 8|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm.process_idle_timeout =.*|pm.process_idle_timeout = 8s|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*pm =.*|pm = ondemand|g' /etc/php*/php-fpm.d/www.conf mkdir -p /var/run/php-fpm7/ chown lighttpd:root /var/run/php-fpm7 sed -i -r 's|^.*listen =.*|listen = /run/php-fpm7/php7-fpm.sock|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^pid =.*|pid = /run/php-fpm7/php7-fpm.pid|g' /etc/php*/php-fpm.conf sed -i -r 's#^user =.*#user = lighttpd#g' /etc/php*/php.ini sed -i -r 's#^group =.*#group = lighttpd#g' /etc/php*/php.ini sed -i -r 's|^.*listen.owner =.*|listen.owner = lighttpd|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*listen.group =.*|listen.group = lighttpd|g' /etc/php*/php-fpm.d/www.conf sed -i -r 's|^.*listen.mode =.*|listen.mode = 0660|g' /etc/php*/php-fpm.d/www.conf rc-update add php-fpm7 default service php-fpm7 restart
After you have php ready, let's integrate it into the current preinstalled web server. We've chosen lighttpd, so the commands are:
mkdir -p /var/www/localhost/cgi-bin sed -i -r 's#\#.*mod_alias.*,.*# "mod_alias",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*include "mod_cgi.conf".*# include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*include "mod_fastcgi.conf".*#\# include "mod_fastcgi.conf"#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*include "mod_fastcgi_fpm.conf".*# include "mod_fastcgi_fpm.conf"#g' /etc/lighttpd/lighttpd.conf cat > /etc/lighttpd/mod_fastcgi_fpm.conf << EOF server.modules += ( "mod_fastcgi" ) index-file.names += ( "index.php" ) fastcgi.server = ( ".php" => ( "localhost" => ( "socket" => "/var/run/php-fpm7/php7-fpm.sock", "broken-scriptfilename" => "enable" )) ) EOF sed -i -r 's|^.*listen =.*|listen = /var/run/php-fpm7/php7-fpm.sock|g' /etc/php*/php-fpm.d/www.conf sed -i -r 'php-fpm7 restart rc-service lighttpd restart echo "<?php echo phpinfo(); ?>" > /var/www/localhost/htdocs/info.php
To test PHP is working correctly, browse to http://ipaddress/info.php
Change "ipaddrs" to the web server's ip address.
The Database: MariaDB installation and configuration
apk add mysql mysql-client tzdata mysql_install_db --user=mysql --datadir=/var/lib/mysql rc-service mariadb start mysql_tzinfo_to_sql /usr/share/zoneinfo/ | mysql -u root mysql sed -i "s|.*max_allowed_packet\s*=.*|max_allowed_packet = 100M|g" /etc/my.cnf.d/mariadb-server.cnf sed -i "s|.*bind-address\s*=.*|bind-address=127.0.0.1|g" /etc/mysql/my.cnf sed -i "s|.*bind-address\s*=.*|bind-address=127.0.0.1|g" /etc/my.cnf.d/mariadb-server.cnf cat > /etc/my.cnf.d/mariadb-server-default-charset.cnf << EOF [client] default-character-set = utf8mb4 [mysql] default-character-set = utf8mb4 EOF cat > /etc/my.cnf.d/mariadb-server-default-highload.cnf << EOF [mysqld] collation_server = utf8mb4_unicode_ci character_set_server = utf8mb4 max_heap_table_size = 32M tmp_table_size = 32M join_buffer_size = 62M innodb_file_format = Barracuda innodb_large_prefix = 1 innodb_buffer_pool_size = 512M innodb_flush_log_at_timeout = 3 innodb_read_io_threads = 32 innodb_buffer_pool_instances = 1 innodb_io_capacity = 5000 innodb_io_capacity_max = 10000 EOF rc-service mariadb restart rc-update add mariadb default
After those commands run the mysql_secure_installation
script, and answer as follows:
- Enter current password for root (for none, enter): it must be provided because we've set it previously. Correct response is
OK, successfully used password, moving on...
- Switch to unix_socket authentication [Y/n] Not used. Must be disabled, answer NO, the response will be
... skipping.
- Change the root password? [Y/n] Press "n" only if you provided a good password, otherwise change it!
- Remove anonymous users? [Y/n] In a production system, we must remove them, so answer Y the correct response is
... Success!
. - Disallow root login remotely? [Y/n] Answer Y proper response is
... Success!
. - Remove test database and access to it? [Y/n] You should removed it, so answer Y the proper response is
... Success!
. - Reload privilege tables now? [Y/n] Aanswer Y proper response is
... Success!
.
After you respond to all of the questions, restart the service with rc-service mariadb restart
The tools: net-snmp and rrdtool
agentAddress tcp:161,tcp6:[::1]:161
, by default the package adds the udp 161 protocol, so if you set it, it will only listen and show info using ipv6 only
apk add net-snmp net-snmp-tools rrdtool cat > /etc/snmp/snmpd.conf << EOF view systemonly included .1.3.6.1.2.1.1 view systemonly included .1.3.6.1.2.1.25.1 rocommunity public localhost rocommunity public default -V systemonly sysLocation Bolivar Upata Venezuela sysContact infoadmin <info@pacificnetwork.com> sysServices 72 EOF rc-update add snmpd default rc-service snmpd restart
Here we need only the commands for cacti. For more info about the topic see:
- Setting_up_traffic_monitoring_using_rrdtool_(and_snmp)
- Setting_up_monitoring_using_rrdtool_(and_rrdcollect)
- Setting_up_A_Network_Monitoring_and_Inventory_System
Cacti Installation
As of Alpine 3.12, Cacti still is in the edge branch, so first we pre-install the dependency packages. Afterward, from edge, we install only cacti.
Installing Cacti Packages
cat > /etc/apk/repositories << EOF http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/main http://dl-cdn.alpinelinux.org/alpine/v$(cat /etc/alpine-release | cut -d'.' -f1,2)/community EOF apk update apk add bash busybox coreutils net-snmp-tools perl rrdtool ttf-dejavu php7-snmp apk add cacti cacti-setup cacti-php7 cacti-lang
Cacti pre Configuration
Cacti is run under the cacti user. We temporally set the file permissions to world writeable, then later, we'll set them correctly.
cat > /etc/lighttpd/mod_cacti.conf << EOF alias.url += ( "/cacti/" => "/usr/share/webapps/cacti/" ) \$HTTP["url"] =~ "^/cacti/" { dir-listing.activate = "disable" } EOF sed -i -r 's#\#.*mod_alias.*,.*# "mod_alias",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#\#.*mod_accesslog.*,.*# "mod_accesslog",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#\#.*mod_setenv.*,.*# "mod_setenv",#g' /etc/lighttpd/lighttpd.conf sed -i -r 's#.*include "mod_cgi.conf".*# include "mod_cgi.conf"#g' /etc/lighttpd/lighttpd.conf checkssl="";checkssl=$(grep 'include "mod_cacti.conf' /etc/lighttpd/lighttpd.conf);[[ "$checkssl" != "" ]] && echo listo || sed -i -r 's#.*include "mod_cgi.conf".*#include "mod_cgi.conf"\ninclude "mod_cacti.conf"#g' /etc/lighttpd/lighttpd.conf rc-service lighttpd restart
Cacti runs under the cacti user, but web server user (apache2, lighttpd) also needs access. So enable group access and as we said, temporally set world-writeable file permissions permissions:
- grant temporary access to the web server
- Create the cacti database and populate it
- Grant Cacti MySQL user access (give it a more secure password):
- Quit from Mysql command prompt:
- Import the initial Cacti MySQL database
- set the user, password, and database name to the cacti config file
- temporarily grant world-write permission to the cacti log and lib directories
chown -R cacti:lighttpd /usr/share/webapps/cacti/;chown -R cacti:lighttpd /var/lib/cacti/ mysql -u root -p -e "CREATE DATABASE cacti;" mysql -u root -p -e "GRANT ALL ON cacti.* TO 'cactiuser'@'localhost' IDENTIFIED BY 'cactipassword';FLUSH PRIVILEGES; mysql -u root -p -e "GRANT GRANT OPTION ON cacti.* TO 'cactiuser'@'localhost';FLUSH PRIVILEGES;" mysql -u root -p -e "GRANT SELECT ON mysql.time_zone_name TO 'cactiuser'@'localhost';" mysql --user=cactiuser -p cactipassword cacti < /usr/share/webapps/cacti/cacti.sql sed -i -r 's#\$database_default.*=.*;#\$database_default = 'cacti';#g' /etc/cacti/config.php sed -i -r 's#\$database_username.*=.*;#\$database_username = 'cactiuser';#g' /etc/cacti/config.php sed -i -r 's#\$database_password.*=.*;#\$database_password = 'cactipassword';#g' /etc/cacti/config.php chmod 777 /var/log/cacti chmod 666 /var/log/cacti/*.log
Cacti web setup install
Login using:
Password= admin user= admin
Next will be prompted to change password:
change password.
In the web page click:
- -> Next
Then select new install if+ it's not selected:
- -> New install, Next
Cacti_Stats.xml.gz
template, so last "check" has no description. You must uncheck last as described here: https://github.com/Cacti/cacti/issues/3313#issuecomment-594114681 Th template can be installed later via the cli as: php -d max_execution_time=90 /usr/share/webapps/cacti/cli/import_package.php
--filename=Cacti_Stats.xml.gz
under the templates directory of the cacti install, but currently it hangs and never gets installed. Reported at https://github.com/Cacti/cacti/issues/3313#issuecomment-601508135
Then finish
- -> Finish
Add to crontab:
Cacti post configuration
chmod 775 /var/log/cacti chmod 664 /var/log/cacti/*.log
Pooler and crontab
As we said, cacti needs a crontab for pool collection of data! These steps are not necesary as of Alpine 3.7 because the cacti package installs all necesary files.
cd /etc/crontabs vi root
copy to the end of the file:
*/5 * * * * lighttpd php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1
In case you are using another web server, you have to modify the "lighttpd" user.
*/5 * * * * "web server user" php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1
Add your devices and you're ready to start monitoring!
cacti plugins
If you used the latest cacti, you must use the latest development version of each plugin especially if you're using php7 on your webserver:
00
cd /usr/share/webapps/cacti/plugins/ wget https://github.com/Cacti/plugin_mikrotik/archive/master.tar.gz -O cacti-1.2.11-plugin-monitor.tar.gz wget https://github.com/Cacti/plugin_gexport/archive/master.tar.gz -O cacti-1.2.11-plugin-gexport.tar.gz wget https://github.com/Cacti/plugin_routerconfigs/archive/master.tar.gz -O cacti-1.2.11-plugin-routerconfigs.tar.g wget https://github.com/Cacti/plugin_monitor/archive/master.tar.gz cacti-1.2.11-plugin-reportit.tar.gz wget https://github.com/Cacti/plugin_reportit/archive/master.tar.gz -O cacti-1.2.11-plugin-reportit.tar.gz
After extract all of the files, change and fix permissions:
cd /usr/share/webapps/cacti/plugins/ chown -R cacti:lighttpd /usr/share/webapps/cacti/ chown -R cacti:lighttpd /var/lib/cacti/
Go to cacti configuration and plugins via the web interface to install and enable all of them.
Re-installing Cacti
Let's suppose you make a mess of everything and want to reinstall all of it from the beginning. Let's get started:
- remove istalled packages
- remove configuration files
- remove databases and users from mysql/postgresql (here only mysql for example)
- reconfigure to not load the cacti web server configuration
- restart services because the cacti pool was erased and lighttpd must reload its config files
apk del cacti cacti-setup cacti-php7 cacti-lang rm -rf /var/log/cacti rm -rf /usr/share/webapps/cacti rm -rf /etc/cacti mysql -u root -p -e "DROP DATABASE cacti;" mysql -u root -p -e "DROP USER 'cactiuser'@'localhost'" sed -i "/include \"mod_cacti.conf\"/d" /etc/lighttpd/lighttpd.conf rc-service php-fpm7 restart rc-service lighttpd restart
After you complete all of these steps, repeat all the steps from Cacti_Installation section.