Rundeck with Nagios


Pre-Request: A LAMP stack is also required.

Install Nagios 4.

https://www.zippyops.com/nagios-installation

Install Build Dependencies:

sudo yum install gcc glibc glibc-common gd gd-devel make net-snmp openssl-devel xinetd unzip -y

Create Nagios User and Group:

sudo useradd nagios

sudo groupadd nagcmd

sudo usermod -a -G nagcmd nagios

Install Nagios Core:

cd /opt

curl -L -O https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.4.6.tar.gz

tar xvf nagios-*.tar.gz

cd nagios-*

Before building Nagios, we must configure it with this command:

./configure --with-command-group=nagcmd 

Now compile Nagios with this command:

make all

Now we can run these make commands to install Nagios, init scripts, and sample configuration files:

sudo make install

sudo make install-commandmode

sudo make install-init

sudo make install-config

sudo make install-webconf

In order to issue external commands via the web interface to Nagios, we must add the webserver user, apache, to the nagcmd group:

sudo usermod -G nagcmd apache

Install Nagios Plugins:

cd /opt; curl -L -O http://nagios-plugins.org/download/nagios-plugins-2.1.1.tar.gz

tar xvf nagios-plugins-*.tar.gz

cd nagios-plugins-*

Before building Nagios Plugins, we must configure it. Use this command:

./configure --with-nagios-user=nagios --with-nagios-group=nagios --with-openssl

Now compile Nagios Plugins with this command:

Make;  sudo make install

Install NRPE

cd /opt; 

curl -L -O http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz

Extract the NRPE archive with this command

tar xvf nrpe-*.tar.gz

Then change to the extracted directory:

cd nrpe-*

Configure NRPE with these commands:

./configure --enable-command-args --with-nagios-user=nagios --with-nagios-group=nagios --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib/x86_64-linux-gnu

Now build and install NRPE and its xinetd startup script with these commands:

make all

sudo make install

sudo make install-xinetd

sudo make install-daemon-config

Open the xinetd startup script in an editor:

sudo vi /etc/xinetd.d/nrpe

only_from = 127.0.0.1, 192.168.1.20(server Name)


Restart the xinetd service to start NRPE:

sudo service xinetd restart

Configure Nagios:

Organize Nagios Configuration

sudo vi /usr/local/nagios/etc/nagios.cfg

Now find an uncomment this line by deleting the #:

cfg_dir=/usr/local/nagios/etc/servers

sudo mkdir /usr/local/nagios/etc/servers

Configure Nagios Contacts

sudo vi /usr/local/nagios/etc/objects/contacts.cfg

email  [email protected] ;   << **CHANGE THIS TO YOUR EMAIL ADDRESS ***

Configure check_nrpe Command

sudo vi /usr/local/nagios/etc/objects/commands.cfg

define command{

        command_name check_nrpe

        command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$

}


Configure Apache

sudo htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Note: If you create a user that is not named “Nagiosadmin”, you will need to edit /usr/local/nagios/etc/cgi.cfg and change all the “Nagiosadmin” references to the user you created.

sudo systemctl daemon-reload

sudo systemctl start nagios.service

sudo systemctl restart httpd.service

sudo chkconfig nagios on


Optional: Restrict Access by IP Address

sudo vi /etc/httpd/conf.d/nagios.conf

Find and comment on the following two lines by adding # symbols in front of them:

Order allow,deny

Allow from all

Then uncomment the following lines, by deleting the # symbols, and add the IP addresses or ranges (space-delimited) that you want to allow to in the Allow from the line:

#  Order deny,allow

#  Deny from all

#  Allow from 127.0.0.1

Now start Nagios and restart Apache to put the change into effect:

sudo systemctl restart nagios.service

sudo systemctl restart httpd.service

Accessing the Nagios Web Interface:

http://192.168.1.20/nagios

Monitor a CentOS 7 Host with NRPE

sudo yum install epel-release

sudo yum install nrpe nagios-plugins-all

sudo vi /etc/nagios/nrpe.cfg

allowed_hosts=127.0.0.1,192.168.1.20

Restart NRPE to put the change into effect:

sudo systemctl start nrpe.service

sudo systemctl enable nrpe.service

Add Host to Nagios Configuration

sudo vi /usr/local/nagios/etc/servers/jiraExampleCom.cfg

# Host Definition

define host {

        use                             Linux-server

        host_name                       jira.example.com

        alias                            My first Apache server

        address                         192.168.1.20

        max_check_attempts              5

        check_period                     24x7

        notification_interval           30

        notification_period                24x7

}

# Ping:

define service {

        use                              generic-service

        host_name                        jira.example.com

        service_description                 PING

        check_command                   check_ping!100.0,20%!500.0,60%

}

# SSH 

define service {

        use                             generic-service

        host_name                       jira.example.com

        service_description               SSH

        check_command                  check_ssh

        notifications_enabled              0

}

Restart Nagios server

sudo systemctl reload nagios.service

Custom Plugin for Nagios server

https://www.zippyops.com/nagios-custom-scripts

Step 1 - Install RPMForge Repository and NRPE on client VPS

rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

yum -y install nagios-nrpe

useradd nrpe && chkconfig nrpe on 

 (OR) 

yum install nagios* -y; useradd nrpe

Step 2 - Create your Bash Script

cd /usr/local/nagios/libexec/

[root@nagios libexec]# cat /localfilesystem.sh

#!/bin/bash

 countWarnings=$(df -h / | grep -v Filesystem | awk '{print $5}' | sed 's/%//g')

 if (($countWarnings<=5)); then

                echo "OK - Usage of /  $countWarnings services in Warning state"

                exit 0

        elif ((6<=$countWarnings && $countWarnings<=9)); then

                                # This case makes no sense because it only adds one warning.

                                # It is just to make an example of all possible exits.

                echo "WARNING - Usage of / $countWarnings services in Warning state"

                exit 1

        elif ((10<=$countWarnings  && $countWarnings<=99)); then

                echo "CRITICAL - Usage of / $countWarnings services in Warning state"

                servername=nagios.example.com filesystem=/

                echo "$servername $filesystem"

                curl -D - \

                -X "POST" -H "Accept: application/json" \

                -H "Content-Type: application/json" \

                -H "X-Rundeck-Auth-Token: 8h7eFvQ1Nb5JrhulUdYrHvAKzwWw18gp" \

                -d "{\"argString\":\"-servername $servername -filesystem $filesystem \"}" \

                http://192.168.1.10:4440/api/16/job/2ed8d8c9-3f46-46d3-ba5f-e71d3211a40c/executions

                exit 2

        else

                echo "UNKNOWN - $countWarnings"

                exit 3

fi

 chmod +x localfilesystem.sh

vi /usr/local/nagios/etc/objects/commands.cfg

# Custom plugins commands...

define command{

command_name localfilesystem.sh

command_line $USER1$/localfilesystem.sh

}

vi /usr/local/nagios/etc/objects/localhost.cfg

# Example - Check current warnings...

define service{

use local-service

host_name localhost

service_description Nagios Server Warnings

check_command localfilesystem

}

sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

 systemctl restart nagios.service

 CENTOS NODE AND UBUNTU NODE CONFIGURATION

How To Create Nagios Plugins With Bash On CentOS:

https://www.zippyops.com/nagios-adding-different-nodes

Step :1 - Install RPMForge Repository and NRPE on client VPS

rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm

yum -y install nagios-nrpe

useradd nrpe && chkconfig nrpe on  (OR) yum install nagios* -y; useradd nrpe

Step :2 - Create your Bash Script

Centos (and) Ubuntu

cd /usr/lib64/nagios/plugins/ (and) /usr/lib/nagios/plugins/ 

[root@nagios libexec]# cat check_warnings.sh

#!/bin/bash

countWarnings=$(df -h / | grep -v Filesystem | awk '{print $5}' | sed 's/%//g')

 if (($countWarnings<=5)); then

                echo "OK - Usage of /  $countWarnings services in Warning state"

                exit 0

        elif ((6<=$countWarnings && $countWarnings<=9)); then

# This case makes no sense because it only adds one warning.

# It is just to make an example of all possible exits.
      echo "WARNING - Usage of / $countWarnings services in Warning state"
     exit 1
elif ((10<=$countWarnings && $countWarnings<=99)); then
       echo "CRITICAL - Usage of / $countWarnings services in Warning state"
       servername=nagios.example.com filesystem=/
        echo "$servername $filesystem"
     curl -D - \
      -X "POST" -H "Accept: application/json" \
     -H "Content-Type: application/json" \
     -H "X-Rundeck-Auth-Token: 8h7eFvQ1Nb5JrhulUdYrHvAKzwWw18gp" \
     -d "{\"argString\":\"-servername $servername -filesystem $filesystem \"}" \
     http://192.168.1.10:4440/api/16/job/2ed8d8c9-3f46-46d3-ba5f-e71d3211a40c/executions
      exit 2
    else
       echo "UNKNOWN - $countWarnings"
      exit 3
 fi

chmod +x check_warnings.sh

Step :3
vi /etc/nagios/nrpe.cfg
allowed_hosts=198.168.1.20
command[usedspace_bash]=/usr/lib64/nagios/plugins/usedspace.sh
service nrpe restart (and) service nagios-nrpe-server restart
vi /usr/local/nagios/etc/objects/commands.cfg


Note: If the name already exists Don’t add this

# Custom plugins commands...
define command{
command_name check_warnings
command_line $USER1$/check_warnings.sh
}

vi /usr/local/nagios/etc/server/jira.cfg
define service {
use generic-service
host_name jira.example.com
service_description Centos.Example.com Disk usage
check_command usedspace_bash
}

(if Ubuntu)

vi /usr/local/nagios/etc/server/ubuntu.cfg
define service {
use generic-service
host_name ubuntu.example.com
service_description Ubuntu.Example.com Disk usage
check_command usedspace_bash
}

sudo /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
systemctl restart nagios.service

Screenshot_5

Nagios and Rundeck jobs for linux 

TO Rundeck

Screenshot_1

Screenshot_2


Screenshot_2a

Screenshot_3

Screenshot_4

In-line script for Linux 

[email protected]@

[email protected]@

[email protected]@

[email protected]@

var=0

if [ "$ostype"  == "Centos" ]; then

if [ $servername == "$hostname" ]; then

DATE=$(date +%Y-%m-%d-%H-%M-%S)

SOURCE=$filesystem

tar fczP /tmp/backup-$DATE.tar.gz $SOURCE

if [ $? -eq 0 ]; then

var=0

echo "var=$var"

else

var=1

echo "var=$var"

fi

fi

else

if [ $servername == "$hostname" ]; then

                DATE=$(date +%Y-%m-%d-%H-%M-%S)

                SOURCE=$filesystem

                tar fczP /tmp/backup-$DATE.tar.gz $SOURCE

                        if [ $? -eq 0 ]; then

                                var=0

                                echo "var=$var"

                        else

                                var=1

                                echo "var=$var"

                        fi

                  fi

          fi

echo "servername=$servername"

echo "filesystem=$filesystem"

echo "hostname=$hostname"

echo "var=$var"


Screenshot_5

Mail.sh common for windows and Linux

Mail server configuration 

https://www.howtoforge.com/tutorial/write-a-custom-nagios-check-plugin/


Mail server configuration 

yum install dnf -y;

dnf install postfix cyrus-sasl-plain mailx -y

sed -i 's/smtp_tls_security_level = may/smtp_tls_security_level = encrypt/' /etc/postfix/main.cf

echo "smtp_tls_security_level = encrypt" >> /etc/postfix/main.cf

echo "smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt" >> /etc/postfix/main.cf 


cat >> /etc/postfix/main.cf << EOF

relayhost = [smtp.gmail.com]:587

smtp_sasl_auth_enable = yes

smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

smtp_sasl_security_options = noanonymous

EOF

vi /etc/postfix/sasl_passwd

[smtp.gmail.com]:587 [email protected]:Zippyops@123

postmap /etc/postfix/sasl_passwd

chown root:root /etc/postfix/sasl_passwd*

chmod 600 /etc/postfix/sasl_passwd*

systemctl enable postfix --now

echo "Test Postfix Gmail Relay" | mail -s "Postfix Gmail Relay" [email protected]

tail -f /var/log/maillog


#!/bin/bash

var='0'

hostname='runcentosnode.zippyops.com'

servername='runcentosnode.zippyops.com'

filesystem='/root/google'

email='[email protected]'

  if [[ $var == 0 || $var == True ]]; then

        echo " filesystem($filesystem) sucessfully TAR on $servername" | mail -s " filesystem($filesystem) sucessfully TAR on $servername" $email

  elif [[ $var == 1 || $var == False ]]; then

        echo " filesystem($filesystem) NOT sucessfully TAR on $servername" | mail -s "filesystem($filesystem) NOT sucessfully TAR on $servername" $email

  else

echo "Execution is not for this machine"

  fi

On windows side configuration

 NSClient++ 

https://www.linuxtechi.com/add-windows-linux-host-to-nagios-server/

Python 

https://www.youtube.com/watch?v=4Rx_JRkwAjY&list=LLxcM_ASJnlfXDjf4fkE_8QA&index=2&t=356s

 After installed NSClient++ and Python

Step: 1 

In addition to these settings, execute the follow commands on your windows server (in a command prompt) to ensure the External Scripts module is correctly loaded:

cd "\Program Files\NSClient++\" 

nscp settings --activate-module CheckExternalScripts --add-defaults

nscp settings --path "/settings/external scripts" --key "allow arguments" --set true

 #Power Shell scripts 

Create New file 

C:\Program Files\NSClient++\scripts with the name check_dummy.ps1

#Content of the check_dummy.ps1 is 

# Invok python file 

C:\Python\Python38\python.exe C:\Users\silambarasan\Documents\redhat.py

#!/usr/bin/env python3.8

import wmi, sys, os, requests

def functionA(var):

    if var<=50:

        print ( "OK - - C:\\ Drive Used Space",var,"%")

        sys.exit(0)

    elif 60<=var & var<=79:

        print ( "Warning - C:\\ Drive Used Space",var,"%"  )

        headers = {

            'Accept': 'application/json',

            'Content-Type': 'application/json',

            'X-Rundeck-Auth-Token': 'GLZA6ZIBJENbPPvePyrRqpOhW8g7gAE9',

        }

        data = '{"argString":"-servername servername -filesystem redhat "}'

        response = requests.post('http://192.168.1.10:4440/api/16/job/f26a56bf-1974-4f7a-b8f4-37162e52188d/executions', headers=headers, data=data)

        sys.exit(1)

    elif 80<=var & var<=99:

        print ( "CRITICAL  - C:\\ Drive Used Space",var,"%")

        sys.exit(2)

    else:

        print ( "UKNOWN  - C:\\ Drive Used Space",var,"%")

        sys.exit(3)

    print ("Good bye!")

if __name__ == '__main__':

    c = wmi.WMI ()

    var=(int(format(c.Win32_LogicalDisk()[0].Freespace))/int(format(c.Win32_LogicalDisk()[0].Size)))*100

    var=int(var)

    functionA(var)

Open a command prompt as an *administrator* on your Windows machine and execute the following command:

Trail running

powershell.exe Set-ExecutionPolicy Bypass

cd "\Program Files\NSClient++\scripts"

powershell.exe -File check_dummy.ps1


Update nsclient.ini Configuration File 

# Open the file C:\Program Files\NSClient++\nsclient.ini in Notepad.

[/settings/external scripts/scripts]

check_dummy_ps1 = cmd /c echo scripts\\check_dummy.ps1 | powershell.exe -command -

From Nagios server side 

Test PowerShell Script From Nagios

/usr/local/nagios/libexec/check_nrpe -H 192.168.1.3 -c check_dummy_ps1 

echo $?


*** Update Command.cfg On nagios server side ***

/usr/local/nagios/etc/objects/command.cfg

define command {

    command_name    check_check_dummy_ps1

    command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -c check_dummy_ps1

}

*** Update windows.cfg On nagios server side ***

/usr/local/nagios/etc/objects/windows.cfg

define service {

    use                     generic-service

    host_name               windows-example-com

    service_description     Disk Space Check

    check_command           check_check_dummy_ps1

}

***Job for Windows.ps1 ****

Connecting windows machines to rundeck.

wget https://github.com/rundeck-plugins/rundeck-winrm-plugin/releases/download/v1.3.4/rundeck-winrm-plugin-1.3.4.jar -P /var/lib/rundeck/libext/

chown -R rundeck. /var/lib/rundeck/libext/

/etc/init.d/rundeckd restart


Step 2./ Configure a Windows Server for WinRM

– On the remote host, open a PowerShell using the Run as Administrator option and execute the following commands to enable the WinRM :

winrm qc

Make these changes [y/n]? y

winrm set winrm/config/service/Auth '@{Basic="true"}'

winrm set winrm/config/client/Auth '@{Basic="true"}'

winrm set winrm/config/service '@{AllowUnencrypted="true"}'

Get-Service -ComputerName windows.example.com -Name winrm | Select Status

Screenshot_5

Screenshot_1

[root@rundeck ~]# cat /var/lib/rundeck/projects/nagios_rundeck/etc/resource.xml

<!--?xml version="1.0" encoding="UTF-8"?-->


pip install requests &&  pip install urllib3

yum -y install python-pip

curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"

python get-pip.py

pip -V

pip 1.3.1 from /usr/lib/python2.7/site-packages (python 2.7)

pip install "pywinrm>=0.3.0"

cd /var/lib/rundeck/libext/

unzip py-winrm-plugin-2.0.6.zip

chown rundeck:rundeck py-winrm-plugin-2.0.6

pip install pywinrm

vi /etc/hosts

192.168.43.5.windows.example.com

Screenshot_1


Screenshot_2


Screenshot_3


*** filesystem.ps1 ***
#!/bin/bash
$os='Windows'
$hostname='windows-ser-com'
$filesystem='C:/rundeck/'
$servername='windows-ser-com'

echo "$hostname"
echo "$filesystem"
echo "$servername"
if("$hostname" -eq "$servername"){
if (Test-Path -path $filesystem)
{
$destination = "C:\backupdirectory\C-Rundeck_Drive_Backup $(Get-Date -Format o | ForEach-Object { $_ -replace ":", "." }).zip"
If(Test-path $destination) {Remove-item $destination}
Add-Type -assembly "system.io.compression.filesystem"
[io.compression.zipfile]::CreateFromDirectory($filesystem, $destination)
$var="True"
echo "var=$var"
}
else
{
echo "$filename not found"
$var1="False"
echo "var=$var"
}
}else{
echo "Excution is not for this machine"
$var2="False"
}

Screenshot_2



Relevant Blogs:

Rundeck window node  

Rundeck - Scheduling jobs 

Introduction to YAML  

Docker volumes

Recent Comments

No comments

Leave a Comment