Python Infrastructure monitor

[browolf]

I wrote this because of having some issues and waiting for people to tell us there's a problem is less than ideal.

 

It uses Flask which can be installed with Pip. https://www.w3schools.com/python/python_pip.asp

If you have smoothwall, the pypi urls can be excluded from https inspection which stops certificate errors.

 

I run the script as a service on an ubuntu server plugged into the core switch. We also have port forwarding on smoothwall so it's accessible from outside. At the moment it displays on a pc/ monitor in our office but planning to get a smart tv on the wall instead. The script works equally well on windows server or 10.

 

You may wonder why it shows iterations on the page - the previous version that worked differently had an issue where it could stop working and you couldn't tell. The number represents every time the page is generated on the server so if the number is going up it's definitely working. It has a 10 second refresh which is specified in the html file.

 

It checks open ports of devices which you can find with nmap. On windows servers I'm using the rdp port 3389

 

This is what it looks like for us.

 

 

It will produce narrower tables on non windows devices aka mobiles.

 

Server.py

 

from flask import Flask, url_for, render_template, request
import datetime as dt
import socket
import os

#Config options=======================================================
#device names and IPs do not represent real devices. 
levels = [(1, 'Core'), (2, 'Hosts'), (3, 'Servers'), (4, 'Buildings')]
#Name, ip, port, level
devices = [('Dc', '10.7.7.1', 3389, 3), ('Dc', '10.7.7.1', 3389,  3), ('Newt', '10.7.7.1', 22,  4), ('Music', '10.7.7.1', 22,  4), ('Science', '10.7.7.1', 22,  4), ('Sync', '10.7.7.1', 3389,  3), ('Develop', '10.7.7.1', 3389,  3), ('Imaging', '10.7.7.1', 22,  3), ('Term01', '10.7.7.1', 3389,  3), ('VPN', '10.7.7.1', 22,  3), ('Hyworks', '10.7.7.1', 3389,  3), ('catering', '10.7.7.112', 3389,  3), ('Solidworks', '10.7.7.1', 3389,  3), ('Core01', '10.7.7.1', 22,  1), ('Internet', '8.8.8.8', 53,  1), ('Core02', '10.7.7.1', 22,  1), ('ESX1', '10.7.7.1', 80,  2), ('Esx2', '10.7.7.1', 80,  2), ('HPSan', '10.7.7.1', 443,  2), ('Print', '10.7.7.1', 3389,  3), ('Pupils', '10.7.7.1', 3389,  3), ('Sql', '10.7.7.1', 3389,  3), ('Staff', '10.7.7.1', 3389,  3), ('Ubiq', '10.7.7.1', 22,  3), ('Ups1', '10.7.7.1', 22,  3), ('Vmw', '10.7.7.1', 22,  3), ('Voicebox', '10.7.7.1', 3389,  3), ('Screens', '10.7.7.1', 22,  3), ('Main', '10.7.7.1', 22,  4), ('ICT', '10.7.7.1', 22,  4), ('DT', '10.7.7.1', 22,  4), ('StuServ', '10.7.7.1', 22,  4), ('PE', '10.7.7.1', 80,  4), ('test', '10.7.7.119', 1234,  4)]
#ipaddress of server running script. 
IPAddr="10.7.7.15"
school="My School"
#====================================================================




#reset iteration counter
iteration_counter=0


template_dir = os.path.abspath('./')
app = Flask(__name__, template_folder=template_dir)
app.debug=True

@app.route('/', methods=['GET', 'POST'])
def index():
   global iteration_counter
   htmlstr=""

   #determine device - used to determine 
   user_agent = request.headers.get("User-Agent").lower()
   cells = 10 if "windows" in user_agent else 2
   print(f"{user_agent} + {cells}")

   def test_port(a,p):
       s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
       s.settimeout(1)
       result = s.connect_ex((a,p))
       return False if result else True   

   def build_table(dfList,c):
       #c is the max number of cells per row. You want less on a phone
       def buildCell(name,result):
           col = "red" if result == False else "#C4D79B" 
           return f"</pre><table>{name} \n  </table>\n"<br>       newstr=""<br>       #create new table<br>       newstr+=("<table>\n\n")
       cellcount=1
       for item in dfList:
           newstr+=("")
           name=item[0]
           res=item[1]
           newstr+=buildCell(name,res)
           if cellcount==c:
               newstr+=("")
               cellcount=0
           cellcount+=1   
           newstr+=("") 
       newstr+=("\n</table>\n\n")<br>       return newstr <br><br>   for level in levels:      <br>       #title<br>       htmlstr+=(f"<h2>{level[1]}</h2>\n")<br>       device_results=[]<br>       filtered_devices = [tup for tup in devices if tup[3]==level[0]]<br>       for device in filtered_devices:<br>           result = test_port(device[1],device[2])<br>           device_results.append([device[0],result])<br><br>       htmlstr+=build_table(device_results,cells)<br>   iteration_counter+=1<br>   return render_template('index.html', htmlstr=htmlstr, iter=iteration_counter, school=school)<br><br>if __name__ == '__main__':<br>   app.run(IPAddr,5001)

 

index.html

   
       {{ school }}  Up Monitor
       
       
       <br />
            p {<br />
                font-family: Shanti;<br />
                font-size: 1.2em;<br />
                margin: 4px;<br />
            }<br />
<br />
            H1 {<br />
                font-family: Shanti;<br />
                font-size: 3.2em;<br />
                margin: 2px;<br />
            }<br />
<br />
            h2 {<br />
                font-family: Shanti;<br />
                margin: 10px;<br />
            }<br />
<br />
            td {<br />
                padding: 8;<br />
                font-family: Shanti;<br />
                font-size: 1.2em;<br />
                margin: 4px;<br />
                text-align: center;<br />
                padding-left: 10px;<br />
                padding-right: 10px;<br />
                <br />
            }<br />
        

   

   
       {{ school }} Up Monitor - Iter: {{ iter }}
       
       
       {{ htmlstr|safe }}
       
   
    
   
       

Edited October 6, 2022 by browolf

[dhicks]

It uses Flask which can be installed with Pip.

 

Is this code released under any particular license, or is it public domain / free for anyone to use and modify?

[browolf]

Is this code released under any particular license, or is it public domain / free for anyone to use and modify?

 

Flask?

[dhicks]

Flask?

 

I meant your code - sorry, the bit of your post I quoted was a bit random. Would you mind if I used your code as a basis for something that runs on my webserver, with some added functionality to pick up settings from an Excel / CSV file?

 

Same goes for your other recent bit of code - I'd like to be able to modify it to run on my webserver and have it available as an installable utility for schools to use.

[browolf]

I meant your code - sorry, the bit of your post I quoted was a bit random. Would you mind if I used your code as a basis for something that runs on my webserver, with some added functionality to pick up settings from an Excel / CSV file?

 

Same goes for your other recent bit of code - I'd like to be able to modify it to run on my webserver and have it available as an installable utility for schools to use.

 

 

 

I have this on Gnu General Public license here: https://github.com/browolf/Infrastructure-Monitor so you can fork it if you like.

 

The other bit of code I did to see if I could do it. Feel free to utilize those concepts. 😄

[browolf]

I meant your code - sorry, the bit of your post I quoted was a bit random. Would you mind if I used your code as a basis for something that runs on my webserver, with some added functionality to pick up settings from an Excel / CSV file?

 

Same goes for your other recent bit of code - I'd like to be able to modify it to run on my webserver and have it available as an installable utility for schools to use.

 

You may also be interested in https://github.com/browolf/network-speed-test-py

[Rooty]

Nice little script!

 

I do a lot of Python automation and scripting as part of my job, and thought I'd offer some pointers if they're of any use or interest to you.

 

Config

Separate out your config from the actual program itself. A simple way of doing this is making a config.json file, importing/parsing it and then accessing a config object like `for devices in config`, or you can cheat a bit by making a config.py file with variables, or a single 'config' multi-level dictionary, and import it at the top of the file. You can add this to your .gitignore, and that makes for better practice with config management and keeping a clean repo. It's worth adding the standard Python .gitignore file in to projects as it stops cache files and similar popping up.

 

Is it still working?

Instead of incrementing a counter, you could record the timestamp/date of the last successful port open check, and display a 'last checked' instead, to make it clearer when there's an issue, and when it occurred. If you wanted to add a feature, you could track the 'last success' date for each port, that way when a port goes down, upon each screen refresh you can display how many seconds since it was last seen like Icinga2 or other monitoring tools.

 

Sockets

Add a `socket.close()` as you're opening sockets and testing that it was successful, but then not doing anything with them, and not triggering sending the FIN packet to the peer.

 

Naming

You've got a mix of PascalCase, camelCase, snake_case, and so on (IPAddr, buildCell, iteration_counter...) as well as non-descriptive vars (a, p, c, s). I'd recommend Googling 'PEP 8 Style Guide' as that's the defacto guide that 99% of developers and projects on GitHub and so on will follow. It doesn't cost anything to be descriptive with variables, so try to avoid the single letter names as it can make things complicated at a glance.

 

If you were interested in using some off the shelf software, Icinga2 is built for this kind of thing, and you can add a TSDB backend for graphing etc. It gives you historical tracking, and the ability to monitor almost all aspects of network devices and systems, great bit of software if you've not heard of it, the docs are worth a read.

 

Hope none of these came across wrong, let me know if you had any questions!

Edited October 23, 2022 by Rooty

[browolf]

Thanks for pointers, I appreciate your advice from someone more experienced

 

I myself have the config in a database and I got rid of the database to make it easier to post online. I have been pondering a config file as then I could probably compile the main script to an exe file for people unfamiliar with python.

[Rooty]

Thanks for pointers, I appreciate your advice from someone more experienced

 

I myself have the config in a database and I got rid of the database to make it easier to post online. I have been pondering a config file as then I could probably compile the main script to an exe file for people unfamiliar with python.

No worries, glad it was of some use/help!

 

There's many ways to do it, I often load in a JSON file because I like the structure of it, and work with it a lot to make REST APIs so it feels natural. There's ways using INI files and the configparser and so on, whatever suits you best: https://docs.python.org/3/library/configparser.html

 

I'm not familiar with .exe compilation unfortunately as I'm strictly *nix! Once you get a lot more advanced you can do things like distribute your packages/programs over pip so that people can simply `pip install mymonitoringtoolthing` and install it that way (pip can install tools as well as packages).

[dhicks]

There's many ways to do it

 

I've cloned the project here:

 

https://github.com/dhicks6345789/Infrastructure-Monitor

 

I'm part-way through changing the script around somewhat, but one of the things I've already done is have it pick up config settings from an Excel (or CSV) file, which might be easier for some people to edit.

 

I'm not familiar with .exe compilation

 

This is where I found Go rather useful - the standard compiler includes cross-compilation for all the platforms Go compiles to, so whatever platform you are developing on you can produce binaries for other platforms.