This is a proof of concept of SysadminDB: a database for System Administrators Available here: https://gitlab.com/matajira/sysadmindb.
The Unique Value Proposition is that SysadminDB stores logs and can be queried using unix/linux powertools such as grep, sed and awk.

In this sense, the user does not need to learn PromQL (used by Prometheus or Loki), neither has to learn Lucene (ElasticSearch), nor any other DSL. The user, mainly a System Administrator, will be able to query logs, and produce metrics and kpis using his well known command line tools.

Demo -> Querying the DB

Dependencies

python3-django
python3-whitenoise
python3-waitress

Version with no dependencies

See the pure_python_no_dependencies branch

Setting up the web server and the tcp server

cd src && python3 main.py

Querying the db through the web server

Visit the site at localhost:8080

Demo -> Sending logs

cd src && python3 main.py
logger --server 127.0.0.1 --port 1999 -T "hello world"  

It is also possible to forward rsyslog to sysadmindb:

sudo su
echo "*.* action(type=\"omfwd\" target=\"127.0.0.1\" port=\"1999\" protocol=\"tcp\")" >> /etc/rsyslog.conf
systemctl restart rsyslog

Check the first terminal to see logs arrive and be stored.

Core of the project

The two most important technical aspects of the code are:

1. Being able to send stdout to a subprocess.
2. Allowing the user to send safely any type of bash command.

For both, the code that permits me to do it is the following:

      # Restrict PATH to a directory with only the binaries that I allow the user to execute
      my_env["PATH"] = str(pathlib.Path(os.path.realpath(__file__)).parent) + "/restricted_bin"
      # Run a restricted shell, norc and no profile to augment security
      p = Popen(["/bin/bash", "--norc", "--noprofile", "--restricted", "-c", users_command], stdout=PIPE, stdin=PIPE, stderr=PIPE, shell=False, env=my_env)
      # Insert the logs to the stdout of the subprocess
      process_result = p.communicate(input=logs.encode())