Using Python Virtual Environments

tl;dr:

Setup:

mkdir my-new-project

cd my-new-project

python -m venv env

Linux Usage:

source env/bin/activate

Powershell Usage:

./env/Scripts/Activate.ps1

Either usage

pip install requests # and anything else you need

Exit

deactivate

Favorite way to troubleshoot Python scripts

I recently discovered a great way to make sure that Python scripts give you the information you need when there's a failure. I often run Python scripts inside Docker containers. They either log locally to a file or send logs to a log aggregator (LM Logs). As such, there's not always someone monitoring the stdout pipe of the Python script. If it fails, often the best piece of information is captured using a try/except block. You can have extra data printed out to stdout or even sent out to the log aggregator. This would look something like this:

>>> try:
...   {}["shrubbery"]
... except Exception as e:
...   print(e)
...
'shrubbery'

Now that wasn't helpful was it? If the only logs we had seen were logs about successful operation then suddenly a log that says "shrubbery", we really wouldn't know what was going on. Luckily, there are a few things we can add to the exception output that clarify things:

>>> import sys
>>> try:
...   {}["shrubbery"]
... except Exception as e:
...   print(f"There was an unexpected error: {e}: \nError on line {sys.exc_info()[-1].tb_lineno}")
...
There was an unexpected error: 'shrubbery':
Error on line 2

If we import the "sys" library, it gives us some options, one of which being the line number on which the failure happened, the failure that popped us out of our try block into the except block. This still doesn't give us everything we might want, but it provides the line number where the error happened. That gives us a great place to start looking at our code to see what happened.

We can do better:

>>> import sys
>>> try:
...   {}["shrubbery"]
... except Exception as e:
...   print(f"There was an unexpected {type(e).__name__} error: {e}: \nError on line {sys.exc_info()[-1].tb_lineno}")
...
There was an unexpected KeyError error: 'shrubbery':
Error on line 2

Ah, very nice. Now we know the type of error, a KeyError, we know the key that caused the error, and we know the line in our code where the error is happening.

There are more options for outputting more data. However, I haven't found more data to be that useful. With this information, I have just what I need and no extra fluff to work through. 

Sending sysLog via Python

I had someone ask me the other day if we could ingest alarms into LogicMonitor from their app. For example, if the app had an exception raised, they'd like to send a message to LogicMonitor to open an alarm. The easiest way I could think of doing this was to have a standalone Python script that sends data via Syslog to LM. This is one way this could be done. I haven't tested this, but it's simple enough to see that it should work:

import logging
import logging.handlers
import sys
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.INFO)
destIpAddress = sys.argv[1]
destPort = sys.argv[2]
handler = logging.handlers.SysLogHandler(address = (ipAddress,514))
my_logger.addHandler(handler)
my_logger.info(sys.argv[3:].join(" "))

You'd call it like this:
> send_syslog.py 192.168.25.64 514 My application had an error in the widget creator service.