I’ve used Grafana for several years. Ever since the first time I used it, I have wanted to sit down and write a server which would provide metrics to it through the Simple JSON datasource plugin. I’m happy to announce that I have finally gotten around to doing just that. The tool is called PyGraf and will be opened sourced in the very near future!
In the meantime, I’d like to share my findings as there’s not a lot of documentation on the topic.
The Simple JSON docs tells us that our backend must support the following endpoints:
/ should return 200 ok. Used for “Test connection” on the datasource config page.
/search used by the find metric options on the query tab in panels.
/query should return metrics based on input.
/annotations should return annotations.
That’s great to know, but how do we go about testing them? Let’s run through that now.
virtualenv, as the name suggests, creates virtual Python environments. If you’re familiar with server virtualisation, virtualenv acts in a similar fashion to virtual machines in that the environments share the same physical hardware, but they’re completely separated from one another.
virtualenv is very useful for when you’re working on multiple projects or using applications that require different versions of the same module. In this post I’ll demonstrate how to install and usage of virtualenv on Windows, though the process is similar for Linux too.
If you haven’t already, install virtualenv like so:
C:\>pip install virtualenv
Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
100% |################################| 1.8MB 461kB/s
Installing collected packages: virtualenv
Successfully installed virtualenv-15.1.0
As the GitHub page says, “Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2.”
The good news is that Boto 3 is extremely well documented. However, the bad news is that it is quite difficult to follow. The documentation starts with a Quickstart guide, followed by a Sample Tutorial followed then by Code Examples. This is all good stuff, though it doesn’t give you much of an understanding of how to actually use Boto 3. For example, we see things such as:
# Create CloudWatch client
cloudwatch = boto3.client('cloudwatch')
# Get the service resource
sqs = boto3.resource('sqs')
But we haven’t yet learned what a client and a resource is, nor do we see sessions mentioned until much later in the documentation. But I digress. Let’s go ahead and get started!
In my previous post I touched on the basics of how you can use pytest to test your code. In this post I’ll be covering how you can use Allure2 to prettify your pytest results.
Allure2 Adapter for pytest
The first thing we need to is install the Allure adapter for Pytest. As the documentation states, this repository contains a plugin for py.test which automatically prepares input data used to generate Allure Report.
Issue the following command to install the adapter:
sudo pip install allure-pytest
If you’re fairly new to coding chances are you’ve run into an issue where you make a minor change in one place, and then end up breaking your script in another place. In order to find out what went wrong you start adding
print statements all over the place to debug your code.
While it sound like a good idea, what you’re actually doing is relying on Python to tell you when you’ve made a syntactical error. However, what if your syntax is find, but your code is incorrect?
For example, say you accidentally changed your
addition function to a multiplication function by replacing the
+ with a
def addition(a, b):
return a * b
In my previous post, Python: Scope, I touched on the topic of Shadowing. In this post I’ll be delving deeper into it.
As Wikipedia says, variable shadowing occurs when a variable declared within a certain scope (decision block, method, or inner class) has the same name as a variable declared in an outer scope.
There are some interesting debates on whether shadowing is a bad thing or not in this StackOverflow Q&A as well as this one. In a nutshell, there are three trains of thought:
- It’s fine to use shadowing.
- You should avoid shadowing by ensuring all names are unique.
- You should avoid shadowing by using functions.
Let’s now run through each of these options to see how they work.
Scope is the term used to define the location(s) in which Python searches for a name to object mapping (e.g a variable).
As described in this StackOverflow post, Python uses the LEGB Rule to locate a definition. LEGB stands for:
- L, Local — Names assigned in any way within a function (
lambda)), and not declared global in that function.
- E, Enclosing-function locals — Name in the local scope of any and all statically enclosing functions (
lambda), from inner to outer.
- G, Global (module) — Names assigned at the top-level of a module file, or by executing a
global statement in a
def within the file.
- B, Built-in (Python) — Names preassigned in the built-in names module
In a nutshell, Python will first look at the local scope for a name to object mapping (e.g
people = 5). If it cannot find one, it will continue going up the hierarchy until it finds one. If it doesn’t find a mapping, it will raise an exception.
To shed some more light on this let’s take a step back and analyse each of the points listed above. I’ll do so in reverse order because that is the way we write Python code, as you’ll see in a moment.
A lot of Python books often mention that “everything in Python is an object”, and “objects are first class citizens”, but they don’t always explain what that these things actually mean. Let’s try to fix that up now.
Everything in Python is an Object
Dive Into Python gives a great explanation:
Different programming languages define “object” in different ways. In some, it means that all objects must have attributes and methods; in others, it means that all objects are subclassable. In Python, the definition is looser; some objects have neither attributes nor methods (more on this in Chapter 3), and not all objects are subclassable (more on this in Chapter 5). But everything is an object in the sense that it can be assigned to a variable or passed as an argument to a function (more in this in Chapter 4).
This is so important that I’m going to repeat it in case you missed it the first few times: everything in Python is an object. Strings are objects. Lists are objects. Functions are objects. Even modules are objects.
There are plenty of articles on the internet that attempt to explain what
if __name__ == "__main__" is and what it does, but (in my humble opinion), the examples are too complex more often than not. With that in mind, this post is aimed at being the most simplest explanation on the planet! :)
What does it do?
This statement is used when you want your code to be used as both a standalone script, as well as a module that can be imported and used by other scripts.
For example, if it is run as a standalone script, you may want to provide a menu to ensure users input all of the necessary information. On the other hand, if it is being imported as a module, perhaps you’d like to avoid the menu all together and instead only use the functions contained the script (e.g
func1 in the example below).
Installing Git on Windows is very similar to installing it on Linux. That might not come as a surprise though because the tools we’ll be using in this post are specifically designed to allow Windows users to utilise Linux packages without needing to install a VM.
The first thing you’ll need to do is download Git for Windows. Once you have done that, install it using all of the default settings. After completing the installation, you will find that you’ve now got three Git applications: