Writing a Grafana backend using the Simple JSON datasource & Flask

I’ve used Grafana for several years. Ever since the first time I used it, I have wanted to sit down and write a server which would provide metrics to it through the Simple JSON datasource plugin. I’m happy to announce that I have finally gotten around to doing just that. The tool is called PyGraf and will be opened sourced in the very near future!

In the meantime, I’d like to share my findings as there’s not a lot of documentation on the topic.

Backend URLs

The Simple JSON docs tells us that our backend must support the following endpoints:

  • / should return 200 ok. Used for “Test connection” on the datasource config page.
  • /search used by the find metric options on the query tab in panels.
  • /query should return metrics based on input.
  • /annotations should return annotations.

That’s great to know, but how do we go about testing them? Let’s run through that now.

Continue reading

Installing & using Python virtualenv

virtualenv, as the name suggests, creates virtual Python environments. If you’re familiar with server virtualisation, virtualenv acts in a similar fashion to virtual machines in that the environments share the same physical hardware, but they’re completely separated from one another.

virtualenv is very useful for when you’re working on multiple projects or using applications that require different versions of the same module. In this post I’ll demonstrate how to install and usage of virtualenv on Windows, though the process is similar for Linux too.

If you haven’t already, install virtualenv like so:

Continue reading

Python: Demystifying AWS’ Boto3

As the GitHub page says, “Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2.”

The good news is that Boto 3 is extremely well documented. However, the bad news is that it is quite difficult to follow. The documentation starts with a Quickstart guide, followed by a Sample Tutorial followed then by Code Examples. This is all good stuff, though it doesn’t give you much of an understanding of how to actually use Boto 3. For example,  we see things such as:

and,

But we haven’t yet learned what a client and a resource is, nor do we see sessions mentioned until much later in the documentation. But I digress. Let’s go ahead and get started!

Continue reading

Allure2: A GUI for your code tests

In my previous post I touched on the basics of how you can use pytest to test your code. In this post I’ll be covering how you can use Allure2 to prettify your pytest results.

Allure2 Adapter for pytest

The first thing we need to is install the Allure adapter for Pytest. As the documentation states, this repository contains a plugin for py.test which automatically prepares input data used to generate Allure Report.

Issue the following command to install the adapter:

Continue reading

Testing your code with pytest

If you’re fairly new to coding chances are you’ve run into an issue where you make a minor change in one place, and then end up breaking your script in another place. In order to find out what went wrong you start adding print statements all over the place to debug your code.

While it sound like a good idea, what you’re actually doing is relying on Python to tell you when you’ve made a syntactical error. However, what if your syntax is find, but your code is incorrect?

For example, say you accidentally changed your addition function to a multiplication function by replacing the + with a *:

Continue reading

Python: Shadowing

In my previous post, Python: Scope, I touched on the topic of Shadowing. In this post I’ll be delving deeper into it.

As Wikipedia saysvariable shadowing occurs when a variable declared within a certain scope (decision block, method, or inner class) has the same name as a variable declared in an outer scope.

There are some interesting debates on whether shadowing is a bad thing or not in this StackOverflow Q&A as well as this one. In a nutshell, there are three trains of thought:

  1. It’s fine to use shadowing.
  2. You should avoid shadowing by ensuring all names are unique.
  3. You should avoid shadowing by using functions.

Let’s now run through each of these options to see how they work.

Continue reading

Python: Scope

Scope is the term used to define the location(s) in which Python searches for a name to object mapping (e.g a variable).

As described in this StackOverflow post, Python uses the LEGB Rule to locate a definition. LEGB stands for:

  • L, Local — Names assigned in any way within a function ( def or lambda)), and not declared global in that function.
  • E, Enclosing-function locals — Name in the local scope of any and all statically enclosing functions ( def or lambda), from inner to outer.
  • G, Global (module) — Names assigned at the top-level of a module file, or by executing a global statement in a def within the file.
  • B, Built-in (Python) — Names preassigned in the built-in names module open,range,SyntaxError,...

In a nutshell, Python will first look at the local scope for a name to object mapping (e.g people = 5). If it cannot find one, it will continue going up the hierarchy until it finds one. If it doesn’t find a mapping, it will raise an exception.

To shed some more light on this let’s take a step back and analyse each of the points listed above. I’ll do so in reverse order because that is the way we write Python code, as you’ll see in a moment.

Continue reading

Python: Everything is an Object & First Class Citizens

A lot of Python books often mention that “everything in Python is an object”, and “objects are first class citizens”, but they don’t always explain what that these things actually mean. Let’s try to fix that up now.

Everything in Python is an Object

Dive Into Python gives a great explanation:

Different programming languages define “object” in different ways. In some, it means that all objects must have attributes and methods; in others, it means that all objects are subclassable. In Python, the definition is looser; some objects have neither attributes nor methods (more on this in Chapter 3), and not all objects are subclassable (more on this in Chapter 5). But everything is an object in the sense that it can be assigned to a variable or passed as an argument to a function (more in this in Chapter 4).

This is so important that I’m going to repeat it in case you missed it the first few times: everything in Python is an object. Strings are objects. Lists are objects. Functions are objects. Even modules are objects.

Continue reading

Python: if __name__ == “__main__”

There are plenty of articles on the internet that attempt to explain what  if __name__ == "__main__" is and what it does, but (in my humble opinion), the examples are too complex more often than not. With that in mind, this post is aimed at being the most simplest explanation on the planet! :)

What does it do?

This statement is used when you want your code to be used as both a standalone script, as well as a module that can be imported and used by other scripts.

For example, if it is run as a standalone script, you may want to provide a menu to ensure users input all of the necessary information. On the other hand, if it is being imported as a module, perhaps you’d like to avoid the menu all together and instead only use the functions contained the script (e.g func1 in the example below).

Continue reading

Installing Git on Windows

Installing Git on Windows is very similar to installing it on Linux. That might not come as a surprise though because the tools we’ll be using in this post are specifically designed to allow Windows users to utilise Linux packages without needing to install a VM.

Installation

The first thing you’ll need to do is download Git for Windows. Once you have done that, install it using all of the default settings. After completing the installation, you will find that you’ve now got three Git applications:

  • Git Bash
  • Git CMD
  • Git GUI

Continue reading