Make Your Python Prettier With Decorators

Many Pythonistas are familiar with using decorators, but far fewer understand what’s happening under the hood and can write their own. It takes a little effort to learn their subtleties but, once grasped, they’re a great tool for writing concise, elegant Python.

This post will briefly introduce the concept, start with a basic decorator implementation, then walk through a few more involved examples one by one.

What is a decorator

Decorators are most commonly used with the @decorator syntax. You may have seen Python that looks something like these examples.

@app.route("/home")
def home():
    return render_template("index.html")

@performance_analysis
def foo():
    pass

@property
def total_requests(self):
    return self._total_requests

To understand what a decorator does, we first have to take a step back and look at some of the things we can do with functions in Python.

def get_hello_function(punctuation):
    """Returns a hello world function, with or without punctuation."""

    def hello_world():
        print("hello world")

    def hello_world_punctuated():
        print("Hello, world!")

    if punctuation:
        return hello_world_punctuated
    else:
        return hello_world


if __name__ == '__main__':
    ready_to_call = get_hello_function(punctuation=True)

    ready_to_call()
    # "Hello, world!" is printed

In the above snippet, get_hello_function returns a function. The returned function gets assigned and then called. This flexibility in the way functions can be used and manipulated is key to the operation of decorators.

As well as returning functions, we can also pass functions as arguments. In the example below, we wrap a function, adding a delay before it’s called.

from time import sleep

def delayed_func(func):
    """Return a wrapper which delays `func` by 10 seconds."""
    def wrapper():
        print("Waiting for ten seconds...")
        sleep(10)
        # Call the function that was passed in
        func()

    return wrapper


def print_phrase(): 
    print("Fresh Hacks Every Day")


if __name__ == '__main__':
    delayed_print_function = delayed_func(print_phrase)
    delayed_print_function()

This can feel a bit confusing at first, but we’re just defining a new function wrapper, which sleeps before calling func. It’s important to note that we haven’t changed the behaviour of func itself, we’ve only returned a different function which calls func after a delay.

When the code above is run, the following output is produced:

$ python decorator_test.py
Waiting for ten seconds...
Fresh Hacks Every Day

Let’s make it pretty

If you rummage around the internet for information on decorators, the phrase you’ll see again and again is “syntactic sugar”. This does a good job of explaining what decorators are: simply a shortcut to save typing and improve readability.

The @decorator syntax makes it very easy to apply our wrapper to any function. We could re-write our delaying code above like this:

from time import sleep

def delayed_func(func):
    """Return `func`, delayed by 10 seconds."""
    def wrapper():
        print("Waiting for ten seconds...")
        sleep(10)
        # Call the function that was passed in
        func()

    return wrapper


@delayed_func
def print_phrase(): 
    print("Fresh Hacks Every Day")


if __name__ == '__main__':
    print_phrase()

Decorating print_phrase with @delayed_func automatically does the wrapping, meaning that whenever print_phrase is called we get the delayed wrapper instead of the original function; print_phrase has been replaced by wrapper.

Why is this useful?

Decorators can’t change a function, but they can extend its behaviour, modify and validate inputs and outputs, and implement any other external logic. The benefit of writing decorators comes from their ease of use once written. In the example above we could easily add @delayed_func to any function of our choice.

This ease of application is useful for debug code as well as program code. One of the most common applications for decorators is to provide debug information on the performance of a function. Let’s write a simple decorator which logs the datetime the function was called at, and the time taken to run.

import datetime
import time
from app_config import log

def log_performance(func):
    def wrapper():
        datetime_now = datetime.datetime.now()
        log.debug(f"Function {func.__name__} being called at {datetime_now}")
        start_time = time.time()

        func()

        log.debug(f"Took {time.time() - start_time} seconds")
    return wrapper


@log_performance
def calculate_squares():
    for i in range(10_000_000):
        i_squared = i**2


if __name__ == '__main__':
    calculate_squares()

In the code above we use our log_performance decorator on a function which calculates the squares of the  numbers 0 to 10,000,000. This is the output when run:

$ python decorator_test.py
Function calculate_squares being called at 2018-08-23 12:39:02.112904
Took 2.5019338130950928 seconds

Dealing with parameters

In the example above, the calculate_squares function didn’t need any parameters, but what if we wanted to make our log_performance decorator work with any function that takes any parameters?

The solution is simple: allow wrapper to accept arguments, and pass those arguments directly into  func. To allow for any number of arguments and keyword arguments, we’ve used *args, **kwargs, passing all of the arguments to the wrapped function.

import datetime
import time
from app_config import log

def log_performance(func):
    def wrapper(*args, **kwargs):
        datetime_now = datetime.datetime.now()
        log.debug(f"Function {func.__name__} being called at {datetime_now}")
        start_time = time.time()

        result = func(*args, **kwargs)

        log.debug(f"Took {time.time() - start_time} seconds")
        return result
    return wrapper


@log_performance
def calculate_squares(n):
    """Calculate the squares of the numbers 0 to n."""
    for i in range(n):
        i_squared = i**2


if __name__ == '__main__':
    calculate_squares(10_000_000) # Python 3!

Note that we also capture the result of the func call and use it as the return value of the wrapper.

Validation

Another common use case of decorators is to validate function arguments and return values.

Here’s an example where we’re dealing with multiple functions which return an IP address and port in the same format.

def get_server_addr():
    """Return IP address and port of server."""
    ...
    return ('192.168.1.0', 8080)

def get_proxy_addr():
    """Return IP address and port of proxy."""
    ...
    return ('127.0.0.1', 12253)

If we wanted to do some basic validation on the returned port, we could write a decorator like so:

PORTS_IN_USE = [1500, 1834, 7777]

def validate_port(func):
    def wrapper(*args, **kwargs):
        # Call `func` and store the result
        result = func(*args, **kwargs)
        ip_addr, port = result

        if port < 1024:
            raise ValueError("Cannot use priviledged ports below 1024")
        elif port in PORTS_IN_USE:
            raise RuntimeError(f"Port {port} is already in use")

        # If there were no errors, return the result
        return result
    return wrapper

Now it’s easy to ensure our ports are validated, we simply decorate any appropriate function with @validate_port.

@validate_port
def get_server_addr():
    """Return IP address and port of server."""
    ...
    return ('192.168.1.0', 8080)

@validate_port
def get_proxy_addr():
    """Return IP address and port of proxy."""
    ...
    return ('127.0.0.1', 12253)

The advantage of this approach is that validation is done externally to the function – there’s no risk that changes to the internal function logic or order will affect validation.

Dealing with function attributes

Let’s say we now want to access some of the metadata of the get_server_addr function above, like the name and docstring.

>>> get_server_addr.__name__
'wrapper'
>>> get_server_addr.__doc__
>>>

Disaster! Since our validate_port decorator essentially replaces the functions it decorates with our wrapper, all of the function attributes are those of wrapper, not the original function.

Fortunately, this problem is common, and the functools module in the standard library has a solution: wraps. Let’s use it in our validate_port decorator, which now looks like this:

from functools import wraps

def validate_port(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        # Call `func` and store the result
        result = func(*args, **kwargs)
        ip_addr, port = result

        if port < 1024:
            raise ValueError("Cannot use priviledged ports below 1024")
        elif port in PORTS_IN_USE:
            raise RuntimeError(f"Port {port} is already in use")

        # If there were no errors, return the result
        return result
    return wrapper

Line 4 indicates that wrapper should preserve the metadata of func, which is exactly what we want. Now when we try and access metadata, we get what we expect.

>>> get_server_addr.__name__
'get_server_addr'
>>> get_server_addr.__doc__
'Return IP address and port of server.'
>>>

Summary

Decorators are a great way to make your codebase more flexible and easy to maintain. They provide a simple way to do runtime validation on functions and are handy for debugging as well. Even if writing custom decorators isn’t your thing, an understanding of what makes them tick will be a significant asset when understanding third-party code and for utilising decorators which are already written.

21 thoughts on “Make Your Python Prettier With Decorators

  1. Thanks. I’ve been coding with python for a couple years now (small projects, nothing major) and had used decorators but never thought to dig into the details. Useful article!

  2. Out of curiosity, how many find the non-decorated, unsugared version easier to read and understand. I know I do, and I find myself mentally converting the decorated version back in to the non-decoreated one if I get to any real complications in the code. Actually I was frustrated when I first came across them, as I had to wade through pages of unhelpful expalnations before I finally understood what I was doing when I called one. Maybe life would be easier if I was the sort of person who just unthinkingly copied great chunks of code from the web without worrying what it was actually doing.

  3. Confused here, in your first decorator example print_phrase() is called which is decorated. Can you also call print_phrase() without decoration? Or once decorated always decorated. Thanks.

    1. Once a function is decorated at definition (using the @ syntax), the function is replaced by the decorated version, so yes, calling print_phrase() after it was defined with @decorated_func will always result in the decorated version. If you need to call a function without decoration, just use the slightly more verbose syntax in the example above it, (ie not using the @ syntax).

      1. OK, I get it. I’ll re-read this post from my now less-confused state and work on the other decorators. I’ll also dig into this decorator, @app.route(“/home”) which I know flask maps the ‘/home’ url to the function home(). You don’t cover this decorator style and that’s fine. Thanks.

  4. “Decorators” and “syntactic sugar” are absolutely awful ways to describe these, and I blame them entirely for the confusion.

    They’re wrappers. Call them wrappers and be done with it. Who thought “decorators! That’s a descriptive and unambiguous term!”

    1. Because the syntax is decorators, but the examples are only of wrappers. There are other uses as well, that do not wrap the function. As you can also set attributes on functions, or store the reference to a function somewhere (flask and python-dbus use this for example)

  5. “Out of curiosity, how many find the non-decorated, unsugared version easier to read and understand. I know I do” ….
    I do too. I never found a ‘need’ to use decorators (wrappers). Going to keep it that way!

    1. For small projects may be, but for large projects that need to apply common behaviour, they can be a great time saver. For example, a `permission_required` decorator can apply a common, well-tested behaviour to a function without having to introduce additional complexity into the function (and the associated test cases).

  6. I disagree with “syntactic sugar” having no meaning. It is something that does not make the code behave better but does make writing/reading it easier. That is sugar. Can do without but its nice to have. Python has quite alot of sugar in its design compared to other languages.

    Some here seem to think that decorators and sugar are the same thing. Decorators are a design pattern and they are extremely useful constructs that come from functional languages. As most tend to learn object orjented in these parts working with ideas from functional languages seems foreign and it sorta is. Functional languages have a bit steeper learning curve but they also produce value in places where OO fails.

    This is how you could regularly call a decorator without the sugar:
    calculate_squares_log_performance = log_performance(calculate_squares)
    calculate_squares_log_performance(n)

    In javascript you could do:
    log_performance(calculate_squares)(n)
    not sure it is the same in python.

    With the @ notation you can just add the decorator to the definiton of the function and it allows you to keep the name of the function the same and what not. No more nesting functions which imho are worse to work with. Each line is concise in what it does. Each function does exactly one thing.

    You can make python have static typing by creating type decorators that check inputs and outputs and throw errors when they arent valid:
    @ accepts(int,int)
    @ returns(float)
    def bar(low,high):

    TL:DR
    Decorators are a software programming pattern. Syntactic sugar is some shorthand to express some functionality in a easier to understand fashion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.