Home » Python » Python Decorator Tutorial

About Sebastian Vinci

Sebastian Vinci
Sebastian is a full stack programmer, who has strong experience in Java and Scala enterprise web applications. He is currently studying Computers Science in UBA (University of Buenos Aires) and working a full time job at a .com company as a Semi-Senior developer, involving architectural design, implementation and monitoring. He also worked in automating processes (such as data base backups, building, deploying and monitoring applications).

Python Decorator Tutorial

Sometimes, we encounter problems that require us to extend the behavior of a function, but we don’t want to change its implementation.

Some of those problems could be: logging spent time, caching, validating parameters, etc.

All these solutions are often needed in more than one function: you often need to log the spent time of every http connection; you often need to cache more than one data base entity; you often need validation in more than one function.

Today we will solve these 3 mentioned problems with Python decorators:

  • Spent Time Logging: We’ll decorate a couple functions to tell us how much time do they take to execute.
  • Caching: We’ll add cache to prevent a function from executing when its called several times with the same parameters.
  • Validating: We’ll validate a function’s input to prevent run time errors.

1. Understanding Functions

Before we can jump into decorators, we need to understand how functions actually work.

In essence, functions are procedures that return a value based on some given arguments.

def my_function(my_arg):
    return my_arg + 1

In Python, functions are first-class objects.

This means that functions can be assigned to a variable:

def my_function(foo):
    return foo + 1
my_var = my_function
print(str(my_var(1))) # prints "2"

They can be defined inside another functions:

def my_function(foo):
    def my_inner_function():
        return 1
    return foo + my_inner_function()
print(str(my_function(1))) # still prints "2"

They can be passed as parameters (higher-order functions):

def my_function(foo, my_parameter_function):
    return foo + my_parameter_function()
def parameter_function(): return 1
print(str(my_function(1, parameter_function))) # still prints "2"

And they can return other functions (also, higher-order functions):

def my_function(constant):
    def inner(foo):
        return foo + constant
    return inner
plus_one = my_function(1)
print(str(plus_one(1))) # still prints "2"

Another thing to notice, is that inner functions have access to the outer scope, that’s why we can use the parameter constant in the inner function of the last example. Also, this access is read-only, we can not modify variables from the outer scope within an inner function.

2. Jumping into decorators

Python decorators provide a nice and simple syntax to call higher-order functions. By definition, a decorator takes a function as a parameter, and returns a wrapper of that given function to extend its behavior without actually modifying it.

Given this definition we can write somthing like:

def decorator(function_to_decorate):
    def wrapper(value):
        print("you are calling {} with '{}' as parameter".format(function_to_decorate.__name__, value))
        return function_to_decorate(value)
    return wrapper


def replace_commas_with_spaces(value):
    return value.replace(",", " ")

function_to_use = decorator(replace_commas_with_spaces)
print(function_to_use("my,commas,will,be,replaces,with,spaces"))

And after execution, the output will look like:

you are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter
my commas will be replaces with spaces

So, what is actually happening here?

We are defining a higher-order function called decorator that receives a function as a parameter and returns a wrapper of that function.

The wrapper just prints to the console the name of the called function and the given parameters before executing the wrapped function. And the wrapped functions just replaces the commas with spaces.

Now we have a decorator written here. But it’s kind of annoying to define the decorator, the function and then assigning the wrapper to another variable to finally be able to use it.

Python provides some sugar syntax to make it easier to write and read, and if we re-write this decorator using it:

def decorator(function_to_decorate):
    def wrapper(value):
        print("you are calling {} with '{}' as parameter".format(function_to_decorate.__name__, value))
        return function_to_decorate(value)
    return wrapper


@decorator
def replace_commas_with_spaces(value):
    return value.replace(",", " ")

print(replace_commas_with_spaces.__name__)
print(replace_commas_with_spaces.__module__)
print(replace_commas_with_spaces.__doc__)
print(replace_commas_with_spaces("my,commas,will,be,replaces,with,spaces"))

We just annotate the function we want to wrap with the decorator function and that’s it. That function will be decorated, and the output will look the same.

wrapper
__main__
None
you are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter
my commas will be replaces with spaces

Now, debugging this can be a real pain, as the replace_commas_with_spaces function is overridden with the wrapper, so its __name__, __doc__ and __module__ will also be overridden (as seen in the output).

To avoid this behavior we will use functools.wraps, that prevents a wrapper from overriding its inner function properties.

from functools import wraps


def decorator(function_to_decorate):
    @wraps(function_to_decorate)
    def wrapper(value):
        print("you are calling {} with '{}' as parameter".format(function_to_decorate.__name__, value))
        return function_to_decorate(value)
    return wrapper


@decorator
def replace_commas_with_spaces(value):
    return value.replace(",", " ")

print(replace_commas_with_spaces.__name__)
print(replace_commas_with_spaces.__module__)
print(replace_commas_with_spaces.__doc__)
print(replace_commas_with_spaces("my,commas,will,be,replaces,with,spaces"))

And now the output will be:

replace_commas_with_spaces
__main__
None
you are calling replace_commas_with_spaces with 'my,commas,will,be,replaces,with,spaces' as parameter
my commas will be replaces with spaces

So, now we know how decorators work in python. Let’s solve our mentioned problems.

3. The Practice

So, we need to implement cache, spent time logging and validations.

Let’s combine them all by solving a bigger problem: palindromes.

Let’s make an algorithm that, given a word, will check if it’s a palindrome. If it isn’t, it will convert it to palindrome.

palindrome.py

def is_palindrome(string_value):
    char_array = list(string_value)
    size = len(char_array)
    half_size = int(size / 2)
    for i in range(0, half_size):
        if char_array[i] != char_array[size - i - 1]:
            return False
    return True


def convert_to_palindrome(v):
    def action(string_value, chars):
        chars_to_append = list(string_value)[0:chars]
        chars_to_append.reverse()
        new_value = string_value + "".join(chars_to_append)
        if not is_palindrome(new_value):
            new_value = action(string_value, chars + 1)
        return new_value
    return action(v, 0)

user_input = input("string to convert to palindrome (exit to terminate program): ")
while user_input != "exit":
    print(str(convert_to_palindrome(user_input)))
    print("------------------------------------------------------")
    user_input = input("string to check (exit to terminate program): ")

Here, we have a function called is_palindrome, which given an input, returns True if its palindrome, or False otherwise.

Then, there is a function called convert_to_palindrome which, given an input, will add just as many characters (reversed, from the beginning) as necessary to make it palindrome.

Also, there is a while that reads the user input until he inputs “exit”.

The output looks like:

string to convert to palindrome (exit to terminate program): anita lava la tina
anita lava la tinanit al aval atina
------------------------------------------------------
string to check (exit to terminate program): anitalavalatina
anitalavalatina
------------------------------------------------------
string to check (exit to terminate program): menem
menem
------------------------------------------------------
string to check (exit to terminate program): mene
menem
------------------------------------------------------
string to check (exit to terminate program): casa
casac
------------------------------------------------------
string to check (exit to terminate program): casaca
casacasac
------------------------------------------------------
string to check (exit to terminate program): exit

As you can see, it works just fine. But we have a couple problems:

  • I don’t know how long does it take it to process the input, or if its related to the length of it. (spent time logging)
  • I don’t want it to process twice the same input, it’s not necessary. (cache)
  • It’s designed to work with words or numbers, so I don’t want spaces around. (validation)

Let’s get dirty here, and start with a spent time logging decorator.

palindrome.py

import datetime
from functools import wraps


def spent_time_logging_decorator(function):
    @wraps(function)
    def wrapper(*args):
        start = datetime.datetime.now()
        result = function(*args)
        end = datetime.datetime.now()
        spent_time = end - start
        print("spent {} microseconds in {} with arguments {}. Result was: {}".format(spent_time.microseconds,
                                                                                     function.__name__, str(args),
                                                                                     result))
        return result

    return wrapper


def is_palindrome(string_value):
    char_array = list(string_value)
    size = len(char_array)
    half_size = int(size / 2)
    for i in range(0, half_size):
        if char_array[i] != char_array[size - i - 1]:
            return False
    return True


@spent_time_logging_decorator
def convert_to_palindrome(v):
    def action(string_value, chars):
        chars_to_append = list(string_value)[0:chars]
        chars_to_append.reverse()
        new_value = string_value + "".join(chars_to_append)
        if not is_palindrome(new_value):
            new_value = action(string_value, chars + 1)
        return new_value

    return action(v, 0)


user_input = input("string to convert to palindrome (exit to terminate program): ")
while user_input != "exit":
    print(str(convert_to_palindrome(user_input)))
    print("------------------------------------------------------")
    user_input = input("string to check (exit to terminate program): ")

It’s pretty simple, we wrote a decorator which returns a wrapper that gets the time before and after executor, and then calculates the spent time and logs it, with the called function, parameters and result. The output looks like:

string to check (exit to terminate program): anitalavalatina
spent 99 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina
anitalavalatina
------------------------------------------------------
string to check (exit to terminate program): exit

As you see, there is plenty information in that log line, and our implementation was not changed at all.

Now, let’s add cache:

palindrome.py

import datetime
from functools import wraps


def cache_decorator(function):
    cache = {}

    @wraps(function)
    def wrapper(*args):
        hashed_arguments = hash(str(args))
        if hashed_arguments not in cache:
            print("result for args {} was not found in cache...".format(str(args)))
            cache[hashed_arguments] = function(*args)
        return cache[hashed_arguments]
    return wrapper


def spent_time_logging_decorator(function):
    @wraps(function)
    def wrapper(*args):
        start = datetime.datetime.now()
        result = function(*args)
        end = datetime.datetime.now()
        spent_time = end - start
        print("spent {} microseconds in {} with arguments {}. Result was: {}".format(spent_time.microseconds,
                                                                                     function.__name__, str(args),
                                                                                     result))
        return result

    return wrapper


def is_palindrome(string_value):
    char_array = list(string_value)
    size = len(char_array)
    half_size = int(size / 2)
    for i in range(0, half_size):
        if char_array[i] != char_array[size - i - 1]:
            return False
    return True


@spent_time_logging_decorator
@cache_decorator
def convert_to_palindrome(v):
    def action(string_value, chars):
        chars_to_append = list(string_value)[0:chars]
        chars_to_append.reverse()
        new_value = string_value + "".join(chars_to_append)
        if not is_palindrome(new_value):
            new_value = action(string_value, chars + 1)
        return new_value

    return action(v, 0)


user_input = input("string to convert to palindrome (exit to terminate program): ")
while user_input != "exit":
    print(str(convert_to_palindrome(user_input)))
    print("------------------------------------------------------")
    user_input = input("string to check (exit to terminate program): ")

This is a very simple implementation of cache. No TTL, no thread safety. It’s just a dictionary which keys are the hash of the arguments. If no value with the given key was found, it creates it, then retrieves it. Output:

string to convert to palindrome (exit to terminate program): anitalavalatina
result for args ('anitalavalatina',) was not found in cache...
spent 313 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina
anitalavalatina
------------------------------------------------------
string to check (exit to terminate program): anitalavalatina
spent 99 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina
anitalavalatina
------------------------------------------------------
string to check (exit to terminate program): exit

There it is. The first execution with “anitalavalatina” outputs a line informing us that the result for that input was not found. But when we input it again, that line is gone. Awesome! But we still receive spaces, let’s validate that:

palindrome.py

import datetime
from functools import wraps


def validation_decorator(validator, if_invalid=None):
    def decorator(function):
        @wraps(function)
        def wrapper(*args):
            if validator(*args):
                return function(*args)
            else:
                return if_invalid
        return wrapper
    return decorator


def cache_decorator(function):
    cache = {}

    @wraps(function)
    def wrapper(*args):
        hashed_arguments = hash(str(args))
        if hashed_arguments not in cache:
            print("result for args {} was not found in cache...".format(str(args)))
            cache[hashed_arguments] = function(*args)
        return cache[hashed_arguments]
    return wrapper


def spent_time_logging_decorator(function):
    @wraps(function)
    def wrapper(*args):
        start = datetime.datetime.now()
        result = function(*args)
        end = datetime.datetime.now()
        spent_time = end - start
        print("spent {} microseconds in {} with arguments {}. Result was: {}".format(spent_time.microseconds,
                                                                                     function.__name__, str(args),
                                                                                     result))
        return result

    return wrapper


def is_palindrome(string_value):
    char_array = list(string_value)
    size = len(char_array)
    half_size = int(size / 2)
    for i in range(0, half_size):
        if char_array[i] != char_array[size - i - 1]:
            return False
    return True


def should_not_contain_spaces(*args):
    return False not in map(lambda x: " " not in str(x), args)


@spent_time_logging_decorator
@validation_decorator(should_not_contain_spaces, "input shouldn't contain spaces.")
@cache_decorator
def convert_to_palindrome(v):
    def action(string_value, chars):
        chars_to_append = list(string_value)[0:chars]
        chars_to_append.reverse()
        new_value = string_value + "".join(chars_to_append)
        if not is_palindrome(new_value):
            new_value = action(string_value, chars + 1)
        return new_value

    return action(v, 0)


user_input = input("string to convert to palindrome (exit to terminate program): ")
while user_input != "exit":
    print(str(convert_to_palindrome(user_input)))
    print("------------------------------------------------------")
    user_input = input("string to check (exit to terminate program): ")

Now, this one is a little tricky. To pass arguments to the decorator we need to wrap it. Yeah, we need a wrapper of the wrapper. Thanks to that, we can pass the validation function and a message if input is invalid. The output looks like:

string to convert to palindrome (exit to terminate program): anita lava la tina
spent 87 microseconds in convert_to_palindrome with arguments ('anita lava la tina',). Result was: input shouldn't contain spaces.
input shouldn't contain spaces.
------------------------------------------------------
string to check (exit to terminate program): anitalavalatina
result for args ('anitalavalatina',) was not found in cache...
spent 265 microseconds in convert_to_palindrome with arguments ('anitalavalatina',). Result was: anitalavalatina
anitalavalatina
------------------------------------------------------
string to check (exit to terminate program): exit

3. Download the Code Project

This was an example of how to write decorators in Python.

Download
You can download the full source code of this example here: python-decorator
(0 rating, 0 votes)
You need to be a registered member to rate this.
3 Comments Views Tweet it!
Do you want to know how to develop your skillset to become a Web Rockstar?
Subscribe to our newsletter to start Rocking right now!
To get you started we give you our best selling eBooks for FREE!
1. Building web apps with Node.js
2. HTML5 Programming Cookbook
3. CSS Programming Cookbook
4. AngularJS Programming Cookbook
5. jQuery Programming Cookbook
6. Bootstrap Programming Cookbook
and many more ....
I agree to the Terms and Privacy Policy

3
Leave a Reply

avatar
1 Comment threads
2 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
2 Comment authors
Akram /kamalSebastian Vincikamal Recent comment authors

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  Subscribe  
newest oldest most voted
Notify of
kamal
Guest
kamal

Hi,

When I run your code when you added : spent_time_logging_decorator

I got :

C:\Python27\python.exe C:/Users/alotfi/vagrantMQ/projects/decorators/palindrome.py
string to convert to palindrome (exit to terminate program): oho
Traceback (most recent call last):
File “C:/Users/alotfi/vagrantMQ/projects/decorators/palindrome.py”, line 43, in
user_input = input(“string to convert to palindrome (exit to terminate program): “)
File “”, line 1, in
NameError: name ‘oho’ is not defined

Process finished with exit code 1

Thanks.