Python Partial: Code Your Intention

February 18, 2018 (Last Modified: March 04, 2018)

PARTIAL APPLICATION is a much underused tool available in the Python functools library. To demonstrate partial, we’ll use sorting by a key as an example. We’ll look at the obvious pythonic solution, a less verbose non-pythonic solution and compare these to partial. Finally we’ll wrap up with a couple more use-case examples.

Sorting by a key is the kind of operation you might have to repeat, so we should make our own custom version. In this blog I require a function to sort my blog posts by the date published, and I need to use this sort function on different filtered lists of blog posts. So it makes sense to make a function. For our example we’ll use a simpler data structure and we’ll sort a list of tuples. Our simpler example is easier to understand, but the technique really shines with complex examples.

`sorted` With `key`

def snd(tup):
    """Second Element"""
    return tup[1]

word = [('l', 3),
        ('e', 2),
        ('o', 5),
        ('h', 1),
        ('l', 3)]

print(sorted(word))
# Output (alphabetical):
##[('e', 2),
## ('h', 1),
## ('l', 3),
## ('l', 3),
## ('o', 5)]

print(sorted(word, key=snd))
# Output (desired order):
##[('h', 1),
## ('e', 2),
## ('l', 3),
## ('l', 3),
## ('o', 5)]

We want a custom sorted

Without Partial

TO CUSTOMISE sorted, we can nest it into another function. The standard, “grunt” way to do this is verbose, which makes a programmer less likely to bother writing it. The lambda version is less verbose, but it’s not considered pythonic.

Lambda expressions to define functions isn’t considered pythonic. There’s an unnecessary cognitive load for Pythonistas unfamiliar with reading this syntax. There must be a better way!

# Standard Python
def snd_sorted(tuple_list):
    """Don't forget your doc string"""
    return sorted(tuple_list, key=snd)

# Less verbose
snd_sorted2 = lambda xs: sorted(xs, key=snd)

snd_sorted(word) == snd_sorted2(word)
## True

The Better Way: Partial

THE PARTIAL application solution encodes what the programmers intention is. snd_sorted and snd_sorted2 are both functions to call sorted with the key kwarg already applied, but neither says that when read in English, both have to be parsed by a reader to see what the authors intention was.

With partial, you can read the code as "snd_sorted is the partial application of sorted where the key is snd". It has captured the authors intention and it makes a powerful tool as we’ll see in the use-case examples. It’s also short and easy to write.

from functools import partial

snd_sorted = partial(sorted, key=snd)

print(snd_sorted(word))
# Output (desired order):
##[('h', 1),
## ('e', 2),
## ('l', 3),
## ('l', 3),
## ('o', 5)]

Use Case 1: String Formatting

# Pie string
pie = "{filling} with {pastry} pastry".format
# Partial pies
steak_pie = partial(pie, filling="Steak")
puff_pie = partial(pie, pastry="puff")
# Make pies
my_pie = steak_pie(pastry="shortcrust")
print(my_pie)
## Steak with shortcrust pastry
cathys_pie = puff_pie(filling="Chicken")
print(cathys_pie)
## Chicken with puff pastry
pork_pie = pie(filling="Pork", pastry="crust")
print(pork_pie)
## Pork with crust pastry

THIS IS in response to a question on stackoverflow, P3trus was asking for a method to apply the arguments to string format at different times in the program. People suggested complex solutions involving classes and some less versatile solutions, but the answer by Saikiran Yerram deserves more upvotes for using partial.

I’ve adapted Saikiran Yerram’s answer to make it more versatile. This is a simple solution that can have the arguments applied in any order. The title of the stackoverflow post is coincidentally: “partial string formatting”: P3trus’ intention is to partially apply the format function. Unlike the other solutions offered, the partial function encodes P3trus’ intention and accomplishes the required task.

Use Case 2: Add Args Without Calling

IN MAKING semantic web applications, I need to return data encoded in different formats depending on the request’s accept headers. Does the request want HTML, JSON-LD, RDF+XML, N3 or Turtle? To do this with Flask, I can use Flask-Accept, which let’s us decorate a route with a mimetype and set a fallback.

The trouble is, I don’t want to write 5 functions to support all those mimetypes. I just want to have one function that accepts a mimetype and returns that serialised data with a fallback to HTML. Flask-Accept doesn’t officially support this, but I looked at their source code and saw the “dictionary of functions” pattern.

# Flask-Accept makes this dictionary from decorators, values are functions:
accept_handlers = {
    'application/xml+rdf': get_xml_rdf,
    'application/ld+json': get_ld_json,
    # and the rest ...
}

# Later called in this way (line 27 in commit fc99339 on 18 Nov 2015)
accept_handlers[mimetype](*args, **kwargs)

In Python, functions are first-class, which is what permits this approach. What I need to do is adapt this so that the functions take the mimetype as an argument.

Trouble is, I can’t change Flask-Accept to always pass the mimetype as an argument. This is where partial comes to the rescue.

RDF_MIMETYPES = {
    "application/ld+json": "json-ld",
    "application/rdf+xml": "pretty-xml",
    "text/n3": "n3",
    "application/x-turtle": "turtle",
    "text/turtle": "turtle"
    }

@app.route("/")
@accept_fallback
def page():
    return render_template(
        "page.html",
        content=get_content("page")
    )

page.accept_handlers = {
    mimetype: partial(page_rdf, mimetype) \
    for mimetype in RDF_MIMETYPES
}

I can partially apply the mimetype arg to the page_rdf function and use that as the values for accept_handlers. RDF_MIMETYPES has been borrowed from my model serialisation.

Now I can return 5 different mimetypes from one URL, depending on the accept headers, and it only took a dictionary generator expression, one decorator, and my page_rdf function to get it.

Conclusion

I’VE OFFERED two use-cases where partial provides a working, concise and versatile solution. However, I think its biggest benefit is that it clearly encodes the authors intention. It also allows higher levels of abstraction and code re-use, but I’ve not talked about that here because I believe those benefits will follow as you begin to see useful applications of partial all over your code.

If you are just starting out in functional programming and are looking to include functional techniques into your current work, partial is an excellent tool that you can import and start playing with immediately. Even if you’re not particularly interested in functional programming, partial will still be a very useful tool.

Until next time, happy coding. Paul

sorted With key