I’ve tried to cram quite a bit into one post but I’ve used headings liberally so you can hop around. My intention is to give both an in-depth look at how to use the Python map() function and to introduce fundamental concepts of functional programming (FP). I’ll also tell you when not to use Python’s map() and how to use generator expressions instead.

You might be surprised to be reading about functional programming and Python in the same post. While it’s true that Python is not a pure functional programming language, understanding functional concepts can help you become a better, more efficient coder.

You don’t need to know about programming paradigms to understand any of this post, and I’ll give a high-level overview of the main paradigms soon.

If you write in a modern high-level language, you’ve likely used a procedural or object-oriented style. Scripts that manipulate data such as in a Pandas DataFrame are types of procedural programming. Writing classes, which include data and methods that act on them, is a type of object-oriented programming.

Hopefully by the end of this post you’ll know a thing or two, including:

• What functional, imperative, procedural and object-oriented programming are
• The foundational concepts of functional programming
• How to use Python map()
• Why functional programming makes big data easier
• What first-class functions are in Python
• How iterables, iterators, list comprehensions and generator expressions work
• When you should use Python map()
• When you should use list comprehensions and generator expressions

We need to get this out of the way: There are a lot of opinions when it comes to programming paradigms, and sometimes it’s hard to sort the facts from people’s preferences. You’ll start forming your own opinions, but here we’ll stick to the least controversial topics and will refrain from making generalizations.

What Is Functional Programming?

Let’s talk about the types of programming you’re likely used to first, then we’ll contrast it to functional programming.

If you learned to program through Python, you’ve probably been programming in an imperative style. Imperative programming refers to code that changes state of the program. A program’s state represents information about the world of the program that subsequent code can change.

If a program has to remember user interactions or the order that customer orders come in, that’s program state. Something as simple as a variable holding a value is a program state. And if your program is required to remember state and your code changes that state, then your code is using an imperative style.

You may be more familiar with the terms object-oriented programming (OOP) and procedural programming. Both OOP and procedural programming are types of imperative programming.

Functional programming is not at all like OOP or procedural programming! It’s a type of declarative programming. If this is all a bit much to remember, refer to the table below.

Imperative Versus Declarative Programming

The relevant hierarchy of programming paradigms goes like this. Imperative programming, which changes state, is contrasted to declarative programming, which is concerned with logic more than flow. Flow is the order of operations happening in a program.

Within imperative, you have both object-oriented programming (OOP) and procedural programming. Object-oriented programming relies heavily on objects, which are an abstraction for holding data and methods that act on that data. In Python and many other languages, objects are created using classes.

Procedural programming is typified by a lot of functions, also called procedures, and the passing of state through the use of return values. Procedural programming doesn’t combine data and methods that act on the data into an object as OOP does.

Under the umbrella of declarative programming, there’s functional programming (finally!), logic programming and others. In functional programming, functions are treated more similarly to how they are in math: Functions exist for the sake of computation and don’t change the program’s state.

The rest of this post will address functional programming fundamentals. Logic programming, another type of declarative programming, is probably less familiar and is used with languages like Prolog. In logic programming, code is written using formal logic statements. It’s associated with artificial intelligence and theorem proving.

Paradigm	Related to	Definition	Typical use cases
Procedural	Type of imperative	Relies on modularity and procedure/function calls	Python scripting, C++
Object oriented (OOP)	Type of imperative	Based on classes and objects (made up of data and methods that act on the data)	Many high-level programming languages, such as Python and C++
Functional	Type of declarative	Functions are treated like mathematical functions; state is unchanged and data is immutable	Big data; Scala, Haskell
Logic	Type of declarative	Programs specify what needs to happen, but not how to do it	Theorem proving
Imperative	Procedural is a subtype	Changes program state; uses mutable data	Scripting, for example with Pandas in Python
Declarative	Logic is a subtype	Describes logic of a computation but not its control flow. Control flow are things like for loops and if/then statements	SQL and other database query languages

Note: This table is not exhaustive but does cover the most important programming paradigms you’ll run into.

Pure Functions Have No Side-Effects

Functional programming does not change state and, in functional programming, functions don’t have side-effects. "Side-effects" is jargon you’ll hear quite a bit in functional programming and means the data or variables that a function changes, beyond the value it’s returning to you. In a function with no side-effects you’ll be calling a function because you want the value it returns and you won’t be expecting the function to change other data or variables. No side-effects will be the biggest change you’ll see in functional programming if imperative programming is your background.

Here’s an example of a function with a side effect because it changes the list passed to it:

import random

def double_list_return_sum(a_list):
    a_list.append(random.randrange(20))
    return sum(a_list)
 
my_list = [1, 2, 3, 4]

print("Function output:", double_list_return_sum(my_list)) # Prints 15
print("my_list after the side effect: ", my_list) # Prints [1, 2, 3, 4, 16]

The above code did two things:

1. It summed up the list passed to it and returned that sum.

2. It appended another element, a randomly generated integer, to the list passed to it.

The second part is called a side effect of the function: It changed a mutable data type and it changed it in a way where the output could be different even on the same exact input. Appending a random number is a toy example, but you can imagine a function that returns different things depending on a file it reads or input from a user.

Now here’s an example of a pure function:

def factorial(n):
    '''
    A factorial takes a number and multiplies it
    This is the formula: n*(n-1)*(n-2)*...*1
    '''
    if n <= 1:
        return 1
    else:
        return factorial(n-1) * n

factorial(5) # Returns 120

In this case, the function didn’t amend any data structures, it only returned to you the factorial of the number you passed it. You could use this function during any time that the program is running, and no matter what else had happened earlier, it would always return the same exact output for a given input. This is an example of a pure function.

In functional programming, you think of functions more like the functions in math class: You always get the same exact output for a given input. That is, if you have a function f(x), if you put in f(5), you’ll always get the same output. These functions are in contrast with functions in imperative programming where a program’s state might affect your output.

In imperative programming, such as OOP or procedural programming, f(5) might return to you 25 on even days and 35 on odd days and that could be completely correct. Or a function, in addition to returning to you a value, might write out to a file or amend a list or DataFrame that you passed in to it. Not so in functional programming.

With a pure function, you can remove a function call and replace it with the value the function would have returned. Being able to replace a function call, or really any expression, with its calculated value is called referential transparency.

Mutable Versus Immutable Data Types: Lists and Tuples

Let’s keep introducing functional programming jargon. In functional programming, you use immutable collections. In Python you might be familiar with collections such as lists, tuples and NumPy arrays. A collection is just a container for more primitive datatypes. If you were to program in an imperative way, you’d use lists and you might add or remove from your list. This is because lists are a mutable datatype. Tuples are similar but are an immutable datatype.

In functional programming, you mostly, but not exclusively, use immutable data types. You can’t change the contents of immutable data types. That might sound restrictive. But, you’re allowed to do things like create new copies of a collection and, while copying, make changes to the data. In that case, you’re not mutating the collection and you can protect your data from client code accidentally changing it.

Is Python a FP Language? Short Answer: No

Within the Python community, people have various opinions about whether it’s best suited to be a procedural or object-oriented language. In this post, we’re going to put that debate aside and introduce how to use Python’s functional programming capabilities. There’s no one correct way to use Python and you should use procedural, OOP and functional paradigms when they’re appropriate, even within the same program.

If the question was, "Is Python a functional programming language?" the answer would be "No." If you wanted to write purely functional code, you might find a different language like Scala or Haskell more appropriate. Though even Scala itself isn’t purely functional! But of course things are more nuanced. Python has a bunch of useful functional-type methods and capabilities built in and they can make your code easier to read and easier to test and debug. We’ll talk a bit more about why functional programming makes programs easier to test in the conclusion, where we give a bit of a defense of FP in Python.

Tour of Functional Programming in Python

You’re now ready for quick tour of the things that make Python a little functional. These concepts apply both to Python and more generally to other functional languages, though the implementation might look different in other languages.

Functions Are First-Class Objects in Python

Another hugely important characteristic of functional programming languages is that they treat functions as first-class objects. You’ll see what that means in a second, but it’s important to note that Python, which is not a pure functional programming language, actually did treat functions as objects from the very beginning of the language!

For a function to be treated like a first-class object means that you can put a function anywhere you can put any primitive, such as a number or a list. Want to assign a function to a variable? You can do so in Python because functions are first-class objects. Want to return a function as a result from another function? You can again do this because functions are first-class objects.

Let’s look at an example of storing a function in a variable:

def a_squared_plus_b(a,b):
    return (a**2)+b

my_func = a_squared_plus_b
print(my_func(2,4)) # Prints 8

In the above, we created a function called a_squared_plus_b() that took in two parameters and returned the first one squared added to the second. We then created a new variable, called my_func, and assigned the function to the variable name.

Note that we didn’t include parentheses after a_squared_plus_b in my_var = a_squared_plus_b. This is because including () means actually running the function, but here you just wanted to put the function itself in a variable. Adding () would add the function to the call stack and execute the code in the function, but we don’t want to do that yet.

Then we called a_squared_plus_b, but through my_func, which now holds the function! Seems crazy at first, but that’s the power of functions as first-class objects.

Here’s an example of passing a function as a parameter to another function. You’ll notice the use of the operator library, which includes several methods for doing math operations:

import operator # We'll be using this package a few times

# A demonstration of some methods in operator
operator.add(2,3) # Returns 5
operator.mul(2,3) # Returns 6

def math_operation(fun,a,b):
    return fun(a,b)

print(math_operation(operator.add,3,5)) # Returns 8
print(math_operation(operator.mul,3,5)) # Returns 15

In the above, the operator module helped us because it gives methods for doing mathematical operations like add and multiply. math_operation() is a user-defined function that takes another function as its first parameter, plus the two numbers that will be used in the operation. Side note: People often use fun as a variable name to represent a function, and that’s what we’ve used in the parameter list to math_operation().

You could use math_operation() to either add or multiply, depending on which function you pass it. It might seem like overkill at this point, though. Why not just use operator.mul() when you need that and operator.add() when you need it? Why bother with this extra architecture of math_operation()?

This example is simple but demonstrates a powerful tenet of clean programming as it helps you add functionality down the road without cluttering your interface. Discussion of interfaces can get controversial with a lot of different opinions and is beyond the scope of this post. What you need to know now, though, is that functions are just like any other object in Python and can be passed to other functions.

Benefits of First-Class Functions: Decorators, Closure, Higher-Order Functions, and Function Factories

There are a lot of cool things you can do in Python because functions are first-class objects. The list includes: function decorators, higher-order functions, closure and, relatedly, function factories.

Here’s a quick summary of each:

• Function decorators are a way to extend what a function can do without changing the function itself. This is accomplished by wrapping an existing function with a function decorator (another function). Python also has syntactic sugar (syntax that makes doing things easier or prettier) to accomplish this.

• Higher-order functions either take functions as input or return functions. math_operation() above was an example of a higher-order function that took an operator as its input.

• Functions can be defined inside other functions. The idea of closure is that an inner function has access to the scope of its enclosing function.

• Function factories refers to using the concept of closure and inner functions to create new functions using a function. It’s kind of weird at first, but it’s useful when you don’t want your client code to know how you make your functions. This allows you to change the implementation behind the scenes as necessary without breaking the code that uses the functions you’ve made from the factory.

Python’s Immutable Objects: Tuples

We’ve said that FP relies on immutable objects. And Python already has immutable types that you’ve likely worked with before: tuples. Tuples are like lists, just without being modifiable. You use them when you want to protect your data from accidental, or even intentional, modification, especially by client code.

What Is Python map()?

Python map() is a built-in method that takes in a function and an iterable. It then applies the function passed into it to every element of the iterable passed in. The reason that Python map() can take a function as a parameter is that functions are first-class objects in Python, which is a foundational FP concept. The real power of Python map() comes from the fact that you get to write your own functions to pass into map(). We’re going to show you a quick Python map() example, then take a detour to review iterables and list comprehensions, then we’ll come back for more details on Python map() itself.

Write Your First Python map()

You’re now ready to code your first Python map() example. Note that this is a toy example and might not be the best solution to this particular problem, but it’s real enough to introduce the syntax and concepts. Remember, Python map() takes in:

1. A function to apply to each item in an iterable.
2. An iterable, such as a list.

If you need to double every item in your list, you might write it this way:

def double(x):
    return 2*x

my_list = [1, 2, 3, 4]

double_list = list(map(double, my_list))
# You could write this in two lines to be even more clear:
# double_map = (map(double, my_list)
# double_list = list(double_map)

print(double_list) # Prints [2, 4, 6, 8]

In the above code, double() takes in a number and returns it multiplied by two. When double() is passed to Python map(), that’s when things get functional: Being able to pass a function as a parameter to another function is possible because functions are first-class objects, a functional programming requirement.

What Python map() returns is a map object. In order to get the returned map object back to a list, you have to wrap the output of Python map() with list() to do the casting. At the end, we’ve printed the list to confirm it’s working.

Short History of Python map()

The Python map() function’s relation to functional programming is straightforward: it’s possible because functions are first-class objects and because of higher-order functions, or functions that take functions as inputs or return functions.

How did Python map() come to be, given that Python is not a functional programming language? In 1994, after several years of users hacking solutions to accomplish similar tasks, map, reduce, filter and anonymous (lambda) functions were added to Python. Users were coming to Python from other, more functional languages, and so were porting their own expectations for how things should work with them.

According to Guido van Rossum, the originator of Python, these elements were added in response to how people were using the language, but have since been "superseded" by list comprehensions and generator expressions, which we’ll talk about below. But don’t worry, Python map() and such are not being removed from Python.

Review of Python Iterables, Iterators and Generator Functions

We’ll quickly review iterables as they’re the type of object Python map() works on, but we’ll leave the details to other resources.

If you’ve used a for loop in Python, you’ve probably used an iterable without knowing it. An iterable is a particular kind of object in Python: it has an iter() method. And that iter() method returns an interator, which itself has a next() method that will give us the next item in the traversal of our list. When the iterator runs out of times, it throws a StopIteration exception. So with that next() method of an iterator, you can traverse that iterable, going from item to item, in a much easier way than you’d be able to do in a language like C.

There’s a bit of subtlety between an iterable and an iterator, but what you need to understand for now is that an iterable is the container object that you’ll be traversing with map().

Let’s make iterables and their benefit in Python concrete. For example, if you knew only a little programming and were trying to write a loop to traverse a list and print out its contents, you might do something like this:

my_list = [1, 2, 3, 4]

for i in range(len(my_list)):
    print(my_list[i])

# Prints
# 1
# 2
# 3
# 4

But because Python’s built-in lists are an iterable, we can refactor our loop in a way that’s kind of magical seeming:

my_list = [1, 2, 3, 4]

for item in my_list:
	    print(item)

# Prints
# 1
# 2
# 3
# 4

That’s a lot less typing! Don’t actually every code like the below snippet, but it does show what was happening under the hood:

my_list = [1, 2, 3, 4]
iterator = my_list.__iter__() # Get the iterator from the iterable

while iterator:
    try:
        print (next(iterator)) # Use the next method on an iterator
    except StopIteration:
        break

What we just saw is that iterators work because they’re a type of object that holds a countable number of items and you can then iterate over those items in order.

Iterators can also be written with a non-countable or infinite number of items, through the use of a generator function. A generator is different from a normal function because it uses a yield keyword to pass control back instead of a return. Generators are useful when we’re dealing with a large amount of data that doesn’t fit into memory. Generators thus allow us to feed in bits of our data at a time to our program. This also allows for infinite generators.

Here’s a fun example of a generator that will infinitely yield all the counting numbers. Notice the use of yield instead of return:

def counting_num():
    count = 0
    while True:
        yield count
        count += 1
    
# Test it:
for i in counting_num():
    print(i) # Prints 1 2 3 4 ... on separate lines until stopped by the user

We see that our counting_num() is a generator because it has a yield keyword instead of a return. We use a for loop to test it because we know from above that for loops act on iterators by calling the next() method, and generators are a type of iterator.

Review of Lambda Functions

As we’ll be using lambda functions below, and because lambda functions are important in functional programming, we’ll take a detour to review them now. Lambda functions are also called anonymous functions because they’re not defined with a name. For example, here’s a straightforward function in Python that just returns whatever you pass it multiplied by 2:

def double(x):
    return x*2

To define the above as a lambda function, it would look like this:

lambda x: x*2

Once the two functions are resolved down to byte code, there’s actually little difference between the two ways of defining the function. In that way, we call lambda functions syntactic sugar for defining a function because it’s simply a faster (less typing) and more concise way of defining a function that you won’t be using again.

But there are restrictions and lambda functions can’t do everything full functions can do. Lambda functions can contain only expressions but not statements. An expression is something that can be evaluated to a single value, whereas statements are anything else. An assignment statement like x = 5 is an example of a statement.

Lambda functions are a functional programming concept because they’re often used alongside higher-order functions (functions that take other functions as parameters). In this way, lambda functions are useful for passing as parameters to other functions. We’ll see an example soon using Python map() and a lambda function.

Deeper Dive Into Python map()

Now that we’ve reviewed iterables and lambdas, let’s think about the types of operations you might want to apply to a given iterable. You might want to

1. Apply some operation to every item in your iterable.
2. Subset your iterable based on a given criteria.

You could do these operations by writing out lots of loops and functions yourself. But what the pre-built Python map() and filter() functions give you is the ability to abstract away a lot of the complexity and just focus on the steps you need to take. This is another major tenet of functional programming: the existence of higher-order functions makes it possible to abstract away control flow into pure functions instead. Control flow determines the order statements are executed in and can be controlled by things like for loops and if/then statements.

What we didn’t mention in our review of iterables above is that iterables can be short, like a list of all students’ names in a classroom, or they can be very long, possibly even infinite if you’re using a generator. This will be common in big data applications, particularly if you’re streaming in large amounts of data. That’s why this functional paradigm is so helpful, because it allows you to apply operations on large amounts of data held in iterables such as a generator.

Now let’s dive into some examples.

Python map() With a Single Iterable

You already saw this example above, but let’s repeat it for completeness. Here’s map() used in its most basic form, with a single iterable:

def double(x):
    return 2*x

my_list = [1, 2, 3, 4]

double_list = list(map(double, my_list))
# You could write this in two lines to be even more clear:
# double_map = (map(double, my_list)
# double_list = list(double_map)

print(double_list) # Prints [2, 4, 6, 8]

In this case, we’re applying the double function to every element of our list.

Note: Remember that Python map() returns to us a map object. If we want to see it as a list (or whatever iterable we passed in), we have to cast it back using list() or similar.

Python map() With Two or More Iterables

If you pass in only a single iterable to Python map(), then the function you give it is applied to each element of the iterable. Once you start passing in multiple iterables, then each element of an iterable is passed in as a parameter to the function given to map. Let’s see it in action with two lists:

import operator

my_list1 = [1, 3, 5, 7]
my_list2 = [2, 4, 6, 8]

newList = list(map(operator.add,my_list1,my_list2)) # Returns [3, 7, 11, 15]

Notice that in this case the function passed to map() takes in two parameters. Two lists were passed in, too, so each list was traversed at the same time and its elements were passed element-wise to the add function.

What happens if one iterable is shorter than the other?

my_list1 = [1, 3, 5, 7]
my_list2 = [2, 4, 6]

def add(a,b):
    return a+b

list(map(add,my_list1,my_list2)) # Returns [3, 7, 11]

In the above case where one list is shorter, the list we get back is as long as the shorter list.

What happens if your function requires more arguments than the number of iterables you pass in? You’d get an error:

my_list1 = [1, 3, 5, 7]
my_list2 = [2, 4, 6]

def add_three_things(a,b,c):
     return a+b
 
list(map(add_three_things,my_list1,my_list2))

Traceback (most recent call last):
  File "", line 1, in 
TypeError: add() missing 1 required positional argument: 'c'

In this case, you got an error because add_three_things() required three arguments, but you gave the Python map() function only two iterables.

Python map() With a Lambda Function

map() is a great place to use lambda functions. Here’s an example using a lambda that squares its input:

my_list = [1, 2, 3, 4, 5]

list(map(lambda x: x*x, my_list)) # Returns [1, 4, 9, 16, 25]

We would use a lambda function instead of defining a function in this case to avoid polluting the namespace if we don’t intend to use the function anywhere else. Note, though, that lambda functions might be harder to read, so you should be careful trading off readability for conciseness.

Let’s do an example where we use a lambda function, plus multiple iterables:

my_list1 = [1, 3, 5, 7]
my_list2 = [2, 4, 6, 8]

list(map(lambda x,y: 2*x+4*y, my_list1, my_list2)) # Returns [10, 22, 34, 46]

In this case, our lambda function itself took in two parameters, x and y, so we needed to pass in two iterables to map.

Python map() With a String as Iterable

In all the examples so far you’ve used a list as your (iterable). But you can use all kinds of containers, including strings and dictionaries. You’ll now use a string as an iterable:

def shift_string_by_one(my_string):
    ''' 
    Takes in a string and moves every letter forward one in ASCII code
    '''
    shifted_word = ""
    for letter in my_string:
        shifted_word += chr(ord(letter)+1)
        # ord() returns an int representing the Unicode character
        # chr() turns it back into a character
    return shifted_word
    
my_string = "abcde"

shift_string_by_one_map = map(shift_string_by_one, my_string)

for item in shift_string_by_one_map:
    print(item)

# Returns
# b
# c
# d
# e
# f

If you use a string as an interable to map(), each letter gets treated as an element, and so if you apply shift_string_by_one() to each element, you get back each letter but moved one ahead in the alphabet.

A Python map() > filter() > reduce() Workflow

Python map() can be useful as part of a functional workflow, along with filter and reduce. Remember above we said that two operations you might want to do on an iterable are

1. Apply an operation.
2. Filter the iterable by some criteria.

Python map() does the first for us, filter() does the second. We might also be interested in doing some operation on the returned iterable that reduces it to a single value; that’s reduce(). We might chain them, either as map() > filter() > reduce() or filter() >map() > reduce() .

Note that in Python 3, reduce() was removed from the main library and was moved to the functools module.

The map() > filter() > reduce() paradigm is common in functional programming as it’s a way to get rid of control flow structure — such as for loops and if/then statements — and replace them with pure functions. map() , filter() , and reduce() are examples of pure functions.

How Python filter() Works

Filter is also an operation on an iterable: it takes a predicate function and an iterable, then tests each element of the iterable against the predicate function; elements that pass the predicate are then filtered out and returned.

Predicates here are basically statements that can take on a true or false value depending on the values of the variables they contain. That is, the predicate x > 5 has no truth value until we know what x is. Once we know what x is, for example, 3 then the predicate x > 5 is false.

Let’s look at a filter example. We’ll want only odd numbers returned from our list:

my_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

list( filter(lambda x: x%2 !=0, my_list)) # Returns [1, 3, 5, 7, 9]

We were looking to filter out only numbers that are odd and so not divisible by 2. We did this by passing in the predicate as a lambda function to filter(). The predicate was x%2 != 0. That is, we want only elements where, when we divide them by 2, they have a remainder and so they’re odd.

How Python reduce() Works

Reduce is the last step in our paradigm. It applies a binary function (one that takes two inputs) on our elements one by one and returns a single value. In case what we pass reduce is empty, we should also give it a starting value, which will be our return value if our iterable is empty.

Let’s look at an example of reduce:

from functools import reduce

my_list = [1, 2, 3, 4, 5]

summation = reduce(lambda x,y: x+y, my_list, 0) # Returns 15

reduce() is applied first to the starting value and element 1 and sums them, as per the binary function we pass in. From 0 plus 1 we get 1, which then becomes the first input to the binary function and the next element in the list, which is 2, for a result of 3. So it looks something like this, remembering that we passed in 0 as the starting value:

Input	Output
(0,1)	1
(1,2)	3
(3,3)	6
(6,4)	10
(10,5)	15

Each line represents the next step of the binary reduce function and the two numbers in parentheses are the inputs, starting with 0 because that’s what we passed to reduce() to start.

A Python map() > filter() > reduce() Example

Let’s run through an example of all three functions operating on a list. We’ll start with the numbers one through five, map() them to take their squares, then filter out only the odd numbers, then reduce() it all to get a sum:

from functools import reduce

my_list = [1, 2, 3, 4, 5]

squares = list( map(lambda x: x*x, my_list) ) # [1, 4, 9, 16, 25]
odds = list( filter(lambda x: x%2!=0, squares) ) # [1, 9, 25]
summation = reduce(lambda x,y: x+y, odds, 0) # 35

This is a trivial but complete example that runs through all the steps of map() > filter() > reduce(), starting from a list of numbers [1, 2, 3, 4, 5], applying a map to square them, then filtering out only the odd numbers, and finally reducing them by summing to get 35.

When Not to Use Python map() and What to Use Instead

Although Python map() isn’t being removed from base Python, at least not anytime soon, it’s not the most Pythonic approach. The Python docs even suggest using list comprehensions and generator expressions in place of map() and filter().

Note: In the next section, we’ll discuss why list comprehensions and generator expressions are the preferred way of doing these two operations in Python, but you should still be familiar with map() and filter().

Use Instead of Python map(): List Comprehensions

List comprehensions can replace Python map() and in fact are a more Pythonic way of operating on lists. List comprehensions are syntactic sugar in Python to achieve what you can do with a for loop. They’re a way of creating a new list using some logic applied to an existing list. We’ve written functions to square a number above, let’s see it as a list comprehension:

squares = [x * x for x in range(10)]
print(squares) # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

The syntax for a list comprehension is:

new_list = [expression for item in iterable]

In our squares case, the expression was x*x and our iterable was the list created by range(). We then applied the expression to every item, or x. We could have used whatever variable name we wanted for x.

We don’t have to stick to a lambda expression in a list comprehension, we could also use a named function, as in this example:

def triple(x):
    return x*3

triples = [triple(x) for x in range(10)]
print(triples) # [0, 3, 6, 9, 12, 15, 18, 21, 24, 27]

Here we used our user-defined function triple() in the list comprehension to triple the values of our list.

List comprehensions are considered more Pythonic than map() because that’s what people are used to seeing in Python. If you want your code to look like idiomatic Python, use list comprehensions instead of map(). There are many ways to add power to your list comprehensions, plus you can apply them to sets, dictionaries and other iterables, not just lists.

Use Instead of Python map(): Generator Expressions

Generator expressions are a generalization of list comprehensions for when you don’t want or can’t create your entire iterable in memory at one time.

Let’s go back to our friend, the squaring function. In order to do the following, Python first creates the entire list asked for by range(10), then applies x*x to it and then deletes the original list from memory:

[x*x for x in range(10)]

That’s a lot of work. We can use a generator expression and get a generator back instead. What this will do behind the scenes is act on the next element only when we use the next() method, just as we saw above with generators and the yield keyword.

To create a generator expression, use parentheses instead of brackets:

gen = (x*x for x in range(10))
       
while gen:
    try:
        print(next(gen))
    except StopIteration:
        break
# 0
# 1
# 4
# 9
# 16
# 25
# 36
# 49
# 64
# 81

As with our generator functions above, we’ll want to make sure that we’re only asking for the next item until the generator runs out, at which point it will raise a StopIteration exception.

Just like list comprehensions, you can add conditionals to generator expressions. You could also use a for loop, as Python knows when it hits a StopIteration exception and will break out of the loop:

gen = (x*x for x in range(10) if x > 5)

for i in gen:
    print(i)

# 36
# 49
# 64
# 81

Here we added the conditional x > 5 in the generator expression. The use of conditionals, in both list comprehensions and generator expressions, would be equivalent to using a filter() in a map/filter/reduce paradigm. The difference between a list comprehension and generator expression is that you’d use a generator expression when dealing with a very large, possibly infinite, iterable, since the generator expression computes only an element at a time as requested.

Writing Better Code With Functional Programming

That was a whirlwind tour of functional programming (FP), with a particular emphasis on the Python map() function. Learning FP is considered tough, but it’s worth it for a few reasons. Even if you’re not going to become a full-time functional programmer, introducing FP concepts into your Python code can make it easier to read and easier to debug.

FP Makes Code Easier to Read and Test

FP makes code easier to read because there are no hidden side-effects from functions. When reading your code, the user doesn’t have to keep track of any changes a function might be making to other data or program state; all she needs to do is look at what the function is returning in order to understand it.

FP concepts also make it much easier to test code. Take the common example of a function that allows a user to buy a cup of coffee and pay for it using a credit card. In an object-oriented world, the code for a single function might return the user back a cup of coffee but also go off and charge a credit card. To test that function would require also testing the functionality of charging a credit card, which is not at all related to our coffee function and in fact happens outside of our function.

Imagine writing a unit test for this function: charging the credit card is a side effect that happens outside our function! How can we test it? We could add a payments object parameter and write a mock for it (a mock is a fake version of some functionality of our code that’s beyond the scope of testing and so always returns a hardcoded correct answer). But regardless, things are tougher.

Why You Should Care About FP

Another theoretical benefit of functional programming is a bit mathematical: FP makes it easier to prove a program’s theoretical correctness. Formal provability goes beyond writing unit tests on a bunch of different inputs and checking that the outputs are correct; it has to do with formal mathematical proofs.

But you’re a practical Python programmer, so do you even need to care about functional programming? If you ever want to work with large data, the answer is yes. Now that you know what functional programming is, we can talk about why it’s the preferred paradigm when working with big data.

Having immutable data is helpful particularly when multiple clients are accessing the data, as happens in parallel programming. In the parallel case, you don’t want some client code to change the data and then some other client code gets different results depending on when they accessed the data. In parallel programming, allowing for mutable state means that our results might not be deterministic: The output might change depending on the order that operations happen to run in.

Functional programming fixes for the issues posed by concurrent access by enforcing immutable data and by using pure functions to ensure that your operations are referentially transparent. Referential transparency means the function call can be replaced by its computed value with no change in operation.

If you’re interested in functional programming and big data, check out Spark. Spark is a framework for doing big data processing in a functional way. It’s written in Scala but has a Python API via PySpark.

Conclusion

You’ve now seen how to use Python map() on an interable and what an iterable is. But possibly more importantly, you learned how map() helps us with a type of programming called functional programming and how that’s different from what you may be probably used to.

Here’s a quick review of what you learned in this post:

• What functional, imperative, procedural and objected oriented programming are
• How Python makes functional programming available to you
• What it means for a function to be a first-class object and how to use this fact in Python
• How to use map() in Python and, most importantly, when to use it
• When not to use map() and what to use instead (list comprehensions and generator expressions)

You can now start using the functional programming principles, including Python map() , in your code. No matter the type of application or script you’re writing, using functional programming principles can make your code easier to write and easier to maintain, and who doesn’t love that?