part_one/4-iterators.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Iterables, iterators and generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So far we have used for loops with many different kinds of things."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python\n",
    ">>> for name in ['theelous3', 'RubyPinch', 'go|dfish']:\n",
    "...     print(name)\n",
    "...\n",
    "theelous3\n",
    "RubyPinch\n",
    "go|dfish\n",
    ">>> for letter in 'abc':\n",
    "...     print(letter)\n",
    "...\n",
    "a\n",
    "b\n",
    "c\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For looping over something is one way to **iterate** over it. \n",
    "\n",
    "Some other things also iterate, for example, `' '.join(['a', 'b', 'c'])` iterates\n",
    "over the list `['a', 'b', 'c']`. \n",
    "\n",
    "If we can iterate over something, then that \"something\" is called **iterable**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For example, strings and lists are\n",
    "iterable, but integers and floats are not.\n",
    "\n",
    "```python\n",
    ">>> for thing in 123:\n",
    "...     print(thing)\n",
    "...\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "TypeError: 'int' object is not iterable\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Iterators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Lists and strings don't change when we iterate over them.\n",
    "\n",
    "```python\n",
    ">>> word = 'hi'\n",
    ">>> for character in word:\n",
    "...     print(character)\n",
    "...\n",
    "h\n",
    "i\n",
    ">>> word\n",
    "'hi'\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "> **Definition**: Iterators are iterables that remember their position."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Examples of Iterators\n",
    "\n",
    "So far, we've been already using **iterators** even if we did not call them with their names.\n",
    "\n",
    "This is because most of the time iterators have been used anonimously withing `for` loops\n",
    "(isn't this the ultimate use of iterators for the majority of cases, anyway?).\n",
    "\n",
    "These functions are (in order of appearance):\n",
    "\n",
    "* `enumerate(iterable)`\n",
    "* `range(start, end, step)`\n",
    "* `dictionary.keys()`\n",
    "* `dictionary.values()`\n",
    "* `dictionary.items()`\n",
    "\n",
    "However, it is always possible to call iterators explicitly, and get an **iterator** object in return.\n",
    "\n",
    "The very same object can be used in a for loop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```python\n",
    ">>> e = enumerate('hello')\n",
    ">>> for pair in e:\n",
    "...     print(pair)\n",
    "...\n",
    "(0, 'h')\n",
    "(1, 'e')\n",
    "(2, 'l')\n",
    "(3, 'l')\n",
    "(4, 'o')\n",
    ">>> for pair in e:\n",
    "...     print(pair)\n",
    "...\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "As reported in the exampl above, the **iterator** actually rememebers the current position. \n",
    "So, Iterators can only be used once, so we need to create a new iterator if\n",
    "we want to do another for loop.\n",
    "\n",
    "In other words: \n",
    "\n",
    "> Once the iterator has been fully consumed, there is no way to get recommence from the beginning\n",
    "> of the sequence, without creating a new iterator object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Note**: **All iterators are iterables, but not all iterables are iterators.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "> **Question**: Any idea for that? What is an example of an iterables that is **not** an iterator?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Iterating manually\n",
    "\n",
    "Iterators have a magic method called `__next__` that gets next value and\n",
    "moves the iterator forward.\n",
    "\n",
    "**Note**: Please remember this when we will talk about Classes, objects, and magic methods.\n",
    "\n",
    "```python\n",
    ">>> e = enumerate('abc')\n",
    ">>> e.__next__()\n",
    "(0, 'a')\n",
    ">>> e.__next__()\n",
    "(1, 'b')\n",
    ">>> e.__next__()\n",
    "(2, 'c')\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "There's also a built-in `next()` function that does the same thing:\n",
    "\n",
    "```python\n",
    ">>> e = enumerate('abc')\n",
    ">>> next(e)\n",
    "(0, 'a')\n",
    ">>> next(e)\n",
    "(1, 'b')\n",
    ">>> next(e)\n",
    "(2, 'c')\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In this example, `e` remembers its position, and every time we call `next(e)` it\n",
    "gives us the next element and moves forward. \n",
    "\n",
    "When it has no more values to give us, calling `next(e)` raises a `StopIteration` Exception:\n",
    "\n",
    "```python\n",
    ">>> next(e)\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "StopIteration\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "There is usually not a good way to check if the iterator is at the end,\n",
    "and it's best to just try to get a value from it and\n",
    "[**catch**](../basics/exceptions.md#catching-exceptions) `StopIteration`.\n",
    "\n",
    "**That's actually what `for` loops do.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Converting to iterators\n",
    "\n",
    "Now we know what iterating over an iterator does. But how about\n",
    "iterating over a list or a string? \n",
    "\n",
    "They are **not** iterators, so we can't call `next()` on them:\n",
    "\n",
    "```python\n",
    ">>> next('abc')\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "TypeError: 'str' object is not an iterator\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "There's a built-in function called `iter()` that converts anything\n",
    "iterable to an iterator.\n",
    "\n",
    "```python\n",
    ">>> i = iter('abc')\n",
    ">>> i\n",
    "<str_iterator object at 0x7f987b860160>\n",
    ">>> next(i)\n",
    "'a'\n",
    ">>> next(i)\n",
    "'b'\n",
    ">>> next(i)\n",
    "'c'\n",
    ">>> next(i)\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "StopIteration\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Calling `iter()` on anything non-iterable gives us an error.\n",
    "\n",
    "```python\n",
    ">>> iter(123)\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "TypeError: 'int' object is not iterable\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Finally, if we try to convert an iterator to an iterator using `iter()` we just\n",
    "get back the same iterator."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Generators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It's possible to create a custom iterator with a class that defines an\n",
    "`__iter__` method that returns self and a `__next__` method that gets\n",
    "the next value. \n",
    "\n",
    "I'm not going to talk about it now because there's a\n",
    "much simpler way to implement iterators. \n",
    "\n",
    "Let's make a function that\n",
    "creates an iterator that behaves like `iter([1, 2, 3])` using the\n",
    "`yield` keyword:\n",
    "\n",
    "```python\n",
    ">>> def first_generator():\n",
    "...     yield 1\n",
    "...     yield 2\n",
    "...     yield 3\n",
    "...\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Note about syntax**: We can only `yield` inside a function, yielding elsewhere raises an\n",
    "error.\n",
    "\n",
    "```python\n",
    ">>> yield 'hi'\n",
    "  File \"<stdin>\", line 1\n",
    "SyntaxError: 'yield' outside function\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's try out our `first_generator` function and see how it works.\n",
    "\n",
    "```python\n",
    ">>> first_generator()\n",
    "<generator object first_generator at 0xb723d9b4>\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Putting a `yield` anywhere in a function makes it return **generators**.\n",
    "**Generators are iterators** with some more features that we don't need\n",
    "to care about."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The **generator** that `simple_generator` returns work just like other iterators:\n",
    "\n",
    "```python\n",
    ">>> g = simple_generator()\n",
    ">>> g\n",
    "<generator object simple_generator at 0xb72300f4>\n",
    ">>> next(g)\n",
    "1\n",
    ">>> next(g)\n",
    "2\n",
    ">>> next(g)\n",
    "3\n",
    ">>> next(g)\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "StopIteration\n",
    ">>> for number in simple_generator():\n",
    "...     print(number)\n",
    "...\n",
    "1\n",
    "2\n",
    "3\n",
    ">>>\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This whole thing may feel kind of insane. If we add some parts between\n",
    "the yields, when do they run? How does Python know when to run what?\n",
    "\n",
    "Let's find out.\n",
    "\n",
    "```python\n",
    ">>> def printygen():\n",
    "...     print(\"starting\")\n",
    "...     yield 1\n",
    "...     print(\"between 1 and 2\")\n",
    "...     yield 2\n",
    "...     print(\"between 2 and 3\")\n",
    "...     yield 3\n",
    "...     print(\"end\")\n",
    "...\n",
    ">>> p = printygen()\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "That's weird! We called it, but it didn't print \"starting\"!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Let's see what happens if we call `next()` on it.\n",
    "\n",
    "```python\n",
    ">>> got = next(p)\n",
    "starting\n",
    ">>> got\n",
    "1\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Now it started, but it's frozen! It's just stuck on that `yield 1`.\n",
    "\n",
    "An easy way to think about this is to compare it to our computers.\n",
    "When we suspend a computer it goes into some kind of stand-by mode,\n",
    "and we can later continue using the computer all of our programs are\n",
    "still there just like they were when we left.\n",
    "\n",
    "A similar thing happens here. Our function is running, but it's just\n",
    "stuck at the yield and waiting for us to call `next()` on it again.\n",
    "\n",
    "```python\n",
    ">>> next(p)\n",
    "between 1 and 2\n",
    "2\n",
    ">>> next(p)\n",
    "between 2 and 3\n",
    "3\n",
    ">>> next(p)\n",
    "end\n",
    "Traceback (most recent call last):\n",
    "  File \"<stdin>\", line 1, in <module>\n",
    "StopIteration\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "Here's a drawing of what's going on:\n",
    "\n",
    "![A picture of printygen.](../images/freeze-melt.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The good news is that **usually we don't need to worry about when\n",
    "exactly the parts between the yields run**. Actually we don't even need\n",
    "to use `iter()` and `next()` most of the time, but I think it's nice to\n",
    "know how for loops work.\n",
    "\n",
    "`yield` is useful when we want the function to output so many things\n",
    "that making a list of them would be too slow or the list wouldn't fit in\n",
    "the computer's memory. So instead of this..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```python\n",
    "def retrieve_results():\n",
    "    results = []\n",
    "    # code that appends things to result\n",
    "    return results\n",
    "```\n",
    "\n",
    "...we can do this:\n",
    "\n",
    "```python\n",
    "def lazy_retrieve_ results():\n",
    "    # code that yields stuff\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Both of these functions can be used like this:\n",
    "\n",
    "```python\n",
    "for thing in retrieve_results():\n",
    "    # do something with thing\n",
    "```\n",
    "\n",
    "however the **lazy** version is more efficient."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It's actually possible to create an iterator that yields an infinite\n",
    "number of things:\n",
    "\n",
    "```python\n",
    ">>> def count():\n",
    "...     current = 1\n",
    "...     while True:\n",
    "...         yield current\n",
    "...         current += 1\n",
    "...\n",
    ">>> c = count()\n",
    ">>> next(c)\n",
    "1\n",
    ">>> next(c)\n",
    "2\n",
    ">>> next(c)\n",
    "3\n",
    ">>> next(c)\n",
    "4\n",
    ">>>\n",
    "```\n",
    "\n",
    "[The itertools module](https://docs.python.org/3/library/itertools.html)\n",
    "contains many useful things like this. For example, `itertools.count(1)`\n",
    "does the same thing as our `count()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Take away message:\n",
    "\n",
    "Python is a language with **batteries included**: _iow_ what is called the \n",
    "**Python standard library** is massive and contains a lots of modules with functions and solutions \n",
    "for usually whatsoever problem.\n",
    "\n",
    "So, **if and only if** you know what you're doing (meaning \"you know already **how** what you're \n",
    "trying to do (should) work, and you would be able to implement it, in principle)  \n",
    "it is better to resuse _existing_ implementations, **without reinventing the wheel**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Summary\n",
    "\n",
    "- An iterable is something that we can for loop over.\n",
    "- An iterator is an iterable that remembers its position.\n",
    "- For loops create an iterator of the iterable and call its `__next__`\n",
    "    method until it raises a StopIteration.\n",
    "- Functions that contain yields return generators. Calling `next()` on a\n",
    "  generator runs it to the next yield and gives us the value it yielded.\n",
    "- [The itertools module](https://docs.python.org/3/library/itertools.html)\n",
    "  contains many useful iterator-related things."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# List Comprehensions\n",
    "\n",
    ">I thought carefully before including this section. If you are brand new to programming, \n",
    "> list comprehensions may look confusing at first. \n",
    "\n",
    "List comprehensions are a **shorthand way** of creating and working with lists. \n",
    "\n",
    "It is good to be aware of list comprehensions, because you will see them in other people's code, \n",
    "and they are really useful when you understand how to use them. \n",
    "\n",
    "That said, if they don't make sense to you yet, don't worry about using them right away. \n",
    "\n",
    "When you have worked with enough lists, you will want to use comprehensions. \n",
    "\n",
    "For now, it is good enough to know they exist, and to recognize them when you see them. \n",
    "\n",
    "If you like them, go ahead and start trying to use them now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Numerical Comprehensions\n",
    "\n",
    "Let's consider how we might make a list of the first ten square numbers. We could do it like this:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    "\n",
    "# Store the first ten square numbers in a list.\n",
    "# Make an empty list that will hold our square numbers.\n",
    "squares = []\n",
    "\n",
    "# Go through the first ten numbers, square them, and add them to our list.\n",
    "for number in range(1,11):\n",
    "    new_square = number**2\n",
    "    squares.append(new_square)\n",
    "    \n",
    "# Show that our list is correct.\n",
    "for square in squares:\n",
    "    print(square)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This should make sense at this point. If it doesn't, go over the code with these thoughts in mind:\n",
    "- We make an empty list called *squares* that will hold the values we are interested in.\n",
    "- Using the *range()* function, we start a loop that will go through the numbers 1-10.\n",
    "- Each time we pass through the loop, we find the square of the current number by raising it to the second power.\n",
    "- We add this new value to our list *squares*.\n",
    "- We go through our newly-defined list and print out each square.\n",
    "\n",
    "Now let's make this code more efficient. We don't really need to store the new square in its own variable *new_square*; we can just add it directly to the list of squares. The line\n",
    "\n",
    "```\n",
    "    new_square = number**2\n",
    "```\n",
    "\n",
    "is taken out, and the next line takes care of the squaring:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "```python \n",
    "# Store the first ten square numbers in a list.\n",
    "# Make an empty list that will hold our square numbers.\n",
    "squares = []\n",
    "\n",
    "# Go through the first ten numbers, square them, and add them to our list.\n",
    "for number in range(1,11):\n",
    "    squares.append(number**2) \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "List comprehensions allow us to **collapse the first three lines of code into one line**. \n",
    "\n",
    "Here's what it looks like:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    ">>> # Store the first ten square numbers in a list.\n",
    ">>> squares = [number**2 for number in range(1,11)]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "It should be pretty clear that this code is more efficient than our previous approach, but it may not be \n",
    "clear what is happening. \n",
    "\n",
    "Let's take a look at everything that is happening in that first line:\n",
    "\n",
    "We define a list called `squares`.\n",
    "\n",
    "Look at the second part of what's in square brackets:\n",
    "\n",
    "    for number in range(1,11)\n",
    "\n",
    "This sets up a loop that goes through the numbers `1-10`, storing each value in the variable `number`. \n",
    "\n",
    "Now we can see what happens to each `number` in the loop:\n",
    "\n",
    "    number**2\n",
    "\n",
    "Each number is raised to the second power, and this is the value that is stored in the list we defined. \n",
    "\n",
    "We might read this line in the following way (pseudocode):\n",
    "\n",
    "```\n",
    "    squares = [raise 'number' to the second power, for each 'number' in the range 1-10]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Another example\n",
    "It is probably helpful to see a few more examples of how comprehensions can be used. Let's try to make the first ten even numbers, the longer way:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    "# Make an empty list that will hold the even numbers.\n",
    "evens = []\n",
    "\n",
    "# Loop through the numbers 1-10, double each one, and add it to our list.\n",
    "for number in range(1,11):\n",
    "    evens.append(number*2)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Here's how we might think of doing the same thing, using a list comprehension:\n",
    "\n",
    "```\n",
    "evens = [multiply each *number* by 2, for each *number* in the range 1-10]\n",
    "```\n",
    "\n",
    "Here is the same line in code:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    "# Make a list of the first ten even numbers.\n",
    "evens = [number * 2 for number in range(1,20)]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Non-numerical comprehensions\n",
    "\n",
    "At this point you might think that **list comprehension** is only limited to numerical computations.\n",
    "\n",
    "Of course, it is **not**. We can use comprehensions with non-numerical lists as well. \n",
    "\n",
    "List comprehension has to be used everytime we would need to \n",
    "**create a new list by iterating another iterable**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In this example we will be using a **dictionary** as the reference iterable to iterate, and we \n",
    "are going to create a list of tuples. More specifically a list of `(key, value)` pairs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    ">>> country_map = {'IT': 'Italy', 'UK': 'United Kingdom', 'DE': 'Germany', 'DK': 'Denmark', 'FR': 'France'}\n",
    ">>> country_list = [(code, country_name) for code, country_name in country_map.items()]\n",
    ">>> print(country_map)\n",
    "{'IT': 'Italy',\n",
    " 'UK': 'United Kingdom',\n",
    " 'DE': 'Germany',\n",
    " 'DK': 'Denmark',\n",
    " 'FR': 'France'}\n",
    ">>> print(country_list)\n",
    "[('IT', 'Italy'),\n",
    " ('UK', 'United Kingdom'),\n",
    " ('DE', 'Germany'),\n",
    " ('DK', 'Denmark'),\n",
    " ('FR', 'France')]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<a name='generator_expressions'></a>Generator Expressions\n",
    "==="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Similarly to **list comprehensions**, in Python it is possible to define **generator expressions**.\n",
    "\n",
    "Generator expressions use the **same** syntax as list comprehensions, \n",
    "but are enclosed in **parentheses** rather than **brackets**.\n",
    "\n",
    "Despite the very slight difference in the syntax, **generator expressions** save memory as they \n",
    "**yield** items one by one using the iterator protocol instead of building \n",
    "a whole list just to feed another constructor.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### GenExps in Function Calls\n",
    "\n",
    "If the generator expression is the single argument in a function call, \n",
    "there is no need to duplicate the enclosing parentheses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    ">>> symbols = '$¢£¥€¤'\n",
    ">>> tuple(ord(symbol) for symbol in symbols)  # generator expression to initialise a tuple\n",
    "(36, 162, 163, 165, 8364, 164)\n",
    ">>>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `ord` function is a built-in function that returns the Unicode code point for a one-character \n",
    "string."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Array Constructor\n",
    "\n",
    "The array constructor takes two arguments, so the parentheses around the generator expression are mandatory. \n",
    "\n",
    "The first argument of the array constructor defines the storage type used for the numbers in the array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "``` python\n",
    "import array\n",
    "array.array('I', (ord(symbol) for symbol in symbols))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Cartesian Product in a generator expression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python \n",
    ">>> colors = ['black', 'white']\n",
    ">>> sizes = ['S', 'M', 'L']\n",
    ">>> for tshirt in ('%s %s' % (c, s) for c in colors for s in sizes):\n",
    "...    print(tshirt)\n",
    "...\n",
    "black S\n",
    "black M\n",
    "black L\n",
    "white S\n",
    "white M\n",
    "white L\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}