2.5. Generator Inspect

2.5.1. Is Generator

>>> from inspect import isgenerator
>>>
>>>
>>> a = [x for x in range(0,5)]
>>> b = (x for x in range(0,5))
>>>
>>> isgenerator(a)
False
>>> isgenerator(b)
True
>>> from inspect import isgenerator
>>>
>>>
>>> data = range(0, 10)
>>>
>>> isgenerator(data)
False

2.5.2. Introspection

>>> data = (x for x in range(0,10))
>>>
>>>
>>> next(data)
0
>>>
>>> data.gi_code  
<code object <genexpr> at 0x..., file "<...>", line 1>
>>>
>>> data.gi_running
False
>>>
>>> data.gi_frame  
<frame at 0x..., file '<...>', line 1, code <genexpr>>
>>>
>>> data.gi_frame.f_locals  
{'.0': <range_iterator object at 0x...>, 'x': 0}
>>>
>>> data.gi_frame.f_code  
<code object <genexpr> at 0x...0, file "<...>", line 1>
>>>
>>> data.gi_frame.f_lineno
1
>>>
>>> data.gi_frame.f_lasti
8
>>>
>>> data.gi_yieldfrom

2.5.3. Memory Footprint

  • sys.getsizeof(obj) returns the size of an obj in bytes

  • sys.getsizeof(obj) calls obj.__sizeof__() method

  • sys.getsizeof(obj) adds an additional garbage collector overhead if the obj is managed by the garbage collector

>>> from sys import getsizeof
>>>
>>>
>>> gen1 = (x for x in range(0,1))
>>> gen10 = (x for x in range(0,10))
>>> gen100 = (x for x in range(0,100))
>>> gen1000 = (x for x in range(0,1000))
>>>
>>> getsizeof(gen1)
112
>>>
>>> getsizeof(gen10)
112
>>>
>>> getsizeof(gen100)
112
>>>
>>> getsizeof(gen1000)
112
>>> from sys import getsizeof
>>>
>>>
>>> com1 = [x for x in range(0,1)]
>>> com10 = [x for x in range(0,10)]
>>> com100 = [x for x in range(0,100)]
>>> com1000 = [x for x in range(0,1000)]
>>>
>>>
>>> getsizeof(com1)
88
>>>
>>> getsizeof(com10)
184
>>>
>>> getsizeof(com100)
920
>>>
>>> getsizeof(com1000)
8856
../../_images/generator-function-performance.png

Figure 2.2. Source: https://www.askpython.com/python/python-yield-examples

2.5.4. Assignments

Code 2.27. Solution
"""
* Assignment: Generator Function Iris
* Complexity: easy
* Lines of code: 8 lines
* Time: 8 min

English:
    1. Write filter for `DATA` which returns `features: list[float]` for given `species: str`
    2. Implement solution using function
    3. Implement solution using generator and `yield` keyword
    4. Compare results of both using `sys.getsizeof()`
    5. What will happen if input data will be bigger?
    6. Note, that in different Python versions you'll get slightly
       different values for getsizeof generator and function:
        a. 112 for generator in Python 3.9
        b. 112 for generator in Python 3.8
        c. 120 for generator in Python 3.7
    7. Run doctests - all must succeed

Polish:
    1. Napisz filtr dla `DATA` zwracający `features: list[float]` dla danego gatunku `species: str`
    2. Zaimplementuj rozwiązanie wykorzystując funkcję
    3. Zaimplementuj rozwiązanie wykorzystując generator i słowo kluczowe `yield`
    4. Porównaj wyniki obu używając `sys.getsizeof()`
    5. Co się stanie, gdy ilość danych będzie większa?
    6. Zwróć uwagę, że w zależności od wersji Python wartości getsizeof
       dla funkcji i generatora mogą się nieznaczenie różnić:
        a. 112 dla generator w Python 3.9
        b. 112 dla generator w Python 3.8
        c. 120 dla generator w Python 3.7
    7. Uruchom doctesty - wszystkie muszą się powieść

Tests:
    >>> import sys; sys.tracebacklimit = 0
    >>> from sys import getsizeof
    >>> from inspect import isfunction, isgeneratorfunction

    >>> assert isfunction(function)
    >>> assert isgeneratorfunction(generator)

    >>> list(function(DATA, 'setosa'))
    [[5.1, 3.5, 1.4, 0.2], [4.7, 3.2, 1.3, 0.2]]
    >>> list(generator(DATA, 'setosa'))
    [[5.1, 3.5, 1.4, 0.2], [4.7, 3.2, 1.3, 0.2]]

    >>> getsizeof(function(DATA, 'setosa'))
    88
    >>> getsizeof(function(DATA*10, 'setosa'))
    248
    >>> getsizeof(function(DATA*100, 'setosa'))
    1656
    >>> getsizeof(generator(DATA, 'setosa'))
    112
    >>> getsizeof(generator(DATA*10, 'setosa'))
    112
    >>> getsizeof(generator(DATA*100, 'setosa'))
    112
"""

DATA = [(5.8, 2.7, 5.1, 1.9, 'virginica'),
        (5.1, 3.5, 1.4, 0.2, 'setosa'),
        (5.7, 2.8, 4.1, 1.3, 'versicolor'),
        (6.3, 2.9, 5.6, 1.8, 'virginica'),
        (6.4, 3.2, 4.5, 1.5, 'versicolor'),
        (4.7, 3.2, 1.3, 0.2, 'setosa')]


# Callable: get `features: list[float]` from `DATA` for given `species`
def function(data: list, species: str):
    ...


# Generator: get `features: list[float]` from `DATA` for given `species`
def generator(data: list, species: str):
    ...