> ## Documentation Index
> Fetch the complete documentation index at: https://genai.codewithsiva.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# 04-Advanced Python Concepts

> Learn advanced Python concepts including functions as objects, higher-order functions, closures, decorators, iterators, generators, and context managers.

This chapter introduces advanced Python concepts that make programs more modular, reusable, and memory efficient. These concepts are widely used in modern Python applications, web frameworks, data processing, and AI libraries.

Since functions are objects, they can be assigned to variables, passed as arguments, returned from other functions, and stored in collections. These capabilities form the foundation for **higher-order functions**, **decorators**, **closures**, **iterators** and **generators**.

## Topics Covered

In this module, you'll learn:

1. [Functions are Objects](#functions-are-objects)
2. [Higher-Order Functions](#higher-order-functions)
3. [Closures](#closures)
4. [Decorators](#decorators)
5. [Iterators](#what-is-an-iterator)
6. [Generators](#what-is-a-generator)
7. [Context Managers](#context-managers)

> **Try Yourself:** [💻 VS Code](vscode://file/Users/sivaprasad/Downloads/DSA%20With%20Python/public/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts-exercises.ipynb) | [🚀 Colab](https://colab.research.google.com/github/prasad230776/genai-course/blob/main/public/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts-exercises-colab.ipynb) | <a href="/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts-exercises.ipynb" download>📥 Download</a><br />
> **Verify Solutions:** [💻 VS Code](vscode://file/Users/sivaprasad/Downloads/DSA%20With%20Python/public/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts.ipynb) | [🚀 Colab](https://colab.research.google.com/github/prasad230776/genai-course/blob/main/public/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts-colab.ipynb) | <a href="/notebooks/workshop-notebooks/04-advanced-concepts/04-advanced-concepts.ipynb" download>📥 Download</a>

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Functions are Objects

Like integers, strings, lists, and dictionaries, **functions are also objects**. Therefore, a function can:

* Be assigned to a variable.
* Be passed as an argument.
* Be returned from another function.
* Be stored in a collection.

Languages that support these capabilities are said to support **first-class functions**.

### Assigning a Function

```python theme={null}
def greet():
    return "Hello"

message = greet

print(message())
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Hello
  ```
</Accordion>

Notice the difference:

```python theme={null}
message = greet      # Function object
message = greet()    # Function call
```

### Storing Functions

```python theme={null}
def add(a, b):
    return a + b


def subtract(a, b):
    return a - b


operations = [add, subtract]

print(operations[0](10, 5))
print(operations[1](10, 5))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  15
  5
  ```
</Accordion>

### Exercise 2

What is the difference between the following statements?

```python theme={null}
f = greet
```

```python theme={null}
f = greet()
```

<Accordion title="Solution">
  * `f = greet` assigns the function object.
  * `f = greet()` calls the function and stores its return value.
</Accordion>

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Higher-Order Functions

A **higher-order function** is a function that:

* Accepts one or more functions as arguments.
* Returns a function.

Since functions are objects, they can be passed to and returned from other functions.

### Passing Functions as Arguments

```python theme={null}
def greet():
    print("Hello")


def execute(func):
    func()


execute(greet)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Hello
  ```
</Accordion>

### Returning Functions

```python theme={null}
def outer():

    def inner():
        print("Hello")

    return inner


message = outer()

message()
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Hello
  ```
</Accordion>

### Example

```python theme={null}
def add(a, b):
    return a + b


def multiply(a, b):
    return a * b


def calculate(operation, a, b):
    return operation(a, b)


print(calculate(add, 10, 5))
print(calculate(multiply, 10, 5))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  15
  50
  ```
</Accordion>

### Exercise 2

Predict the output.

```python theme={null}
def outer():

    def inner():
        return "Hello"

    return inner


func = outer()

print(func())
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Hello
  ```
</Accordion>

<Accordion title="Solution">
  ```text theme={null}
  Hello
  ```
</Accordion>

### Exercise 3

When is a function called a higher-order function?

<Accordion title="Solution">
  A function is called a **higher-order function** if it:

  * Accepts one or more functions as arguments.
  * Returns a function.
</Accordion>

> Higher-order functions form the foundation for closures and decorators.

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Closures

Sometimes we want a function to **remember information** from previous function calls.

A normal function cannot do this because its local variables are destroyed when the function finishes executing.

### Example: Normal Function

```python theme={null}
def counter():

    count = 0

    count += 1

    return count


print(counter())
print(counter())
print(counter())
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  1
  1
  ```
</Accordion>

Each time `counter()` is called, the local variable `count` is created again and initialized to `0`. Therefore, the function cannot remember its previous state.

To preserve the state between function calls, we can use a **closure**.

### What is a Closure?

A **closure** is an inner function that remembers the variables of its enclosing function even after the enclosing function has finished executing.

A closure is created when:

* A function is defined inside another function.
* The inner function uses variables from the outer function.
* The inner function is returned.

### Example

```python theme={null}
def counter():

    count = 0

    def increment():
        nonlocal count

        count += 1
        return count

    return increment


c = counter()

print(c())
print(c())
print(c())
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  2
  3
  ```
</Accordion>

Here, the variable `count` is preserved even after `counter()` has finished executing. Each call to `increment()` updates the same variable instead of creating a new one.

The `nonlocal` keyword allows the inner function to modify a variable defined in the enclosing function.

### Exercise 2

Predict the output.

```python theme={null}
def counter():

    count = 10

    def increment():
        nonlocal count
        count += 5
        return count

    return increment


c = counter()

print(c())
print(c())
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  15
  20
  ```
</Accordion>

<Accordion title="Solution">
  ```text theme={null}
  15
  20
  ```

  The variable `count` is preserved inside the closure and updated on each function call.
</Accordion>

### Exercise 3

Why do we use closures?

<Accordion title="Solution">
  Closures allow a function to remember and preserve variables from its enclosing function even after the enclosing function has finished executing.
</Accordion>

> Closures are widely used for state preservation and form the foundation of **decorators**, where the wrapper function remembers the original function passed to the decorator.

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Decorators

A **decorator** is a function that extends or modifies the behavior of another function **without changing its original code**.

A decorator is a higher-order function because it:

* Accepts a function as an argument.
* Returns another function.

### Creating a Decorator

```python theme={null}
def logger(func):

    def wrapper():
        print("Before function")

        func()

        print("After function")

    return wrapper


def greet():
    print("Hello")


greet = logger(greet)

greet()
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Before function
  Hello
  After function
  ```
</Accordion>

Instead of modifying `greet()`, the decorator returns a new function with additional behavior.

### Using the `@` Syntax

Python provides the `@` syntax as a convenient way to apply decorators.

```python theme={null}
def logger(func):

    def wrapper():
        print("Before function")

        func()

        print("After function")

    return wrapper


@logger
def greet():
    print("Hello")


greet()
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Before function
  Hello
  After function
  ```
</Accordion>

The above code is equivalent to:

```python theme={null}
def greet():
    print("Hello")


greet = logger(greet)
```

#

### Exercise 2

Which statement is equivalent to the following code?

```python theme={null}
@logger
def greet():
    print("Hello")
```

<Accordion title="Solution">
  ```python theme={null}
  def greet():
      print("Hello")


  greet = logger(greet)
  ```

  The `@` syntax is a shorthand for applying a decorator.
</Accordion>

### Decorating Functions with Parameters

The previous decorator works only for functions that do **not** accept any arguments.

```python theme={null}
@logger
def greet():
    print("Hello")
```

Suppose we decorate a function that accepts parameters.

```python theme={null}
@logger
def add(a, b):
    return a + b
```

Our decorator is:

```python theme={null}
def logger(func):

    def wrapper():
        print("Function Started")

        result = func()

        print("Function Completed")

        return result

    return wrapper
```

When we call:

```python theme={null}
add(10, 20)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: logger.<locals>.wrapper() takes 0 positional arguments but 2 were given
  ```
</Accordion>

Python actually executes:

```python theme={null}
wrapper(10, 20)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: name 'wrapper' is not defined
  ```
</Accordion>

Since `wrapper()` does not accept any arguments, Python raises an error.

```text theme={null}
TypeError: wrapper() takes 0 positional arguments but 2 were given
```

One solution is to make the wrapper accept the same parameters.

```python theme={null}
def logger(func):

    def wrapper(a, b):
        print("Function Started")

        result = func(a, b)

        print("Function Completed")

        return result

    return wrapper
```

This works only for functions having exactly two parameters.

To make the decorator work with **any function**, Python provides **argument packing**.

```python theme={null}
def logger(func):

    def wrapper(*args, **kwargs):
        print("Function Started")

        result = func(*args, **kwargs)

        print("Function Completed")

        return result

    return wrapper
```

Here,

* `*args` collects all positional arguments.
* `**kwargs` collects all keyword arguments.
* `func(*args, **kwargs)` forwards all arguments to the original function.

Now the decorator can be applied to functions with any number of arguments.

```python theme={null}
@logger
def multiply(a, b):
    return a * b


@logger
def greet(name):
    print(f"Hello {name}")


print(multiply(10, 5))
greet("Alice")
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Function Started
  Function Completed
  50
  Function Started
  Hello Alice
  Function Completed
  ```
</Accordion>

> The `wrapper()` function is defined inside another function and remembers the original `func` even after the outer function has finished executing. This behavior was possible because of **closure**.

***

An **iterator** is an object that returns one value at a time from a collection, while a **generator** is a special type of iterator created using the `yield` keyword. They provide a memory-efficient way to process data without loading everything into memory at once.

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Learning Objectives

After completing this lesson, you will be able to:

* Understand iterables, iterators, and generators.
* Create iterators using `iter()` and `next()`.
* Build custom iterators.
* Create generators using `yield`.
* Differentiate between `yield` and `return`.
* Create generator expressions.
* Compare iterators and generators.
* Identify real-world use cases of generators.

<div align="right">[Back to Top ↑](#topics-covered)</div>

## What is an Iterator?

An **iterator** is an object that returns one element at a time from a collection. It remembers its current position and produces the next value only when requested.

Python uses iterators internally whenever you iterate over a collection using a `for` loop.

### Iterator Protocol

An iterator implements the following special methods:

* `__iter__()` – Returns the iterator object.
* `__next__()` – Returns the next element.

When no more elements are available, `__next__()` raises a `StopIteration` exception.

### Creating an Iterator

Use the `iter()` function to create an iterator from an iterable.

```python theme={null}
numbers = [10, 20, 30]

iterator = iter(numbers)

print(iterator)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  <list_iterator object at 0x...>
  ```
</Accordion>

### Retrieving Values

Use the `next()` function to retrieve values from an iterator.

```python theme={null}
numbers = [10, 20, 30]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  10
  20
  30
  ```
</Accordion>

### StopIteration

Once all elements are consumed, calling `next()` again raises a `StopIteration` exception.

```python theme={null}
numbers = [10]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  10
  Error executing: 
  ```
</Accordion>

#

### Exercise 2

Predict the output.

```python theme={null}
name = "Python"

it = iter(name)

print(next(it))
print(next(it))
print(next(it))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  P
  y
  t
  ```
</Accordion>

<Accordion title="Solution">
  ```text theme={null}
  P
  y
  t
  ```

  Strings are iterable objects, so they can be converted into iterators using `iter()`.
</Accordion>

### Exercise 3

What exception will be raised by the following code?

```python theme={null}
numbers = [1]

it = iter(numbers)

print(next(it))
print(next(it))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  Error executing: 
  ```
</Accordion>

<Accordion title="Solution">
  A `StopIteration` exception is raised because the iterator has no more elements to return.
</Accordion>

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Creating a Custom Iterator

You can create your own iterator by implementing the `__iter__()` and `__next__()` methods.

* `__iter__()` returns the iterator object.
* `__next__()` returns the next value.
* When all values are consumed, `__next__()` raises a `StopIteration` exception.

### Example

```python theme={null}
class Counter:

    def __init__(self, limit):
        self.current = 1
        self.limit = limit

    def __iter__(self):
        return self

    def __next__(self):
        if self.current > self.limit:
            raise StopIteration

        value = self.current
        self.current += 1
        return value


counter = Counter(5)

for number in counter:
    print(number)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  2
  3
  4
  5
  ```
</Accordion>

### How It Works

1. The `Counter` object is created.
2. The `for` loop calls `__iter__()` to obtain the iterator.
3. The loop repeatedly calls `__next__()`.
4. Each call returns the next value.
5. When the limit is reached, `StopIteration` is raised, ending the loop.

#

### Exercise 2

What happens if `raise StopIteration` is removed from the `__next__()` method?

<Accordion title="Solution">
  The iterator will never indicate that it has finished, causing the loop to continue indefinitely or resulting in incorrect behavior.
</Accordion>

### Exercise 3

Which two special methods must every custom iterator implement?

<Accordion title="Solution">
  Every custom iterator must implement:

  * `__iter__()`
  * `__next__()`
</Accordion>

<div align="right">[Back to Top ↑](#topics-covered)</div>

## What is a Generator?

A **generator** is a special type of iterator created using a function that contains the `yield` keyword. Unlike a normal function that returns all values at once, a generator produces one value at a time and automatically remembers its execution state.

Generators are easier to write than custom iterators because Python automatically implements the iterator protocol for you.

### Creating a Generator

A function becomes a generator as soon as it contains a `yield` statement.

```python theme={null}
def numbers():
    yield 1
    yield 2
    yield 3


gen = numbers()

print(gen)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  <generator object numbers at 0x...>
  ```
</Accordion>

Notice that calling the function does **not** execute it immediately. Instead, it returns a **generator object**.

### Using `next()` with a Generator

The `next()` function starts the generator and retrieves one value at a time.

```python theme={null}
def numbers():
    print("Starting")

    yield 1
    yield 2
    yield 3

    print("Ending")


gen = numbers()

print(next(gen))
print(next(gen))
print(next(gen))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Starting
  1
  2
  3
  ```
</Accordion>

The `"Ending"` message is not printed because the generator pauses after the third `yield`. It executes the remaining statements only when resumed again.

### Using a Generator with a `for` Loop

Generators can be directly used in a `for` loop.

```python theme={null}
def numbers():
    yield 1
    yield 2
    yield 3


for number in numbers():
    print(number)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  2
  3
  ```
</Accordion>

The `for` loop automatically calls `next()` until the generator raises `StopIteration`.

#

### Exercise 2

Predict the output.

```python theme={null}
def greet():
    print("Hello")
    yield 1

g = greet()

print("Generator Created")
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Generator Created
  ```
</Accordion>

<Accordion title="Solution">
  ```text theme={null}
  Generator Created
  ```

  Calling a generator function does not execute its body immediately. It simply creates a generator object. The `"Hello"` message is printed only when the generator starts executing (for example, by calling `next(g)` or iterating over it).
</Accordion>

### Exercise 3

What is the output?

```python theme={null}
def values():
    yield "A"
    yield "B"

for value in values():
    print(value)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  A
  B
  ```
</Accordion>

<Accordion title="Solution">
  ```text theme={null}
  A
  B
  ```

  The `for` loop automatically retrieves values from the generator until it is exhausted.
</Accordion>

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Understanding `yield`

The `yield` keyword is used to produce a value from a generator. Unlike `return`, which terminates a function, `yield` **pauses** the function and preserves its current state. The next time the generator is resumed, execution continues from the statement immediately after the previous `yield`.

### `yield` vs `return`

| `return`                 | `yield`                              |
| ------------------------ | ------------------------------------ |
| Terminates the function  | Pauses the function                  |
| Returns a single value   | Produces one value at a time         |
| Function cannot resume   | Function resumes from the same point |
| Used in normal functions | Used in generator functions          |

### Execution Flow

```python theme={null}
def greet():

    print("Step 1")
    yield "Hello"

    print("Step 2")
    yield "World"

    print("Step 3")


g = greet()

print(next(g))
print(next(g))
next(g)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Step 1
  Hello
  Step 2
  World
  Step 3
  Traceback (most recent call last):
  ...
  StopIteration
  ```
</Accordion>

Notice that the function resumes exactly where it paused after each `yield`.

### State Preservation

One of the biggest advantages of generators is that they automatically preserve the values of local variables.

```python theme={null}
def counter():

    count = 1

    while count <= 3:
        yield count
        count += 1


g = counter()

print(next(g))
print(next(g))
print(next(g))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  1
  2
  3
  ```
</Accordion>

The variable `count` is **not reinitialized** each time. Its value is preserved between successive calls to `next()`.

### Multiple `yield` Statements

A generator can contain multiple `yield` statements.

```python theme={null}
def colors():

    yield "Red"
    yield "Green"
    yield "Blue"


for color in colors():
    print(color)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Red
  Green
  Blue
  ```
</Accordion>

Each `yield` produces one value before the generator pauses.

#

### Exercise 2

Predict the output.

```python theme={null}
def test():

    x = 5
    yield x

    x += 5
    yield x


g = test()

print(next(g))
print(next(g))
```

<Accordion title="Solution">
  Output ?

  <Accordion title="Show Output">
    ```text theme={null}
    5
    10
    ```
  </Accordion>

  The value of `x` is preserved between the two `yield` statements.
</Accordion>

### Exercise 3

What is the main difference between `return` and `yield`?

<Accordion title="Solution">
  * `return` terminates the function and returns a value.
  * `yield` pauses the function, returns a value, preserves its state, and resumes execution when requested again.
</Accordion>

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Generator Expressions

A **generator expression** provides a concise way to create generators. It is similar to a list comprehension but uses **parentheses `()`** instead of square brackets `[]`.

Generator expressions generate values **only when required**, making them memory efficient.

### Syntax

```python theme={null}
(expression for item in iterable)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: name 'iterable' is not defined
  ```
</Accordion>

### Example

```python theme={null}
squares = (x * x for x in range(5))

for square in squares:
    print(square)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  0
  1
  4
  9
  16
  ```
</Accordion>

### Generator Expression vs List Comprehension

```python theme={null}
# List Comprehension
numbers = [x * x for x in range(5)]

# Generator Expression
numbers = (x * x for x in range(5))
```

* A **list comprehension** stores all values in memory.
* A **generator expression** generates values one at a time.

#

### Exercise 2

Which symbol is used to create a generator expression?

<Accordion title="Solution">
  Generator expressions use **parentheses `()`**, whereas list comprehensions use **square brackets `[]`**.
</Accordion>

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Infinite Generators

Generators can produce **infinite sequences** because values are generated only when requested.

### Example

```python theme={null}
def even_numbers():

    number = 0

    while True:
        yield number
        number += 2


gen = even_numbers()

for _ in range(5):
    print(next(gen))
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  0
  2
  4
  6
  8
  ```
</Accordion>

#

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Fibonacci Generator

Generators are commonly used to generate mathematical sequences.

### Example

```python theme={null}
def fibonacci(limit):

    a, b = 0, 1

    while a <= limit:
        yield a
        a, b = b, a + b


for number in fibonacci(20):
    print(number)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  0
  1
  1
  2
  3
  5
  8
  13
  21
  ```
</Accordion>

#

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Memory Efficiency

One of the biggest advantages of generators is **memory efficiency**.

### List Example

```python theme={null}
numbers = [x for x in range(1000000)]
```

The above statement creates **one million values** in memory.

### Generator Example

```python theme={null}
numbers = (x for x in range(1000000))
```

The generator creates **only one value at a time**, significantly reducing memory usage.

### When to Use Generators

Use generators when:

* Working with large datasets.
* Reading large files.
* Processing streaming data.
* Producing values on demand.
* Creating infinite sequences.

#

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Iterator vs Generator

| Feature          | Iterator                      | Generator                  |
| ---------------- | ----------------------------- | -------------------------- |
| Created Using    | Class                         | Function                   |
| Implements       | `__iter__()` and `__next__()` | `yield`                    |
| State Management | Manual                        | Automatic                  |
| Code Size        | More                          | Less                       |
| Lazy Evaluation  | Yes                           | Yes                        |
| Memory Efficient | Yes                           | Yes                        |
| Best Use Case    | Custom iteration logic        | Sequential data generation |

> **Remember:** Every **generator** is an **iterator**, but not every **iterator** is a **generator**.

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Iterable vs Iterator vs Generator

```text theme={null}
                Iterable
       (list, tuple, set, dict)
                   │
             iter(iterable)
                   │
                   ▼
               Iterator
       (__iter__ + __next__)
                   ▲
                   │
         Generator Object
        (Created using yield)
```

* **Iterable** → An object that can produce an iterator.
* **Iterator** → Produces one value at a time.
* **Generator** → A special iterator created using the `yield` keyword.

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Real-World Applications

Generators are commonly used for:

* Reading large files line by line.
* Processing large datasets.
* Streaming data from APIs.
* Log processing.
* Data pipelines.
* Machine learning workflows.
* Infinite sequences.

### Example: Reading a File

```python theme={null}
def read_file(filename):

    with open(filename, "r") as file:
        for line in file:
            yield line.strip()


for line in read_file("data.txt"):
    print(line)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: [Errno 2] No such file or directory: 'data.txt'
  ```
</Accordion>

Instead of loading the entire file into memory, one line is processed at a time.

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Key Takeaways

* An **iterable** is an object that can produce an iterator.
* An **iterator** returns one value at a time using `next()`.
* A **generator** is a simpler way to create an iterator using `yield`.
* The `yield` keyword pauses execution and preserves the function's state.
* Generator expressions provide a concise syntax for creating generators.
* Generators are ideal for processing large datasets because they use **lazy evaluation**.
* Every generator is an iterator, but not every iterator is a generator.

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Check Your Understanding

**Question 1**

What is the purpose of the `iter()` function?

<Accordion title="Solution">
  The `iter()` function converts an iterable into an iterator.
</Accordion>

**Question 2**

Which special methods make an object an iterator?

<Accordion title="Solution">
  `__iter__()` and `__next__()`
</Accordion>

**Question 3**

What is the purpose of the `yield` keyword?

<Accordion title="Solution">
  The `yield` keyword pauses a generator, returns a value, preserves its state, and resumes execution from the same point when requested again.
</Accordion>

**Question 4**

What is the difference between `yield` and `return`?

<Accordion title="Solution">
  * `return` terminates the function.
  * `yield` pauses the function and allows it to continue later.
</Accordion>

**Question 5**

What is a generator expression?

<Accordion title="Solution">
  A generator expression is a concise way to create a generator using parentheses `()`.
</Accordion>

**Question 6**

Why are generators memory efficient?

<Accordion title="Solution">
  Generators create values only when they are requested instead of storing all values in memory.
</Accordion>

**Question 7**

Can generators be used in a `for` loop?

<Accordion title="Solution">
  Yes. A generator is an iterator and can be directly used in a `for` loop.
</Accordion>

**Question 8**

True or False: Every iterator is a generator.

<Accordion title="Solution">
  **False.** Every generator is an iterator, but not every iterator is a generator.
</Accordion>

**Question 9**

Name two real-world use cases of generators.

<Accordion title="Solution">
  Examples include:

  * Reading large files
  * Processing large datasets
  * Streaming API data
  * Log processing
  * Infinite sequences
</Accordion>

***

<div align="right">[Back to Top ↑](#topics-covered)</div>

## Context Managers

Whenever we open external resources such as files or database connections, they should be closed properly. Python provides the **with** statement to handle this automatically.

### Without a Context Manager

```python theme={null}
file = open("sample.txt", "r")
content = file.read()
print(content)
file.close()
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: [Errno 2] No such file or directory: 'sample.txt'
  ```
</Accordion>

If an exception occurs before `close()`, the file may remain open.

### Using a Context Manager

```python theme={null}
with open("sample.txt", "r") as file:
    content = file.read()
    print(content)
```

Output ?

<Accordion title="Show Output">
  ```text theme={null}
  Error executing: [Errno 2] No such file or directory: 'sample.txt'
  ```
</Accordion>

The file is automatically closed after leaving the `with` block.

### Custom Context Manager

A context manager implements two special methods:

* `__enter__()`
* `__exit__()`

```python theme={null}
class Database:
    def __enter__(self):
        print("Connection Opened")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("Connection Closed")

with Database():
    print("Executing Queries")
```

<Accordion title="Show Output">
  ```text theme={null}
  Connection Opened
  Executing Queries
  Connection Closed
  ```
</Accordion>

### Exercise 1

Open a file using the `with` statement and display its contents.

**Sample Input**

```python theme={null}
# Assume file sample.txt exists
```

**Expected Output**

```text theme={null}
Displays the file contents.
```

<Accordion title="Solution">
  ```python theme={null}
  with open("sample.txt", "r") as file:
      print(file.read())
  ```
</Accordion>

### Exercise 2

Create a context manager that prints `"Start"` on entry and `"End"` on exit.

**Sample Input**

```python theme={null}
with Demo():
    print("Inside")
```

**Expected Output**

```text theme={null}
Start
Inside
End
```

<Accordion title="Solution">
  ```python theme={null}
  class Demo:
      def __enter__(self):
          print("Start")
          return self

      def __exit__(self, exc_type, exc_value, traceback):
          print("End")

  with Demo():
      print("Inside")
  ```
</Accordion>

***