# Abstracting over Computation

In object-oriented programming, methods (functions associated with classes or objects) are often considered "second-class citizens" because they are not as flexible as first-class functions. They cannot be passed around as arguments or stored independently in data structures. Instead, they are tied to the objects or classes they belong to, much like how verbs are tied to the nouns they describe.

Python, however, supports the concept of first-class functions. This means that functions in Python are treated as objects. You can assign them to variables, store them in data structures, pass them as arguments to other functions, and even return them as values from other functions. This provides a lot of flexibility and power.

Here's an example of how you might use a first-class function in Python:

```python
def greet(name):
    return f"Hello, {name}!"

def process(func, argument):
    return func(argument)

print(process(greet, "Alice"))  # prints "Hello, Alice!"
```

In this example, the `greet` function is passed as an argument to the `process` function, demonstrating the concept of first-class functions.

This flexibility allows for powerful programming paradigms such as functional programming, where functions can be used to create abstract computations that can be combined in flexible ways. This is different from the object-oriented paradigm, where the focus is more on the objects and their interactions. Both paradigms have their strengths and can often be used together effectively in Python.

### First-Class Functions

```python
def repeat(action:callable, n:int):
    for _ in range(n):
        action()
repeat(do_something,10)
```

The authors suggest that instead of writing a loop each time, the logic can be factored into a function that takes two arguments: a function to be executed and the number of times the function should be executed. The function is defined as repeat() and takes two arguments: action (a function) and n (an integer). The function then uses a for loop to execute the action function n times.

The authors provide an example of how to use the repeat() function by passing a function called do\_something and the integer value 10 as arguments. This will execute the do\_something function 10 times.

The authors use the concept of higher-order functions, where a function can take another function as an argument, to simplify the code and make it more modular. The use of higher-order functions is a common technique in functional programming.

The notation used in the code snippet includes the use of the range() function, which generates a sequence of numbers from 0 up to (but not including) the specified number. The notation for the repeat() function includes the use of the Callable type hint, which specifies that the action argument should be a function.

#### Lambdas

A lambda expression is a function literal that allows us to write a function without giving it a name.Lambdas are great for functions whose bodies are short expressions, but if multiple statements are needed in a function, a named function using "def" must be used instead.

The syntax of the lambda expression is as follows:

lambda argument: expression

```python
expected_value(coin_flip, lambda coin: 1.0 if coin == ”heads” else 0.0)
```

the argument is "coin" and the expression is "1.0 if coin == 'heads' else 0.0".

### Iterative Algorithms

Iterative algorithms converge to the correct result by repeatedly running the algorithm and making smaller and smaller improvements with each iteration. The example given is the approximation of the square root of a number, where the initial guess is repeatedly refined using the formula[^1]:tada:

$$\ x\_{n+1} = \frac{x\_n + \frac{a}{x\_n}}{2} $$

<pre class="language-python"><code class="lang-python"><strong>def sqrt(a: float) -> float:
</strong><strong>     x = a / 2 # initial guess
</strong>     x_n = a
     while abs(x_n - x) > 0.01:
        x = x_n
        x_n = (x + (a / x)) / 2
     return x_n
</code></pre>

A common issue with iterative algorithms is deciding **when to stop iterating**. In the given square root example, the algorithm stops when the change between the current guess and the next guess is less than 0.01. But this **hard-coded** value might not always be appropriate.

A proposed solution is to allow the caller to specify the precision as an argument to the function.

<pre class="language-python"><code class="lang-python">
def sqrt(a: float, threshold: float) -> float:
<strong>    x = a / 2 # initial guess
</strong>    x_n = a
    while abs(x_n - x) > threshold:
        x = x_n
        x_n = (x + (a / x)) / 2
    return x_n
</code></pre>

&#x20;This provides more control to the caller but still isn't perfect. If the function takes too long to reach the desired precision, the caller might want to stop early, which isn't possible with this approach.

#### Iterators and Generators

**Iterators and Generators:** Iterators and generators in Python allow us to handle sequences of data in a memory-efficient way. They can be used to abstract the process of iteration, separating the production of values from their consumption.

**Python's `for` Loop and Iterators:** Python's `for` loop uses an iterator under the hood. This is why the `for` loop works with different types of sequences like lists, dictionaries, sets, and ranges. The section also explains how different types of sequences have different default iterators (e.g., iterating over a dictionary gives you the keys by default).

<pre class="language-python"><code class="lang-python"><a data-footnote-ref href="#user-content-fn-2">for x in [3, 2, 1]: print(x)</a>
<strong><a data-footnote-ref href="#user-content-fn-3">for x in {3, 2, 1}: print(x)</a>
</strong><a data-footnote-ref href="#user-content-fn-4">for x in range(3): print(x)</a>
</code></pre>

**Generators and `yield` keyword:** Generators are a type of iterator that generate values on the fly. The `yield` keyword is used in a function like a return statement, but instead of returning a value and terminating the function, it produces a value and suspends the function’s execution. The function can then be resumed later on from where it left off, allowing it to produce a sequence of results over time, instead of computing them all at once and sending them back in a list, for example.

```python
from typing import Iterator

def sqrt(a: float) -> Iterator[float]:
    x = a / 2  # initial guess
    while True:
        x = (x + (a / x)) / 2
        yield x
```

With this version, we update x at each iteration and then yield the updated value. Instead of getting a single value, the caller of the function gets an iterator that contains an infinite number of iterations; it is up to the caller to decide how many iterations to evalu- ate and when to stop. The sqrt function itself has an infinite loop, but this isn’t a problem because execution of the function pauses at each yield which lets the caller of the function stop it whenever they want. To do 10 iterations of the sqrt algorithm, we could use `itertools.islice`:

**Using Generators with `itertools`:** The Python `itertools` module provides powerful tools for working with iterators and generators. The `itertools.islice` function can be used to consume **a specific number** of values from a generator.

```python
import itertools

iterations = list(itertools.islice(sqrt(25), 10))
iterations[-1]

```

**Implementing a `converge` Function:** The section ends with the implementation of a `converge` function that takes an iterator and a threshold, and yields values from the iterator until two consecutive values are within the threshold of each other. This function uses the `itertools.pairwise` function, which is available in Python 3.10 and later, to get pairs of consecutive values from the iterator.

```python
def converge(values: Iterator[float], threshold: float) -> Iterator[float]:
    for a, b in itertools.pairwise(values):
        yield a
        
        if abs(a - b) < threshold:
            break
```

Each function takes an iterator as an input and returns an iterator as an output. This doesn’t always have to be the case, but we get a major advantage when it is: iterator → iterator operations compose. We can get relatively complex behavior by starting with an iterator (like our sqrt example) then applying multiple operations to it. For example, somebody calling sqrt might want to converge at some threshold but, just in case the algorithm gets stuck for some reason, also have a hard stop at 10,000 iterations. We don’t need to write a new version of sqrt or even converge to do this; instead, we can use `converge` with `itertools.islice`:

```python
results = converge(sqrt(n), 0.001)
capped_results = list(itertools.islice(results, 10000))
```

We can write and test each operation— sqrt, converge, islice—in isolation and get complex behavior by combining them in the right way.

[^1]: The highlighted text is a mathematical equation that represents an iterative process for finding the square root of a number 'a'. The equation is called the Babylonian method or Heron's method and is a type of iterative algorithm. The equation takes the value of 'x' at the nth iteration and calculates the value of 'x' at the (n+1)th iteration.&#x20;

    Here, 'a' is the number whose square root is to be found, 'x' is the estimate of the square root at the nth iteration, and 'x+1' is the estimate of the square root at the (n+1)th iteration. The equation starts with an initial estimate of the square root, say 'x0', and then iteratively refines the estimate until it converges to the actual value of the square root.

    The equation is derived from the fact that if 'x' is an estimate of the square root of 'a', then the average of 'x' and 'a/x' is a better estimate. This is because the actual square root lies between 'x' and 'a/x', and the average of these two values is closer to the actual value.

    The Babylonian method is a fast and efficient way of finding the square root of a number, and it is widely used in computer programs and calculators. It is also a good example of an iterative algorithm, which is a type of algorithm that repeatedly applies a set of instructions until a desired result is obtained.

    Overall, the highlighted text represents an important mathematical equation that is used in various fields, including computer science and engineering.

[^2]: ```
    3 2 1
    ```

[^3]: ```
    1 2 3
    ```

[^4]: ```
    0 1 2
    ```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://combo-of-rl-and-trading.gitbook.io/rl/abstracting-over-computation.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.