Asynchronous functions

So far, we have seen code and functions that are executed sequentially. In order. One instruction after another. Until one function is finished, the next one does not start.

However, this can be inefficient. Imagine a function that makes a request to an external server on the other side of the world. This request may take a few seconds. What do we do in the meantime? Wait and do nothing?

This is not efficient since this external request is blocking our program. Until the server responds, we will be waiting without doing anything. We can use this time to do other tasks.

Luckily, Python provides us with tools for asynchronous programming, where our program can continue to run while waiting. This is widely used in:

• 🕸️ Web scraping.
• 💽 Databases.
• 📁 Input and output with files.
• 🖥️ Graphical interface development.

In all these cases, at some point, we are blocked waiting for an external response. Next, we will see the differences between:

• 🐌 Synchronous programming: The traditional model seen so far, where we finish the previous task to execute the next one.
• 🏎️ Asynchronous programming: When a task blocks us, we continue executing others, allowing us to perform multiple tasks “in parallel.”

🐌 Let’s look at a synchronous example. The following code creates 10 processes. Imagine that the sleep simulates the time it takes for an external service to respond. This program takes 10 seconds to execute. Until one process is finished, the next one is not started.

import time
def process(process_id):
    time.sleep(1)
    print("End process:", process_id)

[process(i) for i in range(10)]

🏎️ Let’s see an asynchronous example using asyncio. In this case, when it detects that we are blocked, it continues executing other code. Therefore, all 10 processes are started at the same time, and after 1 second, everything is completed. We have gone from 10 seconds to 1.

import asyncio

async def process(process_id):
    await asyncio.sleep(1)
    print("End process:", process_id)

async def main():
    await asyncio.gather(*[process(i) for i in range(10)])

asyncio.run(main())

It is important to note that in this case, the order is not guaranteed. Even less so if we depend on something external whose response time can be variable. The use of await indicates where we should wait.

On the other hand, there are packages such as threading that allow similar tasks to be performed, although with a completely different concurrency model, beyond the scope of this book.

It is often useful when working with libraries that do not support async and allows you to take advantage of a multi-core CPU. Note that it does not offer parallel execution due to GIL.

import threading
import time

def process(process_id):
    time.sleep(1)
    print("End process:", process_id, flush=True)

threads = [threading.Thread(target=process, args=(i,)) for i in range(10)]
[i.start() for i in threads]
[i.join() for i in threads]

There is also the multiprocessing package, which does offer parallel execution, and we can benefit from multiple CPUs. This library is useful for computationally intensive tasks such as multiplying very large matrices.

from multiprocessing import Process
import time

def process(process_id):
    time.sleep(1)
    print(f"End process: {process_id}", flush=True)

def main():
    processes = [Process(target=process, args=(i,)) for i in range(10)]
    [p.start() for p in processes]
    [p.join() for p in processes]

if __name__ == '__main__':
    main()