Concurrent message handling (notebook)¶

async-kernel is capable of concurrent message handling. It provides a separate message handler for each channel and msg_type.

As messages are received they are queued for execution by the message handler.

Callers¶

async-kernel provides two Callers for the shell and control channels. The shell's Caller is associated with the thread where the kernel is started, normally this is the MainThread. In CPython, the Caller for the control channel is started as a child of the shell caller with it's own thread and asynchronous backend ('asyncio' or 'trio').

ZMQ message handling (CPython)¶

The kernel interface also starts two additional threads dedicated to receive the messages on the shell and control channels, the specific function call is session.recv(socket, mode=zmq.BLOCKY, copy=False).

Each thread blocks until a new message is received on the zmq socket. When a message is received, the kernel's message_handler is called with detail of the message (Job), the channel and a function that is to be used to send the response corresponding to the message. The message_handler obtains the dedicated handler according to the message_type, run_mode, channel and subshell_id and schedules execution of the dedicated message handler in Caller's thread.

Shell messaging¶

Both execute_request and com_msg are always run in the shell's thread (normally the MainThread). All other messages on the shell channel are run in the control thread.

Control messaging¶

All messages on the control channel are handled in the control thread.

In [1]:

import threading

import ipywidgets as ipw
from aiologic import Event

from async_kernel import utils

kernel = utils.get_kernel()

import threading import ipywidgets as ipw from aiologic import Event from async_kernel import utils kernel = utils.get_kernel()

Execute request run mode¶

The run mode of execute_request can be modified to run an execute_request separately as a task or thread.

There are a few options to modify the run mode.

• Metadata
• Directly in code
• tags
• Message header (in custom messages)

Warning

Only Jupyter lab is known to allow concurrent execution of cells.

Code for example¶

• This example requires ipywidgets
• Ensure you are running an async-kernel

Lets define a function that we'll reuse for the remainder of the notebook.

In [2]:

async def demo():

    print(f"Thread name: '{threading.current_thread().name}'")
    button = ipw.Button(description="Finish")
    event = Event()
    button.on_click(lambda _: event.set())
    display(button)
    await event
    button.close()
    print(f"Finished ... thread name: '{threading.current_thread().name}'")
    return "Finished"

async def demo(): print(f"Thread name: '{threading.current_thread().name}'") button = ipw.Button(description="Finish") event = Event() button.on_click(lambda _: event.set()) display(button) await event button.close() print(f"Finished ... thread name: '{threading.current_thread().name}'") return "Finished"

Lets run it normally (queue)

In [3]:

await demo()

await demo()

Thread name: 'MainThread'

Button(description='Finish', style=ButtonStyle())

Run mode: task¶

task mode instructs the kernel to execute the code in a task separate to the queue, Both task and thread execute modes can be started when the kernel is busy executing. There is no imposed limitation on the number of tasks (or threads) that can be run concurrently.

See also the Caller example on how to call directly.

In [4]:

# task
# Tip: try running this cell while the previous cell is still busy.
await demo()

# task # Tip: try running this cell while the previous cell is still busy. await demo()

Thread name: 'MainThread'

Button(description='Finish', style=ButtonStyle())

Run mode: thread¶

In [5]:

thread

# This time we'll use the tag to run the cell in a worker thread
await demo()

# This time we'll use the tag to run the cell in a worker thread await demo()

Thread name: 'MainThread'

Button(description='Finish', style=ButtonStyle())

In [6]:

# thread
%callers # magic provided by async-kernel

# thread %callers # magic provided by async-kernel

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480 ← current

We can also specify CallerCreateOptions as part of the top line

In [7]:

# thread name="My thread"
%callers

# thread name="My thread" %callers

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480

My thread                   ✓                     My thread        139643756604752 ← current

Asynchronous magic¶

Asynchronous line (%) and cell (%%) magic functions are supported. Any line or cell magic that returns an awaitable is awaited before proceeding.

• thread magic
• asyncio magic
• trio magic

thread magic¶

This will run the code in a thread. When no settings are provided a cell worker thread is used.

Comparing thread magic with thread run mode¶

• thread magic (%%thread) is an asynchronous magic that executes the associated code in a separate thread.
• thread run mode (# thread) instructs the kernel to run the entire cell in a separate thread, bypassing the shell execute request queue.

In [8]:

# Run the magic 'callers' in a caller worker thread.
%thread %callers 

# Run the magic 'callers' in a caller worker thread. %thread %callers

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480 ← current

My thread                   ✓                     My thread        139643756604752

To specify a thread (caller) by name

In [9]:

%%thread name="My executor" 
%callers

%%thread name="My executor" %callers

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480

My thread                   ✓                     My thread        139643756604752

My executor                 ✓                    My executor       139643756600000 ← current

Many of arguments accepted on Caller.get are also supported. Let's use a thread with a trio backend.

In [10]:

%%thread name="My trio executor" backend=trio
%callers

import trio

await trio.sleep(0)

%%thread name="My trio executor" backend=trio %callers import trio await trio.sleep(0)

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480

My thread                   ✓                     My thread        139643756604752

My executor                 ✓                    My executor       139643756600000

My trio executor            ✓                  My trio executor    139643752750912 ← current

Specify the backend¶

Code that is written for a specific backend ('asyncio' or 'trio') can be run in the same thread with one of the following:

• Line magic - The code following the magic on the same line is run using the specified backend.

  • %trio
  • %asyncio

• Cell magic - The code block is run using the specified backend.

  • %%trio
  • %asyncio

Note: trio must be installed for this demo to work.

In [11]:

%asyncio await asyncio.sleep(0) # This code gets run in an asyncio task 
%trio await trio.sleep(0) # trio run as line magic

%asyncio await asyncio.sleep(0) # This code gets run in an asyncio task %trio await trio.sleep(0) # trio run as line magic

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[11], line 1
----> 1 await asyncio.sleep(0) # This code gets run in an asyncio task

NameError: name 'asyncio' is not defined

In [12]:

%%trio # trio cell magic

def print_info():
    from aiologic.lowlevel import current_async_library
    print(f"""
    Kernel backend: {get_ipython().kernel.interface.backend}
    Current backend: { current_async_library()}
    """)   


print_info()
await trio.sleep(0)

%callers

%asyncio print_info()
%asyncio %callers

await trio.sleep(0)

%%trio # trio cell magic def print_info(): from aiologic.lowlevel import current_async_library print(f""" Kernel backend: {get_ipython().kernel.interface.backend} Current backend: { current_async_library()} """) print_info() await trio.sleep(0) %callers %asyncio print_info() %asyncio %callers await trio.sleep(0)

    Kernel backend: asyncio
    Current backend: trio
    

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080 ← current

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480

My thread                   ✓                     My thread        139643756604752

My executor                 ✓                    My executor       139643756600000

My trio executor            ✓                  My trio executor    139643752750912

    Kernel backend: asyncio
    Current backend: asyncio
    

           Name          Running Protected          Thread                 Caller       

─────────────────────────────────────────────────────────────────────────────────────────────

Shell                       ✓       🔐            MainThread       139643859816080 ← current

Control                     ✓       🔐             Control         139643797731952

async_kernel_caller         ✓                async_kernel_caller   139643756594480

My thread                   ✓                     My thread        139643756604752

My executor                 ✓                    My executor       139643756600000

My trio executor            ✓                  My trio executor    139643752750912