Async kernel

Async kernel is an asynchronous Python Jupyter kernel with concurrent message handling.

Highlights

• Concurrent message handling
• Configurable backend
• Debugger client
  • anyio
    • asyncio backend (default)¹
    • trio backend
• IPython shell provides:
  • code execution
  • magic
  • code completions
  • history
• Thread-safe execution and cancellation (utilising aiologic synchronisation primitives).
  • Caller - code execution in a chosen event loop
  • Pending - wait/await/cancel the pending result
  • PendingGroup - An asynchronous context to automatically manage pending created in the context.
• GUI event loops
  • inline
  • ipympl
  • tk
  • qt
• Jupyter Kernel Subshells

Documentation

Installation

pip install async-kernel

Trio backend

To add a kernel spec for a trio backend.

pip install trio
async-kernel -a async-trio

See also: command line usage.

Asynchronous event loops

Async kernel uses Caller for concurrent message handling.

There are two callers:

• Shell - runs in the MainThread handling user related requests².
• Control - runs in a separate thread handling control related requests.

Messaging

Messages are received in a separate thread (per-channel) then handled in the associated thread (shell/control) concurrently according to the determined run mode.

Run mode

The run modes available are:

• RunMode.direct → Caller.call_direct: Run the request in the scheduler.
• RunMode.queue → Caller.queue_call: Run the request in a queue dedicated to the subshell, handler & channel.
• RunMode.task → Caller.call_soon: Run the request in a separate task.
• RunMode.thread → Caller.to_thread: Run the request in a separate worker thread.

These are the currently assigned run modes.

SocketID	shell	control
comm_close	direct	direct
comm_info_request	direct	direct
comm_msg	queue	queue
comm_open	direct	direct
complete_request	thread	thread
create_subshell_request	None	thread
debug_request	None	queue
delete_subshell_request	None	thread
execute_request	queue	queue
history_request	thread	thread
inspect_request	thread	thread
interrupt_request	direct	direct
is_complete_request	thread	thread
kernel_info_request	direct	direct
list_subshell_request	None	direct
shutdown_request	None	direct

Jupyter Kernel Subshells

Async kernel supports kernel subshells (JEP92). Each subshell provides a separate user_ns and shares the user_global_ns with the main shell.

Subshells are a useful means of providing separate namespaces and task management.

Subshells use the same callers as the main shell. This was a deliberate design choice with the advantage being that comm messages are always handled in the shell's thread (the main thread except when the shell is intentionally started in a different thread).

The active shell/subshell is controlled by setting a context variable. This is done by the kernel on a per-message basis. A context manager is also provided so that code can be executed in the context of a specific shell/subshell.

When a subshell is stopped its pending manager will cancel Pending (tasks) created in its context.

Kernel message handlers

No additional sockets are created for subshells.

Message handler methods are cached per channel and subshell ID, so when run mode is 'queue' (such as comm_msg), the message will be handled in a separate queue for the subshell/channel.

Further detail

• MsgType docs.
• Kernel.receive_msg_loop docs.
• Concurrency notebook.

Origin

Async kernel started as a fork of IPyKernel. Thank you to the original contributors of IPyKernel that made Async kernel possible.

---

1. Uvloop is not a dependency of async-kernel but will be used if it has been installed. ↩

2. The Shell can run in other threads with the associated limitations with regard to signalling and interrupts. ↩