Using PennyLane

Release news

Development

API

Internals

  1. Docs
  2. qml.concurrency

qml.concurrency

Warning

This module is intended for developer use only.

This module contains concurrent execution functionality for PennyLane workloads using a task-based executor abstraction.

Executors and backends

Executor implementations in PennyLane build functional abstractions around the following API calls:

# Single function execution on the executor backend
executor.submit(fn: Callable, *args, **kwargs)

# Map provided function to all iterables provided in ``args``,
# with each index in *args running on the executor backend
executor.map(fn: Callable, *args, **kwargs)

# Map provided function to all tuples provided in ``args``,
# with each paired values running on the executor backend
executor.starmap(fn: Callable, args, **kwargs)

These loosely mirror the native Python functions map and itertools.starmap, whilst also following the design from the concurrent.futures.Executor interface. To allow for Liskov substitution, we define a uniform signature for functions with *args and **kwargs, allowing reduced coupling between the caller and the executor backend. This allows an ease of scaling from local execution to remote execution by controlling the executor being instantiated.

Local and remote function execution through an instantiated executor is available through the above API calls, or via a functor-like dispatch mechanism:

executor = create_executor(...)
executor("submit", myfunc, *mydata)

Support functions to query supported backends and initialize them are provided through the following functions.

Supported executors

PennyLane currently has support for a collection of executor implementations using local (Python native standard library) execution, and remote (third-party distributed) implemented backends.

Native Python executors

Submodule for concurrent executors relying on the Python standard library.

All executor functionality in this module is implemented directly using native Python abstractions.

PyNativeExec([max_workers, persist])

Python standard library backed ABC for executor API.

ProcPoolExec([max_workers, persist])

concurrent.futures.ProcessPoolExecutor class executor.

ThreadPoolExec([max_workers, persist])

concurrent.futures.ThreadPoolExecutor class executor.

MPPoolExec([max_workers, persist])

Python standard library executor class backed by multiprocessing.Pool.

SerialExec([max_workers, persist])

Serial Python standard library executor class.

External package-backed executors

Submodule for concurrent executors relying on 3rd-party packages.

All executor functionality in this module is implemented using external packages to handle execution and orchestration.

DaskExec([max_workers, persist, client_provider])

Dask distributed executor wrapper.

MPICommExec([max_workers, persist])

MPICommExecutor abstraction class functor.

MPIPoolExec([max_workers, persist])

MPIPoolExecutor abstraction class executor.

Executor API

Contains concurrent executor abstractions for task-based workloads.

All of the base abstractions for building an executor follow a simplified concurrent.futures.Executor interface. Given the differences observed in support for *args and **kwargs in various modes of execution, the abstractions provide a fixed API to interface with each backend, performing function and argument transformations, where necessary.

To build a new executor backend, the following classes provide scaffolding to simplify abstracting the function call signatures between each backend interface layer.

ExecBackendConfig([submit_fn, map_fn, ...])

Executor backend configuration data-class.

RemoteExec([max_workers, persist])

Abstract base class for defining a task-based parallel executor backend.

IntExec([max_workers, persist])

Executor class for native Python library concurrency support.

ExtExec([max_workers, persist])

Executor class for external packages providing concurrency support.
Previous
Next