feature: Introduce Validating Emulators with Noise Models to the SDK #1017

Altanali · 2024-07-10T23:19:25Z

Issue #, if available:

Description of changes:

This PR introduces the notion of device emulators and emulation passes to the BDK; device emulators provide some form of program compilation and validation and may apply a noise model to the program before simulating the program on the BDK LocalSimulator.

AwsDevice specific emulators are created based on the device properties and capabilities; noise models for gate-based QPUs are created based on calibration data retrieved from Braket service when creating an AwsDevice.

Testing done:

Unit testing covers 99% of emulator passes and noise model generation based on AwsDevice calibration data. (TODO: Complete unit tests for 100% coverage).

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the PR title format described in CONTRIBUTING
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…OpenQASM3.0 sources

…f attribute strings; introduce PyTket-to-QASM translations

…nherit from EmulatorPass

…nstead of gate name strings in GateFidelity data class.

…s using QPU device properties

…l inside AwsDevice

…not supported by Braket

…r than in global setup

codecov · 2024-07-15T22:44:00Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (ee63777) to head (baccbb1).

Additional details and impacted files

@@            Coverage Diff             @@
##              main     #1017    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          135       149    +14     
  Lines         9004      9495   +491     
  Branches      2019      2127   +108     
==========================================
+ Hits          9004      9495   +491

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

maolinml

Very good work! High level comment

For connectivity graph, can we use indirected graph instead of directed graph?
You have spent lots of effort in handling the differences across different devices. Let us add some comments in the code so that others can understand the logic better, which will also help future maintenance and developments.

maolinml · 2024-07-16T18:00:27Z

src/braket/aws/aws_device.py

+            >>> print(result.result().measurement_counts)
+
+        Returns:
+            Emulator: An emulator for this device, if this is not a simulator device.


Should we add in "Returns" that an error will be thrown if the device is not a QPU?

Agreed - added!

maolinml · 2024-07-16T18:05:26Z

src/braket/aws/aws_device.py

+        """
+        A device emulator mimics the restrictions and noise of an AWS QPU by validating and
+        compiling programs before running them on a simulated backend. An emulator can be used
+        as a soft check that a program can run on an AwsDevice.


Let's emphasize that the emulator is tied to the QPU, or 1-1 mapped to the QPU. So maybe let's say "A device emulator mimics the restrictions and noise of the AWS QPU by validating and compiling programs before running them on a simulated backend. An emulator can be used as a soft check that a program can run on the target AwsDevice.

This is a good point, I've made the proposed changes to clarify that the emulator's are specific to the exact AwsDevice being used.

src/braket/aws/aws_device.py

maolinml · 2024-07-16T18:15:55Z

src/braket/aws/aws_device.py

+    ) -> None:
+        """
+        Runs all non-modifying emulator passes on the input program and raises an
+        error if any device-specific criterion are not met by the program. If the


Not super important: can we check the usage of "criterion" vs "criteria" in doc-string and function names? I thought the former is singular and the latter is plural, but I am not the best person to check the grammar..

Your call out is correct, this was a mistake on my part! "Criteria" should have been used here (and in line 926) where we are referencing multiple device constraints/criteria.

maolinml · 2024-07-16T18:19:53Z

src/braket/aws/aws_device.py

+        inputs: Optional[dict[str, float]] = None,
+    ) -> QuantumTask:
+        """Emulate a quantum task specification on this quantum device emulator.
+        A quantum task can be a circuit or an annealing problem. Emulation


Can the task be annealing problem? If yes, why do we want to do annealing, given that Dwave is long gone.

The task cannot be an annealing problem, this was a typo, thank you!

maolinml · 2024-07-16T21:38:04Z

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

+
+class ConnectivityCriterion(EmulatorCriterion):
+    """
+    args:


The argument directed is missing in the docstring.

Also I am a bit lost by the different combination of the arguments. It would be good to summarize the valid argument combinations, or try to somehow simplify this class.

maolinml · 2024-07-16T21:44:20Z

src/braket/emulators/emulator.py

+            task_specification (Union[Circuit, OpenQasmProgram]): Specification of a quantum task
+                to run on device.
+            shots (Optional[int]): The number of times to run the quantum task on the device.
+                Default is `None`.


Shouldn't it be "Default is 0" for shots?

Shots default=None follows the default value for the shots argument in the run methods for aws_device and local_simulator.

maolinml · 2024-07-16T21:53:47Z

src/braket/emulators/emulator.py

+        """
+        Passes the input program through all EmulatorPass objects contained in this
+        emulator and applies the emulator's noise model, if it exists, before
+        retruning the compiled program.


"retruning", typo?

Good catch! Fixed!

maolinml · 2024-07-16T22:00:26Z

src/braket/emulators/emulator_interface.py

+                emulator's validation passes.
+        """
+        for emulator_pass in self._emulator_passes:
+            if isinstance(emulator_pass, EmulatorCriterion):


Why do we need this if-statement?

In validate(), we only want to run the EmulatorCriterion emulator passes that are only used for validation, and are non-modifying; this check is used to avoid modifying EmulatorPass instances (currently, there are none, but if we have mapping/routing or nativization passes, we will want to avoid applying those to the Program when calling validate()).

maolinml · 2024-07-16T22:03:35Z

src/braket/emulators/emulator_passes/criteria/gate_criterion.py

+            circuit (Circuit): The Braket circuit whose gates to validate.
+        """
+        idx = 0
+        while idx < len(circuit.instructions):


High level questions. I thought if there is a verbatim box in the circuit, then all the gates have to be native gates.

Does the following validation satisfies this requirement? I wasn't able to tell yet.

Can we simply check if there is a verbatim box in the circuit, and check against either the supported gate set and native gate set accordingly for all the gates?

If there is a verbatim box in the circuit, all the gates don't have to be native gates - just the ones in the verbatim box. Supported gates outside the verbatim box can still be nativized without any issue!

However, if there is a verbatim box, then all qubit references must be to hardware qubits (this is checked in ConnectivityCriterion).

Because of the comment in response to (1), we need to use both the supported gate set and the native gate set when validating a circuit.

…to ensure user-provided undirected graphs have symmetric forwards/backwards edges

speller26

Still reviewing; great work though!

speller26 · 2024-07-22T03:47:50Z

src/braket/emulators/emulator_interface.py

+            task_specification = emulator_pass(task_specification)
+        return task_specification
+
+    def run_validation_passes(self, task_specification: ProgramType) -> None:


Might be better to call this validate

speller26 · 2024-07-22T03:48:06Z

src/braket/emulators/emulator_interface.py

+    def __init__(self, emulator_passes: Iterable[EmulatorPass] = None):
+        self._emulator_passes = emulator_passes if emulator_passes is not None else []
+
+    def run_program_passes(self, task_specification: ProgramType) -> ProgramType:


May just run_passes?

speller26 · 2024-07-22T03:49:48Z

src/braket/aws/aws_emulator_helpers.py

+)
+
+
+def create_qubit_count_criterion(properties: DeviceCapabilities) -> QubitCountCriterion:


Just qubit_count_criterion is fine; same goes for other create methods

speller26 · 2024-07-22T05:19:50Z

src/braket/aws/aws_emulator_helpers.py

+    return GateCriterion(supported_gates=supported_gates, native_gates=native_gates)
+
+
+@singledispatch


singledispatch produces really ugly documentation when used on public functions; only use it on private methods, like so:

def connectivity_criterion(...): return _connectivity_criterion(...) @singledispatch def _connectivity_criterion(...): ...

Updated! Thank you for this callout, I wasn't aware of that interaction!

speller26 · 2024-07-22T05:22:36Z

src/braket/aws/aws_emulator_helpers.py

+    return GateConnectivityCriterion(gate_connectivity_graph)
+
+
+def get_qpu_gate_translation(


Are the names not already standardized across QPUs? This shouldn't be necessary.

There are a few places where translation to a Braket gate name was necessary, i.e. gate names in Rigetti calibration data ("CPHASE" is used instead of the standard "cphaseshift") or in IonQs native gate set (translating "GPI"/"GPI2" to "GPi"/"GPi2"). I felt this was a little cleaner than doing translations in each place required.

speller26 · 2024-07-22T05:23:36Z

src/braket/aws/aws_noise_models.py

+ The following gate duration values are not available through Braket device
+ calibration data and must be hardcoded.
+"""
+QPU_GATE_DURATIONS = {


Make this private; in fact, make things private whenever possible.

speller26 · 2024-07-22T05:23:59Z

src/braket/aws/aws_noise_models.py

+        self._validate_two_qubit_specs()
+
+
+def create_device_noise_model(properties: DeviceCapabilities, arn: str) -> NoiseModel:


device_noise_model is sufficient

speller26 · 2024-07-22T05:24:42Z

src/braket/emulators/emulator_passes/emulator_pass.py

+        """Runs the emulator pass on the provided program.
+        Args:
+            program (ProgramType): The program to run the emulator pass on.
+        Returns:
+            ProgramType: The program after the emulator pass has been applied.


nit

Suggested change

"""Runs the emulator pass on the provided program.

Args:

program (ProgramType): The program to run the emulator pass on.

Returns:

ProgramType: The program after the emulator pass has been applied.

"""Runs the emulator pass on the provided program.

Args:

program (ProgramType): The program to run the emulator pass on.

Returns:

ProgramType: The program after the emulator pass has been applied.

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

speller26 · 2024-07-22T05:26:02Z

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

+        Args:
+            circuit (Circuit): The Braket circuit whose gate operations to
+                validate.
+        """


Should also add a Raises section

Added a Raises section to all methods where an exception is raised!

…e in aws_emulator_helpers; general documentation cleanup.

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

speller26 · 2024-07-29T07:39:52Z

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

+            or a valid DiGraph or dict representation of a connectivity graph is not provided.
+        """
+
+        if not (connectivity_graph or fully_connected):


Maybe also raise an error if both are provided?

speller26 · 2024-07-29T07:54:14Z

src/braket/emulators/emulator_passes/emulator_pass.py

+ProgramType = TypeVar("ProgramType")
+
+
+class EmulatorPass(ABC):


Should be EmulatorPass(ABC, Generic[ProgramType]); also, how about EmulationPass?

Updated in b698648!

speller26 · 2024-07-29T07:54:34Z

src/braket/emulators/emulator_passes/emulator_pass.py

+from abc import ABC, abstractmethod
+from typing import TypeVar
+
+ProgramType = TypeVar("ProgramType")


For generics, it's fine to use just T = TypeVar("T")

I've opted to use "ProgramType" just as it makes it clearer throughout code/documentation that an EmulatorPass expects some generic program as input - let me know if this approach makes sense!

src/braket/emulators/emulator_passes/criteria/emulator_criterion.py

src/braket/emulators/emulator_passes/criteria/gate_connectivity_criterion.py

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

speller26 · 2024-07-29T08:15:37Z

src/braket/emulators/emulator_passes/criteria/connectivity_criterion.py

+from braket.registers.qubit_set import QubitSet
+
+
+class ConnectivityCriterion(EmulatorCriterion):


ConnectivityCriterion(EmulatorCriterion[Circuit]); also, how about naming all the Criterion subclasses __Validators?

src/braket/emulators/__init__.py

src/braket/emulators/emulator_passes/criteria/gate_criterion.py

src/braket/emulators/emulator.py

src/braket/emulators/emulator_interface.py

src/braket/aws/aws_emulator_helpers.py

… program type in EmulatorPasses

…n a designated directory.

…lar dependencies

…Emulator._get_local_simulator_backend overwriting user provided backend name.

…dk-python into only_validation

…exception raised by an EmulatorPass in order to modify the message with the Emulator name.

…unsupported device capabilities

speller26 · 2024-08-01T06:42:56Z

src/braket/emulation/emulation_passes/gate_device_passes/gate_validator.py

+        supported_gates: Optional[Iterator[str]] = None,
+        native_gates: Optional[Iterator[str]] = None,


Suggested change

supported_gates: Optional[Iterator[str]] = None,

native_gates: Optional[Iterator[str]] = None,

supported_gates: Optional[Iterable[str]] = None,

native_gates: Optional[Iterable[str]] = None,

🤦‍♂️ Thanks for catching this, fixed!

src/braket/aws/aws_device.py

speller26 · 2024-08-01T16:36:53Z

src/braket/aws/aws_emulation.py

+    return GateConnectivityValidator(gate_connectivity_graph)
+
+
+def _get_qpu_gate_translations(


I'll be very happy when this is no longer needed

speller26 · 2024-08-01T16:53:16Z

src/braket/emulation/emulation_passes/gate_device_passes/gate_validator.py

+            self._supported_gates = set(BRAKET_GATES[gate.lower()] for gate in supported_gates)
+        except KeyError as e:
+            raise ValueError(f"Input {str(e)} in supported_gates is not a valid Braket gate name.")
+
+        try:
+            self._native_gates = set(BRAKET_GATES[gate.lower()] for gate in native_gates)


nit: frozenset instead of set

Changed to frozenset!

speller26 · 2024-08-02T00:24:50Z

src/braket/emulation/emulation_passes/emulation_pass.py

This is general enough that it perhaps belongs in its own module (maybe passes or transpilation)? The class itself might even be named just Pass; at the very least, EmulationPass is to specific for what it's actually capable of.

I've moved it to a module called "passes" under the name "BasePass" (to try and avoid collisions with the pass keyword even if the casing is different). Now, ValidationPass inherits directly from BasePass!

speller26 · 2024-08-02T00:40:33Z

Looking good to me, let's see if @krneta has any feedback!

ltnln added 30 commits June 2, 2024 23:35

feat: Introduce emulator criterion

ea85675

feat: Introduce NativeGateCriterion + Unit Tests

28ec84e

feat: Introduce SupportedGateCriterion + Unit Tests

f41c52b

feat: Introduce Connectivity Criterion + Unit Tests

fee7888

feat: Add __init__.py to emulators/criteria module

6f216ab

feat: Add OpenQASM3.0-to-Pytket translations

4e909e5

feat: Introduce PytketProgramContext to generate PytketCircuits from …

e3e4593

…OpenQASM3.0 sources

feat: Begin emulator pass implementation

eda10b7

feat: Change gate translations to use pytket OpType objects instead o…

096aae7

…f attribute strings; introduce PyTket-to-QASM translations

Merge branch 'add_pytket_context' into add_tket_mapping_pass

0483fb9

feat: Introduce Pytket-to-Openqasm Translators

a801ad4

fix: Covert from radians to Tket half-turns in PytketContext

36eff02

feat: add MEASUREMENT_REGISTER_NAME to translation constants

a328916

feat: Introduce EmulatorPass abstraction and have EmulatorCriterion i…

94b4226

…nherit from EmulatorPass

fix: Update __init__.pys for emulator_pass and criteria directories

1d37b0a

fix: Correct import paths in emulator_passes source and tests

52e0465

feat: Introduce Gate Connectivity Criterion

467c27c

fix: Correct qubit order in error print statement

5dca87d

feat: Add GateConnectivityCriterion tests

4c92658

feat: Basic AWS Noise Model

88c9530

fix: Discard IonQ Single Qubit Gate Fidelities and use Gate classes i…

1ff8cfa

…nstead of gate name strings in GateFidelity data class.

Merge branch 'emulation_criteria_basic' into end_to_end_prototype

3355759

fix: Include EmulatorCriterion in __init__.py for emulator_passes module

edef55d

feat: Introduce Emulator Interface and AwsEmulator

14d09e8

fix: Fix gate device noise model file name typo

c229adc

Merge branch 'add_gate_connectivity' into end_to_end_prototype

362ef45

fix: Remove stray print statement

74fecd0

feat: Introduce emulators and instantiate emulators within AWS device…

28e79bc

…s using QPU device properties

feat: Introduce Qubit Count Criteria

dc4a9cf

feat: Move NoiseModel generation to a set of helper functions and cal…

56e730e

…l inside AwsDevice

speller26 requested a review from maolinml July 15, 2024 21:06

ltnln added 3 commits July 15, 2024 14:30

test: Add test for IonQ create_noise_model dispatch for native gates …

20c59a2

…not supported by Braket

fix: Place local_simulator._simulator_device setup in a fixture rathe…

073fb4e

…r than in global setup

fix: Compare arn strings against explicit enum string values

3c9fd67

Altanali changed the title ~~feat: Introduce Validating Emulators with Noise Models to the SDK~~ feature: Introduce Validating Emulators with Noise Models to the SDK Jul 15, 2024

maolinml reviewed Jul 16, 2024

View reviewed changes

ltnln added 2 commits July 18, 2024 09:40

change: Update documentation, add check to GateConnectivityCriterion …

0601c63

…to ensure user-provided undirected graphs have symmetric forwards/backwards edges

change: Clean up GateConnectivityCriterion instantiation logic.

c5b9fc3

speller26 reviewed Jul 22, 2024

View reviewed changes

change: Add Raises documentation and make dispatched functions privat…

34b94ae

…e in aws_emulator_helpers; general documentation cleanup.

speller26 reviewed Jul 29, 2024

View reviewed changes

src/braket/emulators/emulator.py Outdated Show resolved Hide resolved

src/braket/emulators/emulator_interface.py Outdated Show resolved Hide resolved

src/braket/aws/aws_emulator_helpers.py Outdated Show resolved Hide resolved

Altanali and others added 7 commits July 29, 2024 11:56

Merge branch 'amazon-braket:main' into only_validation

70881c6

change: Use 'validator' in place of 'criterion', fix usage of generic…

b698648

… program type in EmulatorPasses

change: Reorganize modules to place all gate-device emulator passes i…

e1cb307

…n a designated directory.

fix: Correct __init__.pys for emulator_passes module and remove circu…

3b500a3

…lar dependencies

change: Rename emulator_passes directory to emulation_passes

7d7279e

change: Remove outdated connectivity_criterion.py file

8c63da6

Merge branch 'main' into only_validation

df11f66

speller26 requested a review from mbeach-aws July 30, 2024 19:47

ltnln added 4 commits July 30, 2024 13:15

fix: Use deepcopy before applying emulation passes to a program, fix …

a79e07d

…Emulator._get_local_simulator_backend overwriting user provided backend name.

Merge branch 'only_validation' of github.com:Altanali/amazon-braket-s…

96e917a

…dk-python into only_validation

fix: Fix Emulator raising a new Exception instance from the original …

f30379c

…exception raised by an EmulatorPass in order to modify the message with the Emulator name.

change: Change error message when trying to create a NoiseModel from …

9bddfbc

…unsupported device capabilities

speller26 reviewed Aug 2, 2024

View reviewed changes

ltnln added 3 commits August 2, 2024 12:27

feat: Replace 'EmulationPass' with more general BasePass class

4626332

change: Remove mentions of 'emulation' from the BasePass module.

e3ca8a5

change: Fix style in BasePass.py

baccbb1

		)


		def create_qubit_count_criterion(properties: DeviceCapabilities) -> QubitCountCriterion:

		return GateCriterion(supported_gates=supported_gates, native_gates=native_gates)


		@singledispatch

		return GateConnectivityCriterion(gate_connectivity_graph)


		def get_qpu_gate_translation(

		self._validate_two_qubit_specs()


		def create_device_noise_model(properties: DeviceCapabilities, arn: str) -> NoiseModel:

		ProgramType = TypeVar("ProgramType")


		class EmulatorPass(ABC):

		from braket.registers.qubit_set import QubitSet


		class ConnectivityCriterion(EmulatorCriterion):

		supported_gates: Optional[Iterator[str]] = None,
		native_gates: Optional[Iterator[str]] = None,

		return GateConnectivityValidator(gate_connectivity_graph)


		def _get_qpu_gate_translations(

feature: Introduce Validating Emulators with Noise Models to the SDK #1017

Are you sure you want to change the base?

feature: Introduce Validating Emulators with Noise Models to the SDK #1017

Conversation

Altanali commented Jul 10, 2024 • edited Loading

Merge Checklist

General

Tests

codecov bot commented Jul 15, 2024 • edited Loading

Codecov Report

maolinml left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

speller26 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Altanali Jul 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

speller26 commented Aug 2, 2024

Altanali commented Jul 10, 2024 •

edited

Loading

codecov bot commented Jul 15, 2024 •

edited

Loading

Altanali Jul 30, 2024 •

edited

Loading