Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal global backend _Global #1440

Merged
merged 69 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
bdbdc7e
GlobalBackend -> _Global
Sep 9, 2024
98d07ff
GlobalBackend -> _Global
Sep 9, 2024
b3fcc2a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 9, 2024
f8a3cfc
disable not callable error
Sep 9, 2024
5b312bf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 9, 2024
4697a9f
disable not callable error
Sep 9, 2024
e58157b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 9, 2024
583d26b
fix bug
Sep 10, 2024
2419d7d
add funcs
Sep 10, 2024
e8126a2
Merge branch 'master' into qibo_global
Sep 16, 2024
32ed23a
update test funcs
Sep 16, 2024
435c5f5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 16, 2024
94cb879
update review
Sep 18, 2024
53fa831
Merge branch 'master' into qibo_global
Oct 1, 2024
efb22c5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 1, 2024
b2f82d5
update test files
Oct 7, 2024
c828f67
Merge branch 'master' into qibo_global
Oct 7, 2024
8b3a95a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
352015b
fix test
Oct 7, 2024
59b0c71
qcnn.py get_backend -> get_backend_name
Oct 7, 2024
907bebb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
7cd16a4
remove resolve_global
Oct 7, 2024
a3c993d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
79ef360
remove resolve_global
Oct 7, 2024
b3614a4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
69189f4
remove resolve_global
Oct 7, 2024
0f68533
fix test_backends_global
Oct 7, 2024
b9b10d7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
2dc6797
add reset_global
Oct 7, 2024
cf39210
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 7, 2024
b7209ad
add reset global
Oct 7, 2024
f0a4c4c
Merge branch 'qibo_global' of github.com:qiboteam/qibo into qibo_global
Oct 8, 2024
c6391b6
fix error mitigation
Oct 8, 2024
23f6054
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 8, 2024
a63fbe9
update docs (replace 'GlobalBackend')
Oct 8, 2024
a3083d7
Merge branch 'qibo_global' of github.com:qiboteam/qibo into qibo_global
Oct 8, 2024
15be594
coverage
Oct 8, 2024
6a1c921
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 8, 2024
d4670f6
fix test_set_get_transpiler
Oct 8, 2024
6b04315
Merge branch 'qibo_global' of github.com:qiboteam/qibo into qibo_global
Oct 8, 2024
5c32260
Merge branch 'master' into qibo_global
Oct 8, 2024
dcd318a
add clear in conftest.py
Oct 9, 2024
3e99140
pytest autouse & default transpiler prototype
Oct 9, 2024
535c5a4
hw backend prototype
Oct 10, 2024
eba4f5e
hw backend prototype
Oct 10, 2024
5613c4c
fix: hw attributes check
Oct 11, 2024
b44c13a
fix: temporary _default_transpiler for current transpiler
Oct 14, 2024
11548da
feat: create NativeGates from list[str]
Oct 14, 2024
fcfb579
Merge branch 'master' into qibo_global
Oct 14, 2024
e0292cc
fix: return value of default transpiler
Oct 14, 2024
3c29f61
feat: test files for _Global._default_transpiler
Oct 15, 2024
3eeff46
fix: unused import/vars
Oct 15, 2024
a4ee205
fix: else after return
Oct 15, 2024
be12cf9
fix: improve star/line/grid connectivity helper functions
Oct 15, 2024
84dc20c
fix: remove Nativegates.from_gate(str)
Oct 15, 2024
bdb31b5
fix: remove _Global._backend usage / deprecate get_backend_name
Oct 15, 2024
6fe4205
fix: remove pragma / improve _execute_circuit in error_mitigation.py
Oct 15, 2024
9e7cc6b
fix: add pragma
Oct 15, 2024
8c02e78
Merge branch 'master' into qibo_global
Oct 15, 2024
6c77fde
fix: minor docstring update
Oct 16, 2024
e913307
feat: enable initializing NativeGates from list[str]
Oct 16, 2024
933f08c
fix: pylint error
Oct 16, 2024
08107cd
fix: Flag OR operation in FlagMeta
Oct 16, 2024
0990076
fix: Restore previous flag meta implementation
alecandido Oct 16, 2024
5d260e3
fix: add docstring in FlagMeta
Oct 16, 2024
3bc74dd
Merge branch 'qibo_global' of github.com:qiboteam/qibo into qibo_global
Oct 16, 2024
731b8a1
fix: undef variable
Oct 16, 2024
33a9984
fix: type hint update
Oct 18, 2024
65bdfee
fix: type hint update for python3.9
Oct 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/api-reference/qibo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2655,7 +2655,7 @@ As for the other backends, the Clifford backend can be set with
import qibo
qibo.set_backend("clifford", platform="numpy")

by specifying the engine used for calculation, if not provided the current :class:`qibo.backends.GlobalBackend` is used
by specifying the engine used for calculation, if not provided the current backend is used

.. testcode:: python

Expand Down
8 changes: 3 additions & 5 deletions doc/source/code-examples/advancedexamples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1297,9 +1297,8 @@ As observable we can simply take :math:`Z_0 Z_1 Z_2` :

from qibo.symbols import Z
from qibo.hamiltonians import SymbolicHamiltonian
from qibo.backends import GlobalBackend

backend = GlobalBackend()
backend = qibo.get_backend()

# Define the observable
obs = np.prod([Z(i) for i in range(nqubits)])
Expand Down Expand Up @@ -2103,10 +2102,9 @@ Multiple transpilation steps can be implemented using the :class:`qibo.transpile

# Define connectivity as nx.Graph
def star_connectivity():
Q = [i for i in range(5)]
chip = nx.Graph()
chip.add_nodes_from(Q)
graph_list = [(Q[i], Q[2]) for i in range(5) if i != 2]
chip.add_nodes_from(list(range(5)))
graph_list = [(i, 2) for i in range(5) if i != 2]
chip.add_edges_from(graph_list)
return chip

Expand Down
3 changes: 3 additions & 0 deletions src/qibo/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,15 @@
get_device,
get_precision,
get_threads,
get_transpiler,
get_transpiler_name,
list_available_backends,
matrices,
set_backend,
set_device,
set_precision,
set_threads,
set_transpiler,
)
from qibo.config import (
get_batch_size,
Expand Down
153 changes: 124 additions & 29 deletions src/qibo/backends/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import os
from importlib import import_module

import networkx as nx
import numpy as np

from qibo.backends.abstract import Backend
Expand Down Expand Up @@ -65,10 +66,11 @@ def list_available(self) -> dict:
return available_backends


class GlobalBackend(NumpyBackend):
"""The global backend will be used as default by ``circuit.execute()``."""
class _Global:
_backend = None
_transpiler = None
# TODO: resolve circular import with qibo.transpiler.pipeline.Passes

_instance = None
_dtypes = {"double": "complex128", "single": "complex64"}
_default_order = [
{"backend": "qibojit", "platform": "cupy"},
Expand All @@ -78,39 +80,88 @@ class GlobalBackend(NumpyBackend):
{"backend": "pytorch"},
]

def __new__(cls):
if cls._instance is not None:
return cls._instance
@classmethod
def backend(cls):
"""Get the current backend. If no backend is set, it will create one."""
if cls._backend is not None:
return cls._backend
cls._backend = cls._create_backend()
log.info(f"Using {cls._backend} backend on {cls._backend.device}")
return cls._backend

backend = os.environ.get("QIBO_BACKEND")
if backend: # pragma: no cover
@classmethod
def _create_backend(cls):
backend_env = os.environ.get("QIBO_BACKEND")
if backend_env: # pragma: no cover
# Create backend specified by user
platform = os.environ.get("QIBO_PLATFORM")
cls._instance = construct_backend(backend, platform=platform)
backend = construct_backend(backend_env, platform=platform)
else:
# Create backend according to default order
for kwargs in cls._default_order:
try:
cls._instance = construct_backend(**kwargs)
backend = construct_backend(**kwargs)
break
except (ImportError, MissingBackend):
pass

if cls._instance is None: # pragma: no cover
if backend is None: # pragma: no cover
raise_error(RuntimeError, "No backends available.")
return backend

@classmethod
def set_backend(cls, backend, **kwargs):
cls._backend = construct_backend(backend, **kwargs)
log.info(f"Using {cls._backend} backend on {cls._backend.device}")

log.info(f"Using {cls._instance} backend on {cls._instance.device}")
return cls._instance
@classmethod
def transpiler(cls):
"""Get the current transpiler. If no transpiler is set, it will create one."""
if cls._transpiler is not None:
return cls._transpiler

cls._transpiler = cls._default_transpiler()
return cls._transpiler

@classmethod
def set_transpiler(cls, transpiler):
cls._transpiler = transpiler
# TODO: check if transpiler is valid on the backend

@classmethod
def set_backend(cls, backend, **kwargs): # pragma: no cover
def _default_transpiler(cls):
from qibo.transpiler.optimizer import Preprocessing
from qibo.transpiler.pipeline import Passes
from qibo.transpiler.placer import Trivial
from qibo.transpiler.router import Sabre
from qibo.transpiler.unroller import NativeGates, Unroller
Comment on lines +133 to +137
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you can't import this top-level as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, moving all of them to the top causes errors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you reconstruct the reason?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the circular dependency. Would you like to address this now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to address this now?

Nope, I'm opening a separate issue for circular dependencies in general.

Because of the circular dependency.

The part I was interested in was which circular dependency (i.e. reconstructing the circle).

Copy link
Member Author

@csookim csookim Oct 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The loop is roughly structured like this: many modules in qibo.__init__.py import the backends at the top of their files. (Because of _check_backend() and get_backend()) If we import the transpiler at the top of backends.__init__.py, it causes a circular import because many transpiler modules rely on qibo.models.Circuit, which depends on the backend.

Untitled Diagram drawio (1)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed all from qibo.models import Circuit in qibo/transpiler, and it worked. Most of the Circuit imports were only used for type hints. In the other cases, I moved from qibo.models import Circuit inside the functions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this is truly a qibo-core-related problem once again, since the Circuit should be one of the most basic instances, and instead is effectively depending on the backends' machinery through its .execute() method...

Thanks for the investigation, I lifted your comment into #1490, you can resolve this conversation.


qubits = cls._backend.qubits
natives = cls._backend.natives
connectivity_edges = cls._backend.connectivity
if (
cls._instance is None
or cls._instance.name != backend
or cls._instance.platform != kwargs.get("platform")
qubits is not None
and natives is not None
and connectivity_edges is not None
):
cls._instance = construct_backend(backend, **kwargs)
log.info(f"Using {cls._instance} backend on {cls._instance.device}")
# only for q{i} naming
node_mapping = {q: i for i, q in enumerate(qubits)}
edges = [
(node_mapping[e[0]], node_mapping[e[1]]) for e in connectivity_edges
]
connectivity = nx.Graph(edges)

return Passes(
connectivity=connectivity,
passes=[
Preprocessing(connectivity),
Trivial(connectivity),
Sabre(connectivity),
Unroller(NativeGates[natives]),
],
)

return Passes(passes=[])


class QiboMatrices:
Expand Down Expand Up @@ -147,53 +198,97 @@ def create(self, dtype):


def get_backend():
return str(GlobalBackend())
"""Get the current backend."""
return _Global.backend()


def set_backend(backend, **kwargs):
csookim marked this conversation as resolved.
Show resolved Hide resolved
GlobalBackend.set_backend(backend, **kwargs)
"""Set the current backend.

Args:
backend (str): Name of the backend to use.
kwargs (dict): Additional arguments for the backend.
"""
_Global.set_backend(backend, **kwargs)


def get_transpiler():
"""Get the current transpiler."""
return _Global.transpiler()


def get_transpiler_name():
"""Get the name of the current transpiler as a string."""
return str(_Global.transpiler())


def set_transpiler(transpiler):
"""Set the current transpiler.

Args:
transpiler (Passes): The transpiler to use.
"""
_Global.set_transpiler(transpiler)


def get_precision():
return GlobalBackend().precision
"""Get the precision of the backend."""
return get_backend().precision


def set_precision(precision):
GlobalBackend().set_precision(precision)
matrices.create(GlobalBackend().dtype)
"""Set the precision of the backend.

Args:
precision (str): Precision to use.
"""
get_backend().set_precision(precision)
matrices.create(get_backend().dtype)


def get_device():
return GlobalBackend().device
"""Get the device of the backend."""
return get_backend().device


def set_device(device):
"""Set the device of the backend.

Args:
device (str): Device to use.
"""
parts = device[1:].split(":")
if device[0] != "/" or len(parts) < 2 or len(parts) > 3:
raise_error(
ValueError,
"Device name should follow the pattern: /{device type}:{device number}.",
)
backend = GlobalBackend()
backend = get_backend()
backend.set_device(device)
log.info(f"Using {backend} backend on {backend.device}")


def get_threads():
return GlobalBackend().nthreads
"""Get the number of threads used by the backend."""
return get_backend().nthreads


def set_threads(nthreads):
"""Set the number of threads used by the backend.

Args:
nthreads (int): Number of threads to use.
"""
if not isinstance(nthreads, int):
raise_error(TypeError, "Number of threads must be integer.")
if nthreads < 1:
raise_error(ValueError, "Number of threads must be positive.")
GlobalBackend().set_threads(nthreads)
get_backend().set_threads(nthreads)


def _check_backend(backend):
if backend is None:
return GlobalBackend()
return get_backend()

return backend

Expand Down
20 changes: 19 additions & 1 deletion src/qibo/backends/abstract.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import abc
from typing import Union
from typing import Optional, Union

from qibo.config import raise_error

Expand Down Expand Up @@ -29,6 +29,24 @@ def __repr__(self):
else:
return f"{self.name} ({self.platform})"

@property
@abc.abstractmethod
def qubits(self) -> Optional[list[str]]: # pragma: no cover
"""Return the qubit names of the backend. If :class:`SimulationBackend`, return None."""
raise_error(NotImplementedError)

@property
@abc.abstractmethod
def connectivity(self) -> Optional[list[tuple[str, str]]]: # pragma: no cover
"""Return the available qubit pairs of the backend. If :class:`SimulationBackend`, return None."""
raise_error(NotImplementedError)

@property
@abc.abstractmethod
def natives(self) -> Optional[list[str]]: # pragma: no cover
"""Return the native gates of the backend. If :class:`SimulationBackend`, return None."""
raise_error(NotImplementedError)

@abc.abstractmethod
def set_precision(self, precision): # pragma: no cover
"""Set complex number precision.
Expand Down
12 changes: 12 additions & 0 deletions src/qibo/backends/numpy.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,18 @@ def __init__(self):
np.complex128,
)

@property
def qubits(self):
return None

@property
def connectivity(self):
return None

@property
def natives(self):
return None

def set_precision(self, precision):
if precision != self.precision:
if precision == "single":
Expand Down
2 changes: 1 addition & 1 deletion src/qibo/gates/abstract.py
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ def matrix(self, backend=None):
Args:
backend (:class:`qibo.backends.abstract.Backend`, optional): backend
to be used in the execution. If ``None``, it uses
:class:`qibo.backends.GlobalBackend`. Defaults to ``None``.
the current backend. Defaults to ``None``.

Returns:
ndarray: Matrix representation of gate.
Expand Down
6 changes: 3 additions & 3 deletions src/qibo/gates/channels.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def to_choi(self, nqubits: Optional[int] = None, order: str = "row", backend=Non
vectorization is done block-wise. Defaut is ``"row"``.
backend (:class:`qibo.backends.abstract.Backend`, optional):
backend to be used in the execution. If ``None``,
it uses :class:`qibo.backends.GlobalBackend`.
it uses the current backend.
Defaults to ``None``.

Returns:
Expand Down Expand Up @@ -121,7 +121,7 @@ def to_liouville(self, nqubits: int = None, order: str = "row", backend=None):
it raises ``NotImplementedError``. Defaut is ``"row"``.
backend (:class:`qibo.backends.abstract.Backend`, optional):
backend to be used in the execution. If ``None``,
it uses :class:`qibo.backends.GlobalBackend`.
it uses the current backend.
Defaults to ``None``.

Returns:
Expand Down Expand Up @@ -160,7 +160,7 @@ def to_pauli_liouville(
Pauli elements in the basis. Default is "IXYZ".
backend (:class:`qibo.backends.abstract.Backend`, optional): backend
to be used in the execution. If ``None``, it uses
:class:`qibo.backends.GlobalBackend`.
the current backend.
Defaults to ``None``.

Returns:
Expand Down
2 changes: 1 addition & 1 deletion src/qibo/gates/special.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ def matrix(self, backend=None):
"""Returns matrix representation of special gate.

Args:
backend (:class:`qibo.backends.abstract.Backend`, optional): backend to be used in the execution. If ``None``, it uses :class:`qibo.backends.GlobalBackend`. Defaults to ``None``.
backend (:class:`qibo.backends.abstract.Backend`, optional): backend to be used in the execution. If ``None``, it uses the current backend. Defaults to ``None``.

Returns:
ndarray: Matrix representation of special gate.
Expand Down
Loading
Loading