Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Treat delegated execution separately #5184

Merged
merged 2 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 10 additions & 15 deletions docs/source/plugins/developing_plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1142,10 +1142,10 @@ executed in the background while you continue to use the App.
There are a variety of options available for configuring whether a given
operation should be delegated or executed immediately.

.. _operator-delegation-configuration:
.. _operator-execution-options:

Delegation configuration
~~~~~~~~~~~~~~~~~~~~~~~~
Execution options
~~~~~~~~~~~~~~~~~

You can provide the optional properties described below in the
:ref:`operator's config <operator-config>` to specify the available execution
Expand Down Expand Up @@ -1183,12 +1183,12 @@ user to choose between the supported options if there are multiple:
.. image:: /images/plugins/operators/operator-execute-button.png
:align: center

.. _operator-execution-options:
.. _dynamic-execution-options:

Execution options
~~~~~~~~~~~~~~~~~
Dynamic execution options
~~~~~~~~~~~~~~~~~~~~~~~~~

Operators can implement
Operators may also implement
:meth:`resolve_execution_options() <fiftyone.operators.operator.Operator.resolve_execution_options>`
to dynamically configure the available execution options based on the current
execution context:
Expand Down Expand Up @@ -1238,14 +1238,9 @@ of the current view:
# Force delegation for large views and immediate execution for small views
return len(ctx.view) > 1000

.. note::

If :meth:`resolve_delegation() <fiftyone.operators.operator.Operator.resolve_delegation>`
is not implemented or returns `None`, then the choice of execution mode is
deferred to
:meth:`resolve_execution_options() <fiftyone.operators.operator.Operator.resolve_execution_options>`
to specify the available execution options as described in the previous
section.
If :meth:`resolve_delegation() <fiftyone.operators.operator.Operator.resolve_delegation>`
is not implemented or returns `None`, then the choice of execution mode is
deferred to the prior mechanisms described above.

.. _operator-reporting-progress:

Expand Down
177 changes: 105 additions & 72 deletions docs/source/plugins/using_plugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -594,8 +594,8 @@ operator can be invoked like so:
dataset = foz.load_zoo_dataset("quickstart")
compute_metadata = foo.get_operator("@voxel51/utils/compute_metadata")

# Schedule a delegated operation to (re)compute metadata
compute_metadata(dataset, overwrite=True, delegate=True)
# (Re)compute the dataset's metadata
compute_metadata(dataset, overwrite=True)

.. note::

Expand All @@ -614,13 +614,11 @@ follows:
sample_collection,
overwrite=False,
num_workers=None,
delegate=False,
):
ctx = dict(view=sample_collection.view())
params = dict(
overwrite=overwrite,
num_workers=num_workers,
delegate=delegate,
)
return foo.execute_operator(self.uri, ctx, params=params)

Expand Down Expand Up @@ -655,71 +653,63 @@ data, you can access it via the ``result`` property of the returned
result = await op(...)
print(result.result) # {...}

.. _delegating-function-calls:

Delegating function calls
-------------------------
Requesting delegation
~~~~~~~~~~~~~~~~~~~~~

The
`@voxel51/utils/delegate <https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/utils>`_
operator provides a general purpose utility for
:ref:`delegating execution <delegated-operations>` of an arbitrary function
call that can be expressed in any of the following forms:
Operators that support :ref:`delegated execution <delegated-operations>` can
support this via the `__call__()` syntax by passing the
`request_delegation=True` flag to
:func:`execute_operator() <fiftyone.operators.execute_operator>`.

- Execute an arbitrary function: `fcn(*args, **kwargs)`
- Apply a function to a dataset or view:
`fcn(dataset_or_view, *args, **kwargs)`
- Call an instance method of a dataset or view:
`dataset_or_view.fcn(*args, **kwargs)`

Here's some examples of delegating common tasks that can be expressed in the
above forms:
In fact, the
Copy link
Contributor

@ritch ritch Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a kitchen sink example of __call__, request_delegation, allow_delegated_execution, adding a delegate property, and determining if an operator was scheduled (which we know is missing). In other words, an example operator that illustrates everything that you should do when implementing allow_delegated_execution.

Copy link
Contributor Author

@brimoor brimoor Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

determining if an operator was scheduled (which we know is missing)

I fixed this here: it is available via ctx.delegated

adding a delegate property

This is now effectively deprecated. Plugin devs should always use execution_options() as pictured here, since the plugin framework can be trusted as authoritative on which execution modes the current deployment supports

I think we need a kitchen sink example of call, request_delegation, allow_delegated_execution,

That's what I tried to do here. There are multiple examples of increasing complexity here, including a last one that links over to here for additional context

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`@voxel51/utils/compute_metadata <https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/utils>`_
operator does just that:

.. code-block:: python
:linenos:

import fiftyone as fo
import fiftyone.operators as foo
import fiftyone.zoo as foz
class ComputeMetadata(foo.Operator):
return foo.OperatorConfig(
...
allow_immediate_execution=True,
allow_delegated_execution=True,
)

dataset = foz.load_zoo_dataset("quickstart")
delegate = foo.get_operator("@voxel51/utils/delegate")
def __call__(
self,
sample_collection,
overwrite=False,
num_workers=None,
delegate=False,
):
ctx = dict(view=sample_collection.view())
params = dict(
overwrite=overwrite,
num_workers=num_workers,
)
return foo.execute_operator(
self.uri,
ctx,
params=params,
request_delegation=delegate,
)

# Compute metadata
delegate("compute_metadata", dataset=dataset)
which means that it can be invoked like so:

# Compute visualization
delegate(
"fiftyone.brain.compute_visualization",
dataset=dataset,
brain_key="img_viz",
)
.. code-block:: python
:linenos:

# Export a view
delegate(
"export",
view=dataset.to_patches("ground_truth"),
export_dir="/tmp/patches",
dataset_type="fiftyone.types.ImageClassificationDirectoryTree",
label_field="ground_truth",
)
compute_metadata = foo.get_operator("@voxel51/utils/compute_metadata")

# Load the exported patches into a new dataset
delegate(
"fiftyone.Dataset.from_dir",
dataset_dir="/tmp/patches",
dataset_type="fiftyone.types.ImageClassificationDirectoryTree",
label_field="ground_truth",
name="patches",
persistent=True,
)
# Schedule a delegated operation to (re)compute metadata
compute_metadata(dataset, overwrite=True, delegate=True)

.. _direct-operator-execution:

Direct execution
----------------

You can also programmatically execute any operator by directly calling
You can programmatically execute any operator by directly calling
:func:`execute_operator() <fiftyone.operators.execute_operator>`:

.. code-block:: python
Expand All @@ -738,25 +728,11 @@ You can also programmatically execute any operator by directly calling
dataset_type="COCO",
labels_path=dict(absolute_path="/tmp/coco/labels.json"),
label_field="ground_truth",
delegate=False, # False: execute immediately, True: delegate
)
}

foo.execute_operator("@voxel51/io/export_samples", ctx)

In the above example, the `delegate=True/False` parameter controls whether
execution happens immediately or is
:ref:`delegated <operator-delegated-execution>` because the operator implements
its
:meth:`resolve_delegation() <fiftyone.operators.operator.Operator.resolve_delegation>`
as follows:

.. code-block:: python
:linenos:

def resolve_delegation(self, ctx):
return ctx.params.get("delegate", False)

.. note::

In general, to use
Expand Down Expand Up @@ -787,13 +763,11 @@ data, you can access it via the ``result`` property of the returned
result = await foo.execute_operator("@an-operator/with-results", ctx)
print(result.result) # {...}

.. _requesting-operator-delegation:

Requesting delegation
---------------------
~~~~~~~~~~~~~~~~~~~~~

If an operation supports both immediate and delegated execution as specified
either by its :ref:`configuration <operator-delegation-configuration>` or
If an operation supports both immediate and
:ref:`delegated execution <delegated-operations>` as specified by its
:ref:`execution options <operator-execution-options>`, you can request
delegated execution by passing the `request_delegation=True` flag to
:func:`execute_operator() <fiftyone.operators.execute_operator>`:
Expand Down Expand Up @@ -823,6 +797,65 @@ operator's input modal when executing it from within the App:
delegation_target="overnight",
)

.. _delegating-function-calls:

Delegating function calls
-------------------------

The
`@voxel51/utils/delegate <https://github.com/voxel51/fiftyone-plugins/tree/main/plugins/utils>`_
operator provides a general purpose utility for
:ref:`delegating execution <delegated-operations>` of an arbitrary function
call that can be expressed in any of the following forms:

- Execute an arbitrary function: `fcn(*args, **kwargs)`
- Apply a function to a dataset or view:
`fcn(dataset_or_view, *args, **kwargs)`
- Call an instance method of a dataset or view:
`dataset_or_view.fcn(*args, **kwargs)`

Here's some examples of delegating common tasks that can be expressed in the
above forms:

.. code-block:: python
:linenos:

import fiftyone as fo
import fiftyone.operators as foo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
delegate = foo.get_operator("@voxel51/utils/delegate")

# Compute metadata
delegate("compute_metadata", dataset=dataset)

# Compute visualization
delegate(
"fiftyone.brain.compute_visualization",
dataset=dataset,
brain_key="img_viz",
)

# Export a view
delegate(
"export",
view=dataset.to_patches("ground_truth"),
export_dir="/tmp/patches",
dataset_type="fiftyone.types.ImageClassificationDirectoryTree",
label_field="ground_truth",
)

# Load the exported patches into a new dataset
delegate(
"fiftyone.Dataset.from_dir",
dataset_dir="/tmp/patches",
dataset_type="fiftyone.types.ImageClassificationDirectoryTree",
label_field="ground_truth",
name="patches",
persistent=True,
)

.. _delegated-operations:

Delegated operations
Expand Down Expand Up @@ -893,7 +926,7 @@ delegated operations and execute them serially in its process.

You must also ensure that the
:ref:`allow_legacy_orchestrators <configuring-fiftyone>` config flag is set
in the environment where you run the App, e.g. by setting:
in the environment where you run the App/SDK, e.g. by setting:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider highlighting this requirement more prominently.

This environment configuration is crucial for using delegated operations. Consider moving this information to the beginning of the delegated operations section or adding it to a "Prerequisites" subsection.


.. code-block:: shell

Expand Down
13 changes: 7 additions & 6 deletions fiftyone/operators/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,12 +261,13 @@ async def execute_or_delegate_operator(
ctx.request_params["delegated"] = True
metadata = {"inputs_schema": None, "outputs_schema": None}

try:
metadata["inputs_schema"] = inputs.to_json()
except Exception as e:
logger.warning(
f"Failed to resolve inputs schema for the operation: {str(e)}"
)
if inputs is not None:
try:
metadata["inputs_schema"] = inputs.to_json()
except Exception as e:
logger.warning(
f"Failed to resolve inputs schema for the operation: {str(e)}"
)

op = DelegatedOperationService().queue_operation(
operator=operator.uri,
Expand Down
Loading