Adds support for Python 3.12 (PrefectHQ#11306)

Co-authored-by: zangell44 <[email protected]>
Co-authored-by: Zach Angell <[email protected]>

.github/workflows/docker-images.yaml

@@ -26,6 +26,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
+          - "3.12"
 
     steps:
       - name: Set up QEMU

.github/workflows/python-tests.yaml

@@ -1,7 +1,5 @@
 name: Unit tests
 
-# Note: Conda support for 3.11 is pending. See https://github.com/ContinuumIO/anaconda-issues/issues/13082
-
 env:
   # enable colored output
   # https://github.com/pytest-dev/pytest/issues/7443
@@ -61,6 +59,7 @@ jobs:
           - "3.9"
           - "3.10"
           - "3.11"
+          - "3.12"
         pytest-options:
           - "--exclude-services"
           - "--only-services"
@@ -144,8 +143,7 @@ jobs:
           docker run --rm prefecthq/prefect-dev:${{ steps.get_image_tag.outputs.image_tag }} prefect version
 
       - name: Build Conda flavored test image
-        # Not yet supported for 3.11, see note at top
-        if: ${{ matrix.build-docker-images && matrix.python-version != '3.11' }}
+        if: ${{ matrix.build-docker-images }}
         uses: docker/build-push-action@v5
         with:
           context: .

.github/workflows/windows-pull-request.yaml

@@ -36,6 +36,8 @@ jobs:
         python-version:
           - "3.9"
           - "3.10"
+          - "3.11"
+          - "3.12"         
 
       fail-fast: false
 

.github/workflows/windows-tests.yaml

@@ -16,6 +16,8 @@ jobs:
         python-version:
           - "3.9"
           - "3.10"
+          - "3.11"
+          - "3.12"
 
       fail-fast: false
 
@@ -51,4 +53,4 @@ jobs:
       - name: Run tests
         run: |
           # Parallelize tests by scope to reduce expensive service fixture duplication
-          pytest tests -vv --numprocesses auto --dist loadscope --exclude-services --durations=25
+          pytest tests -vv --numprocesses auto --dist loadscope --exclude-services --durations=25

Dockerfile

@@ -20,8 +20,8 @@ WORKDIR /opt/ui
 
 RUN apt-get update && \
     apt-get install --no-install-recommends -y \
-        # Required for arm64 builds
-        chromium \
+    # Required for arm64 builds
+    chromium \
     && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 # Install a newer npm to avoid esbuild errors
@@ -46,8 +46,8 @@ WORKDIR /opt/prefect
 
 RUN apt-get update && \
     apt-get install --no-install-recommends -y \
-        gpg \
-        git=1:2.* \
+    gpg \
+    git=1:2.* \
     && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 # Copy the repository in; requires full git history for versions to generate correctly
@@ -96,13 +96,13 @@ WORKDIR /opt/prefect
 # - git: Required for retrieving workflows from git sources
 RUN apt-get update && \
     apt-get install --no-install-recommends -y \
-        tini=0.19.* \
-        build-essential \
-        git=1:2.* \
+    tini=0.19.* \
+    build-essential \
+    git=1:2.* \
     && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 # Pin the pip version
-RUN python -m pip install --no-cache-dir pip==22.3.1
+RUN python -m pip install --no-cache-dir pip==23.3.1
 
 # Install the base requirements separately so they cache
 COPY requirements-client.txt requirements.txt ./

docs/concepts/flows.md

@@ -160,7 +160,7 @@ def my_flow(name: str, date: datetime.datetime):
     pass
 
 # creates a flow run called 'marvin-on-Thursday'
-my_flow(name="marvin", date=datetime.datetime.utcnow())
+my_flow(name="marvin", date=datetime.datetime.now(datetime.timezone.utc))
 ```
 
 Additionally this setting also accepts a function that returns a string for the flow run name:
@@ -170,7 +170,7 @@ import datetime
 from prefect import flow
 
 def generate_flow_run_name():
-    date = datetime.datetime.utcnow()
+    date = datetime.datetime.now(datetime.timezone.utc)
 
     return f"{date:%A}-is-a-nice-day"
 
@@ -478,7 +478,7 @@ from datetime import datetime
 @flow
 def what_day_is_it(date: datetime = None):
     if date is None:
-        date = datetime.utcnow()
+        date = datetime.now(timezone.utc)
     print(f"It was {date.strftime('%A')} on {date.isoformat()}")
 
 what_day_is_it("2021-01-01T02:00:19.180906")
@@ -906,7 +906,7 @@ if __name__ == "__main__":
 ## Pausing or suspending a flow run
 
 Prefect provides you with the ability to halt a flow run with two functions that are similar, but slightly different.
-When a flow run is paused, code execution is stopped and the process continues to run. 
+When a flow run is paused, code execution is stopped and the process continues to run.
 When a flow run is suspended, code execution is stopped and so is the process.
 
 ### Pausing a flow run
@@ -1081,6 +1081,7 @@ async def greet_user():
 
     logger.info(f"Hello, {user_input.name}!")
 ```
+
 Running this flow will create a flow run. The flow run will advance until code execution reaches `pause_flow_run`, at which point it will move into a `Paused` state. 
 Execution will block and wait for resumption. 
 

docs/concepts/tasks.md

@@ -22,15 +22,15 @@ search:
 
 # Tasks
 
-A task is a function that represents a discrete unit of work in a Prefect workflow. Tasks are not required — you may define Prefect workflows that consist only of flows, using regular Python statements and functions. Tasks enable you to encapsulate elements of your workflow logic in observable units that can be reused across flows and subflows. 
+A task is a function that represents a discrete unit of work in a Prefect workflow. Tasks are not required — you may define Prefect workflows that consist only of flows, using regular Python statements and functions. Tasks enable you to encapsulate elements of your workflow logic in observable units that can be reused across flows and subflows.
 
 ## Tasks overview
 
 Tasks are functions: they can take inputs, perform work, and return an output. A Prefect task can do almost anything a Python function can do.  
 
 Tasks are special because they receive metadata about upstream dependencies and the state of those dependencies before they run, even if they don't receive any explicit data inputs from them. This gives you the opportunity to, for example, have a task wait on the completion of another task before executing.
 
-Tasks also take advantage of automatic Prefect [logging](/concepts/logs/) to capture details about task runs such as runtime, tags, and final state. 
+Tasks also take advantage of automatic Prefect [logging](/concepts/logs/) to capture details about task runs such as runtime, tags, and final state.
 
 You can define your tasks within the same file as your flow definition, or you can define tasks within modules and import them for use in your flow definitions. All tasks must be called from within a flow. Tasks may not be called from other tasks.
 
@@ -122,7 +122,7 @@ def my_task(name, date):
 @flow
 def my_flow():
     # creates a run with a name like "hello-marvin-on-Thursday"
-    my_task(name="marvin", date=datetime.datetime.utcnow())
+    my_task(name="marvin", date=datetime.datetime.now(datetime.timezone.utc))
 ```
 
 Additionally this setting also accepts a function that returns a string to be used for the task run name:
@@ -132,7 +132,7 @@ import datetime
 from prefect import flow, task
 
 def generate_task_name():
-    date = datetime.datetime.utcnow()
+    date = datetime.datetime.now(datetime.timezone.utc)
     return f"{date:%A}-is-a-lovely-day"
 
 @task(name="My Example Task",
@@ -220,7 +220,6 @@ or _failed_ if it returned a string.
 !!! note "Retries don't create new task runs"
     A new task run is not created when a task is retried. A new state is added to the state history of the original task run.
 
-
 ### A real-world example: making an API request
 
 Consider the real-world problem of making an API request. In this example,
@@ -338,7 +337,7 @@ def some_task_with_exponential_backoff_retries():
 
 ### Configuring retry behavior globally with settings
 
-You can also set retries and retry delays by using the following global settings. These settings will not override the `retries` or `retry_delay_seconds` that are set in the flow or task decorator. 
+You can also set retries and retry delays by using the following global settings. These settings will not override the `retries` or `retry_delay_seconds` that are set in the flow or task decorator.
 
 ```
 prefect config set PREFECT_FLOW_DEFAULT_RETRIES=2
@@ -377,12 +376,12 @@ def hello_flow(name_input):
     hello_task(name_input)
 ```
 
-Alternatively, you can provide your own function or other callable that returns a string cache key. A generic `cache_key_fn` is a function that accepts two positional arguments: 
+Alternatively, you can provide your own function or other callable that returns a string cache key. A generic `cache_key_fn` is a function that accepts two positional arguments:
 
 - The first argument corresponds to the `TaskRunContext`, which stores task run metadata in the attributes `task_run_id`, `flow_run_id`, and `task`.
 - The second argument corresponds to a dictionary of input values to the task. For example, if your task is defined with signature `fn(x, y, z)` then the dictionary will have keys `"x"`, `"y"`, and `"z"` with corresponding values that can be used to compute your cache key.
 
-Note that the `cache_key_fn` is _not_ defined as a `@task`. 
+Note that the `cache_key_fn` is _not_ defined as a `@task`.
 
 !!! note "Task cache keys"
     By default, a task cache key is limited to 2000 characters, specified by the `PREFECT_API_TASK_CACHE_KEY_MAX_LENGTH` setting.
@@ -468,9 +467,9 @@ def caching_task():
 
 ## Timeouts
 
-Task timeouts are used to prevent unintentional long-running tasks. When the duration of execution for a task exceeds the duration specified in the timeout, a timeout exception will be raised and the task will be marked as failed. In the UI, the task will be visibly designated as `TimedOut`. From the perspective of the flow, the timed-out task will be treated like any other failed task. 
+Task timeouts are used to prevent unintentional long-running tasks. When the duration of execution for a task exceeds the duration specified in the timeout, a timeout exception will be raised and the task will be marked as failed. In the UI, the task will be visibly designated as `TimedOut`. From the perspective of the flow, the timed-out task will be treated like any other failed task.
 
-Timeout durations are specified using the `timeout_seconds` keyword argument. 
+Timeout durations are specified using the `timeout_seconds` keyword argument.
 
 ```python
 from prefect import task, get_run_logger
@@ -502,6 +501,7 @@ See [state returned values](/concepts/task-runners/#using-results-from-submitted
     If you just need the result from a task, you can simply call the task from your flow. For most workflows, the default behavior of calling a task directly and receiving a result is all you'll need.
 
 ## Wait for
+
 To create a dependency between two tasks that do not exchange data, but one needs to wait for the other to finish, use the special [`wait_for`](/api-ref/prefect/tasks/#prefect.tasks.Task.submit) keyword argument:
 
 ```python
@@ -618,7 +618,7 @@ Prefect has built-in functionality for achieving this: task concurrency limits.
 
 Task concurrency limits use [task tags](#tags). You can specify an optional concurrency limit as the maximum number of concurrent task runs in a `Running` state for tasks with a given tag. The specified concurrency limit applies to any task to which the tag is applied.
 
-If a task has multiple tags, it will run only if _all_ tags have available concurrency. 
+If a task has multiple tags, it will run only if _all_ tags have available concurrency.
 
 Tags without explicit limits are considered to have unlimited concurrency.
 
@@ -627,9 +627,9 @@ Tags without explicit limits are considered to have unlimited concurrency.
 
 ### Execution behavior
 
-Task tag limits are checked whenever a task run attempts to enter a [`Running` state](/concepts/states/). 
+Task tag limits are checked whenever a task run attempts to enter a [`Running` state](/concepts/states/).
 
-If there are no concurrency slots available for any one of your task's tags, the transition to a `Running` state will be delayed and the client is instructed to try entering a `Running` state again in 30 seconds (or the value specified by the `PREFECT_TASK_RUN_TAG_CONCURRENCY_SLOT_WAIT_SECONDS` setting). 
+If there are no concurrency slots available for any one of your task's tags, the transition to a `Running` state will be delayed and the client is instructed to try entering a `Running` state again in 30 seconds (or the value specified by the `PREFECT_TASK_RUN_TAG_CONCURRENCY_SLOT_WAIT_SECONDS` setting).
 
 !!! warning "Concurrency limits in subflows"
     Using concurrency limits on task runs in subflows can cause deadlocks. As a best practice, configure your tags and concurrency limits to avoid setting limits on task runs in subflows.
@@ -639,7 +639,6 @@ If there are no concurrency slots available for any one of your task's tags, the
 !!! tip "Flow run concurrency limits are set at a work pool and/or work queue level"
     While task run concurrency limits are configured via tags (as shown below), [flow run concurrency limits](https://docs.prefect.io/latest/concepts/work-pools/#work-pool-concurrency) are configured via work pools and/or work queues.
 
-
 You can set concurrency limits on as few or as many tags as you wish. You can set limits through:
 
 - Prefect [CLI](#cli)
@@ -651,7 +650,7 @@ You can set concurrency limits on as few or as many tags as you wish. You can se
 You can create, list, and remove concurrency limits by using Prefect CLI `concurrency-limit` commands.
 
 ```bash
-$ prefect concurrency-limit [command] [arguments]
+prefect concurrency-limit [command] [arguments]
 ```
 
 | Command | Description                                                      |
@@ -664,24 +663,24 @@ $ prefect concurrency-limit [command] [arguments]
 For example, to set a concurrency limit of 10 on the 'small_instance' tag:
 
 ```bash
-$ prefect concurrency-limit create small_instance 10
+prefect concurrency-limit create small_instance 10
 ```
 
 To delete the concurrency limit on the 'small_instance' tag:
 
 ```bash
-$ prefect concurrency-limit delete small_instance
+prefect concurrency-limit delete small_instance
 ```
 
 To view details about the concurrency limit on the 'small_instance' tag:
 
 ```bash
-$ prefect concurrency-limit inspect small_instance
+prefect concurrency-limit inspect small_instance
 ```
 
 #### Python client
 
-To update your tag concurrency limits programmatically, use [`PrefectClient.orchestration.create_concurrency_limit`](../../api-ref/prefect/client/orchestration/#prefect.client.orchestration.PrefectClient.create_concurrency_limit). 
+To update your tag concurrency limits programmatically, use [`PrefectClient.orchestration.create_concurrency_limit`](../../api-ref/prefect/client/orchestration/#prefect.client.orchestration.PrefectClient.create_concurrency_limit).
 
 `create_concurrency_limit` takes two arguments:
 
@@ -711,7 +710,6 @@ async with get_client() as client:
 
 If you wish to query for the currently set limit on a tag, use [`PrefectClient.read_concurrency_limit_by_tag`](/api-ref/prefect/client/orchestration/#prefect.client.orchestration.PrefectClient.read_concurrency_limit_by_tag), passing the tag:
 
-
 To see _all_ of your limits across all of your tags, use [`PrefectClient.read_concurrency_limits`](/api-ref/prefect/client/orchestration/#prefect.client.orchestration.PrefectClient.read_concurrency_limits).
 
 ```python

docs/getting-started/installation.md

@@ -17,6 +17,8 @@ search:
 
 Prefect requires Python 3.8 or newer.
 
+Python 3.12 support is experimental, as not all dependencies to support it yet. If you encounter any errors, please [open an issue](https://github.com/PrefectHQ/prefect/issues/new?assignees=&labels=needs%3Atriage%2Cbug&projects=&template=1_general_bug_report.yaml).
+
 <p align="left">
     <a href="https://pypi.python.org/pypi/prefect/" alt="Python Versions">
         <img src="https://img.shields.io/pypi/pyversions/prefect?color=0052FF&labelColor=090422" /></a>
@@ -104,6 +106,7 @@ Server type:         ephemeral
 Server:
   Database:          sqlite
   SQLite version:    3.42.0
+
 ```
 </div>
 
@@ -123,11 +126,11 @@ If you're using Windows Subsystem for Linux (WSL), see [Linux installation notes
 
 ## Linux installation notes
 
-Linux is a popular operating system for running Prefect. You can use [Prefect Cloud](/ui/cloud/) as your API server, or [host your own Prefect server](/host/) backed by [PostgreSQL](/concepts/database/#configuring_a_postgresql_database). 
+Linux is a popular operating system for running Prefect. You can use [Prefect Cloud](/ui/cloud/) as your API server, or [host your own Prefect server](/host/) backed by [PostgreSQL](/concepts/database/#configuring_a_postgresql_database).
 
 For development, you can use [SQLite](/concepts/database/#configuring_a_sqlite_database) 2.24 or newer as your database. Note that certain Linux versions of SQLite can be problematic. Compatible versions include Ubuntu 22.04 LTS and Ubuntu 20.04 LTS.
 
-Alternatively, you can [install SQLite on Red Hat Enterprise Linux (RHEL)](#install-sqlite-on-rhel) or use the `conda` virtual environment manager and configure a compatible SQLite version. 
+Alternatively, you can [install SQLite on Red Hat Enterprise Linux (RHEL)](#install-sqlite-on-rhel) or use the `conda` virtual environment manager and configure a compatible SQLite version.
 
 ## Using a self-signed SSL certificate
 
@@ -140,7 +143,6 @@ If the certificate is not part of your system bundle, you can set the
 
 ***Note:*** Disabling certificate validation is insecure and only suggested as an option for testing!
 
-
 ## Proxies
 
 Prefect supports communicating via proxies through environment variables. Simply set `HTTPS_PROXY` and `SSL_CERT_FILE` in your environment, and the underlying network libraries will route Prefect’s requests appropriately. Read more about using Prefect Cloud with proxies [here](https://discourse.prefect.io/t/using-prefect-cloud-with-proxies/1696).
@@ -149,7 +151,7 @@ Prefect supports communicating via proxies through environment variables. Simply
 
 ### SQLite
 
-You can use [Prefect Cloud](/ui/cloud/) as your API server, or [host your own Prefect server](/host/) backed by [PostgreSQL](/concepts/database/#configuring_a_postgresql_database). 
+You can use [Prefect Cloud](/ui/cloud/) as your API server, or [host your own Prefect server](/host/) backed by [PostgreSQL](/concepts/database/#configuring_a_postgresql_database).
 
 By default, a local Prefect server instance uses SQLite as the backing database. SQLite is not packaged with the Prefect installation. Most systems will already have SQLite installed, because it is typically bundled as a part of Python. Prefect requires SQLite version 3.24.0 or later.
 
@@ -236,4 +238,5 @@ For more information about these environment variables, see the [cURL
 documentation](https://everything.curl.dev/usingcurl/proxies/env).
 
 ## Next steps
-Now that you have Prefect installed and your environment configured, you may want to check out the [Tutorial](/tutorial/) to get more familiar with Prefect.
+
+Now that you have Prefect installed and your environment configured, you may want to check out the [Tutorial](/tutorial/) to get more familiar with Prefect.

requirements-client.txt

@@ -15,7 +15,9 @@ jsonschema >= 3.2.0, < 5.0.0
 orjson >= 3.7, < 4.0
 packaging >= 21.3, < 24.3
 pathspec >= 0.8.0
-pendulum >= 2.1.2, < 3.0.0
+# https://github.com/PrefectHQ/prefect/issues/11619
+pendulum < 3.0; python_version < '3.12'
+pendulum >= 3.0.0, <4; python_version >= '3.12'
 # the version constraints for pydantic are merged with those from fastapi 0.103.2
 pydantic[email]>=1.10.0,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0
 python_dateutil >= 2.8.2, < 3.0.0

setup.cfg

@@ -68,6 +68,9 @@ filterwarnings =
     ignore:Prefect will drop support for Python 3.7:FutureWarning
     ignore:`PREFECT_API_URL` uses `/account/` but should use `/accounts/`.:UserWarning
     ignore:`PREFECT_API_URL` should have `/api` after the base URL.:UserWarning
+    # datetime.datetime.utcnow() is deprecated as of Python 3.12, waiting on 3rd party fixes in boto3 https://github.com/boto/boto3/issues/3889
+    ignore:datetime\.datetime\.utcnow\(\) is deprecated and scheduled for removal in a future version\..*:DeprecationWarning
+    ignore:datetime\.datetime\.utcfromtimestamp\(\) is deprecated and scheduled for removal in a future version\..*:DeprecationWarning
 
 [mypy]
 # TODO: We will allow definitions to be untyped for now; in the future we will want to