Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EDU-3510: Python: Added paragraph that warns about pytest.fixture(scope="session") with test server #3113

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

edmondop
Copy link

What does this PR do?

Addresses #3111 . When running tests in a project I noticed:

  • Time skipping wasn't working even if it was enabled
  • samples-python test never reuse the same workflow environment with timeskipping

This additional paragraph should warn developers familiar with pytest.fixtures not to use the scope session for caching the workflow environment

@fairlydurable
Copy link
Contributor

fairlydurable commented Oct 10, 2024

Created SDK-2941 to review

@fairlydurable fairlydurable changed the title Python: Added paragraph that warns about pytest.fixture(scope="session") with test server EDU-3510: Python: Added paragraph that warns about pytest.fixture(scope="session") with test server Nov 18, 2024
Comment on lines +49 to +64
# This might break because it will reuse the workflow environment
pytest.fixture(scope="session")
def workflow_environment():
return WorkflowEnvironment.start_time_skipping()

def test_1(workflow_environment):
pass

def test_2(workflow_environment):
pass

# Correct: if you don't specify a scope, the default scope is function,
# so a new workflow environment will be created for each test
@pytest.fixture
def workflow_environment():
return WorkflowEnvironment.start_time_skipping()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit strange to have the primary code be the wrong code and have the same code correctly below (which you don't usually redefine the same function in the same code block). Maybe something like:

Suggested change
# This might break because it will reuse the workflow environment
pytest.fixture(scope="session")
def workflow_environment():
return WorkflowEnvironment.start_time_skipping()
def test_1(workflow_environment):
pass
def test_2(workflow_environment):
pass
# Correct: if you don't specify a scope, the default scope is function,
# so a new workflow environment will be created for each test
@pytest.fixture
def workflow_environment():
return WorkflowEnvironment.start_time_skipping()
# Notice how this doesn't specify a "scope" like "session" scope. If you don't
# specify a scope, the default scope is function, so a new workflow
# environment will be created for each test.
@pytest.fixture
def workflow_environment():
return WorkflowEnvironment.start_time_skipping()
def test_1(workflow_environment):
pass
def test_2(workflow_environment):
pass

Or if you must have the incorrect way be the in the initial block, can you move the correct way into its own separate block? It's a bit confusing to see the same method defined twice in the same block.



# This might break because it will reuse the workflow environment
pytest.fixture(scope="session")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pytest.fixture(scope="session")
@pytest.fixture(scope="session")

If you do keep this, need @

@@ -36,6 +36,34 @@ Some SDKs have support or examples for popular test frameworks, runners, or libr

One recommended framework for testing in Python for the Temporal SDK is [pytest](https://docs.pytest.org/), which can help with fixtures to stand up and tear down test environments, provide useful test discovery, and make it easy to write parameterized tests.

### A note on pytest fixtures

When using pytest fixtures to setup databases, application servers and common infrastructure it is not unusual to use the `session` scope to ensure the fixture is shared among multiple tests. However, with the test server it is important that each test is executed in isolation. Launching a test server per test introduce negligible overhead and ensures your test do not conflict with each other
Copy link
Member

@cretz cretz Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When using pytest fixtures to setup databases, application servers and common infrastructure it is not unusual to use the `session` scope to ensure the fixture is shared among multiple tests. However, with the test server it is important that each test is executed in isolation. Launching a test server per test introduce negligible overhead and ensures your test do not conflict with each other
When using pytest fixtures to setup databases, application servers and common infrastructure it is not unusual to use the `session` scope to ensure the fixture is shared among multiple tests. However, with the time-skipping test server it is important that each test is executed in isolation. Launching a test server per test introduce negligible overhead and ensures your test do not conflict with each other

Notice, this only applies to the time skipping test server, not the local test server where it is very normal to share it across tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants