Skip to content

Commit

Permalink
Fix flaky test test_bulk_upsert (#229)
Browse files Browse the repository at this point in the history
## Ticket

n/a

## Changes
 - Make the IDs deterministic and guaranteed to be unique

## Context for reviewers
- Since the IDs were random, there was a small chance (but [higher than
you might intuitively
expect](https://en.wikipedia.org/wiki/Birthday_problem)!) of generating
a collision. This would trigger integrity errors or cause the behavior
of the test to be not what you'd expect (a new "insert" would actually
be a "modify", for example)

## Testing
First, (temporarily) add pytest-repeat:
`poetry add pytest-repeat`
Then run the test several thousand times:
`make test args="tests/src/db/test_bulk_ops.py --count 2500"`
You should observe some failures because a duplicate ID is generated:
<img width="921" alt="Screenshot 2024-06-17 at 8 37 58 AM"
src="https://github.com/navapbc/template-application-flask/assets/31424131/98ff3d3d-0262-4e19-bce8-abef14c4e920">
Then, re-run after applying the patch:
<img width="930" alt="Screenshot 2024-06-17 at 8 53 47 AM"
src="https://github.com/navapbc/template-application-flask/assets/31424131/b785f485-d747-4bc4-91e1-43b752a028c0">
  • Loading branch information
KevinJBoyer authored Jun 17, 2024
1 parent 36382ae commit f2d19d5
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion app/tests/src/db/test_bulk_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,14 @@ class Number:
num: int


_next_id = 0


def get_random_number_object() -> Number:
global _next_id
_next_id += 1
return Number(
id=str(random.randint(1000000, 9999999)),
id=str(_next_id),
num=random.randint(1, 10000),
)

Expand Down

0 comments on commit f2d19d5

Please sign in to comment.