Add migrations hash/checksum for factory_dump.json (nautobot#5876)

* Fix nautobot#5858 - add hash/checksum for factory_dump.json

.gitignore

@@ -30,5 +30,5 @@ jobs/*
 htmlcov/*
 /nautobot/project-static/docs/
 /examples/example_*/**/static/example_*/docs/
-development/factory_dump.json
+development/factory_dump*.json
 node_modules

changes/5858.housekeeping

@@ -0,0 +1 @@
+Enhanced the test runner to include a hash of applied database migrations as part of the factory dump filename, reducing the likelihood of using the wrong cached data for a given branch.

nautobot/core/tests/runner.py

@@ -1,8 +1,10 @@
 import copy
+import hashlib
 
 from django.conf import settings
 from django.core.management import call_command
 from django.db import connections
+from django.db.migrations.recorder import MigrationRecorder
 from django.test.runner import _init_worker, DiscoverRunner, ParallelTestSuite
 from django.test.utils import get_unique_databases_and_mirrors, NullTimeKeeper, override_settings
 import yaml
@@ -113,6 +115,14 @@ def setup_databases(self, **kwargs):
                             command += ["--seed", settings.TEST_FACTORY_SEED]
                         if self.cache_test_fixtures:
                             command += ["--cache-test-fixtures"]
+                            # Use the list of applied migrations as a unique hash to keep fixtures from differing
+                            # branches/releases of Nautobot in separate files.
+                            hexdigest = hashlib.shake_128(
+                                ",".join(
+                                    sorted(f"{m.app}.{m.name}" for m in MigrationRecorder.Migration.objects.all())
+                                ).encode("utf-8")
+                            ).hexdigest(10)
+                            command += ["--fixture-file", f"development/factory_dump.{hexdigest}.json"]
                         with time_keeper.timed(f'  Pre-populating test database "{alias}" with factory data...'):
                             db_command = [*command, "--database", alias]
                             call_command(*db_command)

nautobot/docs/development/core/getting-started.md

@@ -668,10 +668,9 @@ invoke unittest --cache-test-fixtures --keepdb --parallel --skip-docs-build
 invoke unittest --cache-test-fixtures --keepdb --parallel --skip-docs-build --label nautobot.core.tests
 ```
 
-When switching between significantly different branches of the code base (e.g. `main` vs `develop` vs `next`), you'll need to remove the cached test factory data, and for once omit the `--keepdb` option so that the test database can be destroyed and recreated appropriately:
+When switching between significantly different branches of the code base (e.g. `main` vs `develop` vs `next`), you'll need to for once omit the `--keepdb` option so that the test database can be destroyed and recreated appropriately:
 
 ```no-highlight
-rm development/factory_dump.json
 invoke unittest --cache-test-fixtures --parallel
 ```
 

nautobot/docs/development/core/testing.md

@@ -105,10 +105,19 @@ Nautobot's custom [test runner](https://docs.djangoproject.com/en/3.2/topics/tes
 
 +++ 1.5.11
 
-To reduce the time taken between multiple test runs, a new argument has been added to the `nautobot-server test`, `invoke unittest` and `invoke integration-test` commands: `--cache-test-fixtures`. When running one of these commands with `--cache-test-fixtures` for the first time, after the factory data has been generated it will be saved to a `factory_dump.json` file in the `development` directory. On subsequent runs of unit or integration tests, the factory data will be loaded from the file instead of being generated again. This can significantly reduce the time taken to run tests. It's a good idea to let this file be regenerated after pulling new code from the repository, as the factory data may have changed.
+To reduce the time taken between multiple test runs, a new argument has been added to the `nautobot-server test`, `invoke unittest` and `invoke integration-test` commands: `--cache-test-fixtures`. When running one of these commands with `--cache-test-fixtures` for the first time, after the factory data has been generated it will be saved to a `factory_dump.json` file in the `development` directory. On subsequent runs of unit or integration tests, the factory data will be loaded from the file instead of being generated again. This can significantly reduce the time taken to run tests.
 
 Factory caching is disabled by default. When using the `invoke` commands to run tests, caching can be enabled by default for your development environment by setting the `cache_test_fixtures` key to `True` in the `invoke.yml` file.
 
++/- 2.2.7 "Hashing of migrations in the factory dump"
+    The test runner now calculates a hash of applied database migrations and uses that as a key when creating/locating the factory data file. This serves as a way to avoid inadvertently using cached test data from the wrong branch or wrong set of migrations, and reduces the frequency with which you might need to manually delete the fixture file. For example, the set of migrations present in `develop` might result in a `factory_dump.966e2e1ed4ae5f924d54.json`, while those in `next` might result in `factory_dump.72b71317c5f5c047493e.json` - both files can coexist, and when you switch between branches during development, the correct one will automatically be selected.
+
+!!! tip
+    Although changes to the set of migrations defined will automatically invalidate an existing factory dump, there are two other cases where you will currently need to manually remove the file in order to force regeneration of the factory data:
+
+    1. When the contents of an existing migration file are modified (the hashing implementation currently can't detect this change).
+    2. When the definition of a factory is changed or a new factory is added.
+
 ## Performance Tests
 
 +++ 1.5.0