diff --git a/hel_django_auditlog_extra/README.md b/hel_django_auditlog_extra/README.md index 7b6322ae..e8ae99b1 100644 --- a/hel_django_auditlog_extra/README.md +++ b/hel_django_auditlog_extra/README.md @@ -1,6 +1,30 @@ # Django Auditlog Extra -A module that fixes some issues and provides some reusable tools for Django application in the context of **City of Helsinki**, that uses **Django Auditlog with Django Graphene** or Django Rest Framework. +A module that fixes some issues and provides some reusable tools for Django application using `django-auditlog` in the context of **City of Helsinki**, that uses **Django Auditlog with Django Graphene** or Django Rest Framework. + + + + +- [Context](#context) + - [Django Auditlog](#django-auditlog) + - [Django Graphene](#django-graphene) + - [Django Rest Framework](#django-rest-framework) +- [FAQ](#faq) +- [Installation](#installation) +- [Features](#features) + - [Context manager](#context-manager) + - [`set_request_path`](#set_request_path) + - [Middleware](#middleware) + - [`AuditlogMiddleware`](#auditlogmiddleware) + - [Graphene Decorators](#graphene-decorators) + - [`auditlog_access`](#auditlog_access) + - [Mixins](#mixins) + - [`AuditlogAdminViewAccessLogMixin`](#auditlogadminviewaccesslogmixin) + - [Utilities](#utilities) + - [`AuditLogConfigurationHelper`](#auditlogconfigurationhelper) + - [Initialization examples for `AuditLogConfigurationHelper`:](#initialization-examples-for-auditlogconfigurationhelper) + + ## Context @@ -62,7 +86,27 @@ Django REST framework is a powerful and flexible toolkit that makes it easier to ## FAQ -There have been some incompatibility issues with `django-auditlog` and `django-graphene`. Some solutions to those, and if you got any questions, issues or just need some more details, see the [FAQ.md](./docs/FAQ.md). +There have been some incompatibility issues with `django-auditlog` and `django-graphene`. Some solutions to that and answers to some other common questions, issues and more details, see the [FAQ.md](./docs/FAQ.md). + +## Installation + +**Dependencies (TODO):** +For actor handling, django-auditlog and test usage: + +- **`django.contrib.auth`**: Django built-in +- **`django.contrib.contenttypes`**: Django built-in +- **`auditlog`**: django-auditlog + +**`settings.py`**: + +```python +INSTALLED_APPS = [ + "django.contrib.auth", + "django.contrib.contenttypes", + "auditlog", + # ... + "hel_django_auditlog_extra.apps.HelDjangoAuditLogExtraConfig", +``` ## Features @@ -70,144 +114,217 @@ There have been some incompatibility issues with `django-auditlog` and `django-g Code reference: [context.py](./context.py). -- `set_request_path` +#### `set_request_path` - Store the request path in the LogEntry's `additional_data` field. + Store the request path in the LogEntry's `additional_data` field. - This context manager uses a ContextVar to store the request path and - connects a signal receiver to automatically add it to LogEntry instances. - It uses a unique signal_duid to prevent duplicate signals when nested. + This context manager uses a ContextVar to store the request path and + connects a signal receiver to automatically add it to LogEntry instances. + It uses a unique signal_duid to prevent duplicate signals when nested. - NOTE: This is used by the [`AuditlogMiddleware`](#middleware). +NOTE: This is used by the [`AuditlogMiddleware`](#middleware). ### Middleware Code reference: [middleware.py](./middleware.py). -- `AuditlogMiddleware` +#### `AuditlogMiddleware` - Extends the `auditlog.middleware.AuditlogMiddleware` to fix an issue - with setting the actor in the audit log context. + Extends the `auditlog.middleware.AuditlogMiddleware` to fix an issue + with setting the actor in the audit log context. - This middleware extends the `auditlog.middleware.AuditlogMiddleware` to - address a potential issue where the actor (user and their IP address) - might not be available when `auditlog` attempts to access it. + This middleware extends the `auditlog.middleware.AuditlogMiddleware` to + address a potential issue where the actor (user and their IP address) + might not be available when `auditlog` attempts to access it. - It achieves this by explicitly setting the actor in the - audit log context before the request is processed. This ensures that - audit logs accurately reflect the user responsible for each action. + It achieves this by explicitly setting the actor in the + audit log context before the request is processed. This ensures that + audit logs accurately reflect the user responsible for each action. - This fix is based on the suggestion provided in: - https://github.com/jazzband/django-auditlog/issues/115#issuecomment-1682234986 + This fix is based on the suggestion provided in: + https://github.com/jazzband/django-auditlog/issues/115#issuecomment-1682234986 - Additionally, this middleware sets the request path in the audit log - context, providing more context for each logged action. + Additionally, this middleware sets the request path in the audit log + context, providing more context for each logged action. - To use, add `"hel_django_auditlog_extra.middleware.AuditlogMiddleware"` to the list of the middlewares in `settings.py`, (instead of the one that `django-auditlog` offers): +To use, add `"hel_django_auditlog_extra.middleware.AuditlogMiddleware"` to the list of the middlewares in `settings.py`, (instead of the one that `django-auditlog` offers): - ```python - MIDDLEWARE = [ - "django.middleware.security.SecurityMiddleware", - "django.contrib.sessions.middleware.SessionMiddleware", - "corsheaders.middleware.CorsMiddleware", - "django.middleware.locale.LocaleMiddleware", - "django.middleware.common.CommonMiddleware", - "django.middleware.csrf.CsrfViewMiddleware", - "django.contrib.auth.middleware.AuthenticationMiddleware", - "django.contrib.messages.middleware.MessageMiddleware", - "django.middleware.clickjacking.XFrameOptionsMiddleware", - "hel_django_auditlog_extra.middleware.AuditlogMiddleware", - ] - ``` +```python +MIDDLEWARE = [ + "django.middleware.security.SecurityMiddleware", + "django.contrib.auth.middleware.AuthenticationMiddleware", + # ...other middlewares that manipulates request context... + "hel_django_auditlog_extra.middleware.AuditlogMiddleware", +] +``` ### Graphene Decorators Code reference: [graphene_decorators.py](./graphene_decorators.py). -- `auditlog_access` +#### `auditlog_access` - Decorator to init audit logging to a Graphene DjangoObjectType's get_node method. + Decorator to init audit logging to a Graphene DjangoObjectType's get_node method. - Uses the `accessed` signal to log the access of the node. + Uses the `accessed` signal to log the access of the node. - To use this decorator for a GraphQL Node, add it to the Node-class that is implementing a `DjangoObjectType`. The decorator will then wrap the `get_node` -function, that is inherited from the `DjangoObjectType`. +To use this decorator for a GraphQL Node, add it to the Node-class that is implementing a `DjangoObjectType`. The decorator will then wrap the `get_node` -function, that is inherited from the `DjangoObjectType`. - ```python - @auditlog_access - class ChildNode(DjangoObjectType): +```python +@auditlog_access +class ChildNode(DjangoObjectType): - # fields... + # fields... - class Meta: - model = Child - # meta... + class Meta: + model = Child + # meta... - # methods... - @classmethod - @login_required - def get_node(cls, info, id): - try: - return cls._meta.model.objects.user_can_view(info.context.user).get(id=id) - except cls._meta.model.DoesNotExist: - return None - ``` + # methods... + @classmethod + @login_required + def get_node(cls, info, id): + try: + return cls._meta.model.objects.user_can_view(info.context.user).get(id=id) + except cls._meta.model.DoesNotExist: + return None +``` ### Mixins Code reference: [mixins.py](./mixins.py). -- `AuditlogAdminViewAccessLogMixin` +#### `AuditlogAdminViewAccessLogMixin` - A mixin for Django Admin views to log access events using `django-auditlog`. + A mixin for Django Admin views to log access events using `django-auditlog`. - This mixin automatically logs accesses to the `change_view`, `history_view`, - and `delete_view` in the Django Admin. It also provides an option to log - accesses to the `changelist_view` (list view). + This mixin automatically logs accesses to the `change_view`, `history_view`, + and `delete_view` in the Django Admin. It also provides an option to log + accesses to the `changelist_view` (list view). - By default, only access to individual object views (change, history, delete) - is logged. To enable logging for the list view, set the - `write_accessed_from_list_view` attribute to `True` in your `ModelAdmin`. - Please note that this will trigger a very intensive logging and a lots of - access log data will be created and stored! + By default, only access to individual object views (change, history, delete) + is logged. To enable logging for the list view, set the + `write_accessed_from_list_view` attribute to `True` in your `ModelAdmin`. + Please note that this will trigger a very intensive logging and a lots of + access log data will be created and stored! - Attributes: - write_accessed_from_list_view (bool): - A flag to enable/disable logging access from the list view. - Defaults to `False`. + Attributes: + write_accessed_from_list_view (bool): + A flag to enable/disable logging access from the list view. + Defaults to `False`. - Example: +Example: - ```python - from django.contrib import admin - from .models import MyModel +```python +from django.contrib import admin +from .models import MyModel - @admin.register(MyModel) - class MyModelAdmin(AuditlogAdminViewAccessLogMixin, admin.ModelAdmin): - write_accessed_from_list_view = True # Enable list view logging - # ... other admin configurations ... - ``` +@admin.register(MyModel) +class MyModelAdmin(AuditlogAdminViewAccessLogMixin, admin.ModelAdmin): + write_accessed_from_list_view = True # Enable list view logging + # ... other admin configurations ... +``` ### Utilities Code reference: [utils.py](./utils.py). -- `AuditLogConfigurationHelper` +#### `AuditLogConfigurationHelper` + + A helper class for managing audit log configuration in your Django project. + + This class provides methods to: + + - Retrieve all models in your project. + - Identify models that are not explicitly configured for audit logging. + - Raise an error if any models are not configured when + `AUDITLOG_INCLUDE_ALL_MODELS` is enabled. + + This helps ensure that all models are either explicitly included or excluded + from audit logging, preventing accidental omissions. + +NOTE: The `AuditLogConfigurationHelper` can be used only after all the apps are ready. + +Example usage: Use when the audit log registry is already configured... + +```python +AuditLogConfigurationHelper.raise_error_if_unconfigured_models() +``` + +##### Initialization examples for `AuditLogConfigurationHelper`: + +- One place to call this helper function would be in the ready-function of the last app's configuration, when all the models are already registered. You can even add an `apps.py` file into your main Django project folder and then add the project app to the list of the `INSTALLED_APPS` in project's `settings.py`. The important thing is that it should be the last one with models. + + **`apps.py`**: + + ```python + from django.apps import AppConfig + + class MainProjectConfig(AppConfig): + name = "mainproject" + + def ready(self): + from hel_django_auditlog_extra.utils import AuditLogConfigurationHelper + + AuditLogConfigurationHelper.raise_error_if_unconfigured_models() + ``` + + **`settings.py`**: + + ```python + INSTALLED_APPS = [ + "django.contrib.auth", + "django.contrib.contenttypes", + "django.contrib.sessions", + "django.contrib.messages", + "django.contrib.staticfiles", + "auditlog", + # local apps + "events", + "hel_django_auditlog_extra.apps.HelDjangoAuditLogExtraConfig", + "mainproject", + "django_cleanup.apps.CleanupConfig", # This must be included last + ] + ``` + +- Another way to achieve this would be using a `post_migrate` signal receiver, which is called after the migration process (`python manage.py migrate`) is done: + + **`signals.py`**: + + ```python + from django.apps import apps + from django.db.models.signals import post_migrate + from django.dispatch import receiver + + from hel_django_auditlog_extra.utils import AuditLogConfigurationHelper + + + @receiver(post_migrate) + def check_audit_log_configuration(sender, **kwargs): + """ + Signal receiver to check audit log configuration after all apps are migrated. + """ + if apps.ready: + AuditLogConfigurationHelper.raise_error_if_unconfigured_models() + ``` + + Remember that signals and receivers needs to be connected (for example in `apps.py`): - A helper class for managing audit log configuration in your Django project. + ```python + from django.apps import AppConfig + from django.core.signals import request_finished - This class provides methods to: - - Retrieve all models in your project. - - Identify models that are not explicitly configured for audit logging. - - Raise an error if any models are not configured when - `AUDITLOG_INCLUDE_ALL_MODELS` is enabled. + class MyAppConfig(AppConfig): + ... - This helps ensure that all models are either explicitly included or excluded - from audit logging, preventing accidental omissions. + def ready(self): + # Implicitly connect signal handlers decorated with @receiver. + from . import signals - Example usage: Use when the audit log registry is already configured... + # Explicitly connect a signal handler. + request_finished.connect(signals.my_callback) + ``` - ```python - AuditLogConfigurationHelper.raise_error_if_unconfigured_models() - ``` + Reference: https://docs.djangoproject.com/en/5.1/topics/signals/#listening-to-signals. diff --git a/hel_django_auditlog_extra/docs/FAQ.md b/hel_django_auditlog_extra/docs/FAQ.md index dc442267..6fbc2529 100644 --- a/hel_django_auditlog_extra/docs/FAQ.md +++ b/hel_django_auditlog_extra/docs/FAQ.md @@ -1,4 +1,24 @@ -# Django-auditlog incompatibility issues with Django-graphene +# FAQ + + + + +- [Django-auditlog incompatibility issues with Django-graphene](#django-auditlog-incompatibility-issues-with-django-graphene) + - [Graphene's Error Handling and its Impact on Authentication](#graphenes-error-handling-and-its-impact-on-authentication) + - [Graphene's Unique Authentication Challenges](#graphenes-unique-authentication-challenges) + - [A custom Django authentication middleware implemenation for GraphQL vs the initial one](#a-custom-django-authentication-middleware-implemenation-for-graphql-vs-the-initial-one) + - [Automatic logging doesn't log the Actor](#automatic-logging-doesnt-log-the-actor) +- [Users can't remove their own account because of actor field](#users-cant-remove-their-own-account-because-of-actor-field) +- [How does Django use SimpleLazyObject during the authentication?](#how-does-django-use-simplelazyobject-during-the-authentication) +- [Why are login attempts not logged with auditlog?](#why-are-login-attempts-not-logged-with-auditlog) +- [Does django-auditlog track failed actions (e.g., failed saves, deletes)?](#does-django-auditlog-track-failed-actions-eg-failed-saves-deletes) + - [Should I log failed actions with django-auditlog?](#should-i-log-failed-actions-with-django-auditlog) + - [How can I log failed actions with django-auditlog?](#how-can-i-log-failed-actions-with-django-auditlog) + - [How can I create a separate model for logging specific errors?](#how-can-i-create-a-separate-model-for-logging-specific-errors) + + + +## Django-auditlog incompatibility issues with Django-graphene The Django-auditlog does not provide any automatic support for writing access logs to the audit logs. It only provides an automated way to handle object write logs. By access logs, we mean logs that record when a user accesses or interacts with a particular view or resource, as opposed to modifying an object in the database. @@ -6,7 +26,7 @@ While it is a widely used Django app, `django-auditlog` does not support writing For example, if a user accesses a GraphQL view to fetch data, the audit log entry created by `django-auditlog` might show the actor as `system` (which is used for `AnonymousUser` or `None`), even though the user was authenticated by Graphene's middleware. This is because `set_actor` was called in Django's middleware before Graphene had a chance to authenticate the user. -## Graphene's Error Handling and its Impact on Authentication +### Graphene's Error Handling and its Impact on Authentication > Why does the Django Graphene have it's own `GRAPHENE.MIDDLEWARE` configuration? @@ -30,7 +50,7 @@ Django middleware -> Routing to a GraphQL view -> Graphene middleware -> Graphen In essence, Graphene's middleware provides a way to add a layer of processing and customization specifically tailored to your GraphQL API, separate from the broader Django application middleware. -## Graphene's Unique Authentication Challenges +### Graphene's Unique Authentication Challenges The Graphene can provide public and private query fields. The queries like `introspection query` are generally public. Because the Django middlewares are executed before the GraphQL view is, and because the Django generally handles the authentication with Django middlewares (in city of Helsinki context usually with `django-helusers` auth middlewares), **the authentication errors are raised before the request handling gets to the Graphene**. @@ -38,7 +58,7 @@ Generally, the Graphene handles the GraphQL errors in it's own scope. Instead of > Question: Should we somehow set (or mark) the GraphQL endpoint public in the Django routing? -## A custom Django authentication middleware implemenation for GraphQL vs the initial one +### A custom Django authentication middleware implemenation for GraphQL vs the initial one > Installation instructions of the django-graphql-jwt: https://django-graphql-jwt.domake.io/quickstart.html#installation. @@ -94,7 +114,7 @@ class JWTMiddleware: Since the JWTMiddleware is a Graphene middleware (not Django middleware), the `auth_error` is raised as a GraphQL error (HTTP200 with an `error` field populated in the response). -## Automatic logging doesn't log the Actor +### Automatic logging doesn't log the Actor "Automatic Logging doesn't log the Actor": This is an issue tracked in: https://github.com/jazzband/django-auditlog/issues/115. @@ -124,7 +144,7 @@ Eventhough the `auditlog.middleware.AuditlogMiddleware` is applied after the `dj In this provided fix, the user is set as a `SimpleLazyObject`, which means that the actor won't be set immediately, but will be resolved later, when the authentication is (hopefully) already done. -# Users can't remove their own account because of actor field +## Users can't remove their own account because of actor field "Users can't remove their own account because of actor field": This is an issue tracked in https://github.com/jazzband/django-auditlog/issues/245. @@ -135,7 +155,7 @@ with disable_auditlog(): user.delete() ``` -# How does Django use SimpleLazyObject during the authentication? +## How does Django use SimpleLazyObject during the authentication? Django uses `SimpleLazyObject` during authentication to optimize database access and improve performance. Here's how it works: @@ -175,7 +195,7 @@ def my_view(request): In this example, the database query to fetch the user happens only when `request.user.username` is accessed. If the user isn't authenticated, the database is never queried. -# Why are login attempts not logged with auditlog? +## Why are login attempts not logged with auditlog? The city of Helsinki's audit logs do not typically record individual login attempts. This is because most of our services rely on external authorization instead of local logins. @@ -187,11 +207,11 @@ Here's why: - **Reducing Log Volume**: Logging every API request would create an overwhelming amount of data, making it difficult to find relevant information. -# Does django-auditlog track failed actions (e.g., failed saves, deletes)? +## Does django-auditlog track failed actions (e.g., failed saves, deletes)? No, django-auditlog primarily focuses on logging successful changes to your models. It doesn't automatically record failed actions like validation errors or database exceptions. -## Should I log failed actions with django-auditlog? +### Should I log failed actions with django-auditlog? Logging every failed action can lead to very verbose logs, consume significant storage, and potentially impact performance. However, there are cases where it's beneficial: @@ -200,11 +220,13 @@ Logging every failed action can lead to very verbose logs, consume significant s - **Business Logic:** Monitor failed actions related to core processes (e.g., payments, orders). - **Compliance:** Adhere to industry regulations that require logging specific failed actions. -## How can I log failed actions with django-auditlog? +By default, the `django-auditlog` does not log the failed actions. It would be better to do that with some other logger. Solution could be on web server's access log level and also some tools like `django-axe` could be used to prevent brute-forcing and attacking. + +### How can I log failed actions with django-auditlog? While you _could_ use `auditlog.LogEntry` for this, it's generally better to create a separate model (e.g., `ErrorLogEntry`) or use a different logging mechanism altogether. This provides clearer separation, more flexibility, and better performance. -## How can I create a separate model for logging specific errors? +### How can I create a separate model for logging specific errors? Define a new model to store the relevant information about the errors you want to track: @@ -212,16 +234,15 @@ Define a new model to store the relevant information about the errors you want t from django.db import models class ErrorLogEntry(models.Model): - timestamp = models.DateTimeField(auto_now_add=True) - error_type = models.CharField(max_length=255) - message = models.TextField() - traceback = models.TextField(blank=True, null=True) + is_sent = models.BooleanField(default=False, verbose_name=_("is sent")) + message = models.JSONField(verbose_name=_("message")) + created_at = models.DateTimeField(auto_now_add=True, verbose_name=_("created at")) # Add other relevant fields like user, affected object, etc. ``` Then, in your signal handlers, middleware, or decorators, catch the specific errors and create instances of this model to log them. -Example (using signals and a separate model): +**Example** (using signals and a separate model): ```python from django.db.models.signals import pre_save @@ -234,8 +255,9 @@ def log_validation_error(sender, instance, **kwargs): instance.full_clean() except ValidationError as e: ErrorLogEntry.objects.create( - error_type="ValidationError", - message=str(e), - # ... other relevant fields + message={ + "status": "ValidationError" + # ... other relevant fields + }, ) ```