Persistence / Retries for background tasks

[nileshtrivedi]

This is a speculative question. I came to know about DEP 0014 (Django Enhancement Proposal) - which plans to standardize and build support for background tasks within Django itself (using asyncio).

Starlette already supports Background Tasks. Would it be too difficult to add persistence for these background jobs, say with PostgreSQL's SKIP LOCKED queries or ValKey ? 🤔

I have a FastAPI + PostgreSQL app in production that needs Celery and Redis for background jobs. If Starlette (and therefore FastAPI) supported persisted background jobs with Postgres as backend, it would simplify my infra a lot.

[adriangb]

You can easily write your own background task system, it is loosely coupled. See for example https://github.com/adriangb/asgi-background.

[nileshtrivedi]

Thanks for the pointer. It's helpful. 👍

Saw this on your asgi-background repo:

Unlike Starlette, we do not execute background tasks within the ASGI request/response cycle. Instead we schedule them in a TaskGroup that is bound to the application's lifespan

This seems quite useful as a built-in feature for Starlette. Has it been considered/evaluated for merging?

[adriangb]

There's no plan to incorporate it into Starlette.