Start a mutation and give a response before it's done

[maxboone]

I've been struggling with this for a few days and can't seem to find out how to do this properly in Ariadne. Anything with asyncio and awaits block the actual response.

I want a resolver for a mutation pull_data from an external source. This pull should be executed on the server and the user should upon (the start of the) execution receive a "confirmed" message.

However, this pull can take hours and shouldn't block the request / resolver.

How do I asynchrously start that job, confirm that it's started with the user - but keep it running in the background. The job stores the result on the server which can in turn be queried later through a query.

[rafalp]

You need to use separate worker handling the tasks queue like this (eg. Celery). Trying to solve this with just async/await in mutation is too little (what about handling errors/acknowledging result/retrying on failure?) and in wrong place (you will spend HTTP connection pool and resources on extra tasks not required for HTTP responses).

[maxboone]

Ah, makes sense - so the GraphQL (ariadne) application itself shouldn't handle the actual pull.

In this case I'm trying to build the following mutation, that should just return a True if acknowledged (possibly in the future expand this with errors, but irrelevant for now):

pullFile(
        name: String!,
        url: String!
): Boolean!

After which the user can do a query:

extend type Query {
    pulls: [RunningPull]!
}

type RunningPull{
    name: String!
    date: String!
    type: String!
    source: String!
    progress: Int!
}

Which lists the pulls that are currently running.

I guess the proposed (and proper) design in this case would be to have the Ariadne application push a task / request to a separate application (e.g. Celery) that actually handles the file pulls and have pulls query that application as well?

[rafalp]

Correct. Don't do time-consuming work in HTTP workers. Schedule it in tasks queue and let clients check the progress.