Run functions sequentially with same input and environment schemas?

[jam-fran]

Hi! I've run into a few use cases where I'd like to combine 2 or more domain functions where:

1. They are run sequentially, not in parallel (like the behavior of pipe or sequence)
2. They are all passed the same input and environment (like the behavior of and or collect)

I might be missing something or misreading the docs - is this currently possible? If not, any ideas for a workaround or elegant way to handle this? Thank you!

[gustavoguichard]

Hi @jam-fran , I just had a scenario where I felt a similar need but ended up solving it with the built-in methods.
I hope it makes sense to you.

Scenario

I have a getUser (DomainFunction<{ user: User }>) and a getUserAssessments (DomainFunction<{ assessments: Assessment[] }>) that takes the user as input and can be used sequentially with a pipe operator.
I want to merge their results like this: DomainFunction<{ user: User, assessments: Assessment[] }> so I can do:

const getData = map(
  sequence(getUser, getUserAssessments),
  mergeObjects,
);

All good.

Now I have a new requirement that imposes a challenge: I want to be able to override some properties of User on the fly to make different queries for getUserAsessments but I do want the full user in my result.

Here is my solution:

First I start changing my getUserAssessments' inputSchema:

// from this
z.object({ user: userSchema })

// to this
z.object({ user: userSchema, overrides: partialUserSchema.optional() })

Now all I need to do is to create a passthrough domain function:

const passthrough: makeDomainFunction(z.any())((input) => input)

Now I can merge the passthrough with getUser and send my overrides like so:

const getUserPassthrough = map(all(getUser, passthrough), mergeObjects)
const getData = map(
  sequence(getUserPassthrough, getUserAssessments),
  mergeObjects,
);

// than in my loader:
export async function loader({ request, params }: DataFunctionArgs) {
  const result = await getData({ ...params, overrides: inputFromUrl(request) });
  // ...
}

And I can now change the user properties by using a query string: /my-route?age=20

That's it

I hope it helps you finding a nice way to forward your input to sequential domain functions ;)

[diogob]

@jam-fran is the sequencing important in this case because of the order of the side-effects?

[atreib]

Hey hey! I'd a similar use-case as the two examples above.

I have a sequence of functions (not parallel), but I need to merge the output of the "previous function" with the input for the "next function", and it should return a single merged output of all functions.

I needed something like this:

it("Should aggregate the output on the input of the next function", async () => {
  const input = { a: 1 };

  const dfReturnsB = makeDomainFunction(z.any())(() => ({
    b: 2,
  }));

  const dfReturnsC = makeDomainFunction(
    z.object({ a: z.number(), b: z.number() })
  )(({ a, b }) => ({ a, b, c: 3 }));

  const dfReturnsD = makeDomainFunction(z.any())(() => ({
    d: 4,
  }));

  const aggregatedDomainFunction = aggregate(
    dfReturnsB,
    dfReturnsC,
    dfReturnsD
  );

  const aggregationResult = await fromSuccess(aggregatedDomainFunction)(input);

  expect(aggregationResult).toEqual({
    a: 1,
    b: 2,
    c: 3,
    d: 4,
  });
});

And I have ended up with the following function:

import {
  DomainFunction,
  Result,
  SuccessResult,
  UnpackAll,
  mergeObjects,
} from "domain-functions";

function aggregate<Fns extends DomainFunction[]>(
  ...fns: Fns
): DomainFunction<UnpackAll<Fns>> {
  return async function (initialInput: unknown, environment?: unknown) {
    const results = [];
    let inputs = [initialInput];

    let prevResult: undefined | Result<unknown>;
    for await (const fn of fns as DomainFunction[]) {
      if (prevResult?.success) inputs.push(prevResult.data);
      const result = await fn(mergeObjects(inputs), environment);
      if (!result.success) return result;
      prevResult = result;
      results.push(result.data);
    }

    return {
      success: true,
      data: mergeObjects(results),
      inputErrors: [],
      environmentErrors: [],
      errors: [],
    } as SuccessResult<UnpackAll<Fns>>;
  };
}

export { aggregate };

Maybe this approach can help with your cases @jam-fran @gustavoguichard 🙂

Also I would be glad to contribute to the repo, if this function makes sense for the library.
Cheers

[gustavoguichard]

The only problem with this aggregate implementation is that it might throw if one DF does not return an object.
So the suggestion is to wrap this code in try/catch block, catch any possible error and return an ErrorResult so it doesn't break the consistency of domain-functions.

For instance, take a look at this bit of code in the implementation of merge or the implementation of branch, or map ;)

[gustavoguichard]

I'm reopening this issue as I've been stumbling upon this necessity so I want to investigate if @atreib 's solution could be a good addition to this library.
Andre, I'll circle back to you whenever I test it... would you be willing to send a PR ?

[diogob]

couldn't this be built with a sequence + map?

[atreib]

@diogob I don't think so 🤔 Because with the aggregate(), every step receives the same input + the output of the previous functions. And with the sequence() + map(), they would all receive the same initial input, right? I'm open to think on other approaches though 🙂

@gustavoguichard Yeah, 100%!

[diogob]

@atreib could you solve your use case moving this initial input to the environment?

In this way the whole composition would have access to it and you still have the output from the previous function in a sequence.

[atreib]

@diogob Yes, you're right! I'm able to do it using the environment + sequence + map 🙂 Although I will keep using this little helper in my codebase, because the code looks easier to read.

Thank you for the idea!

[gustavoguichard]

Folks, I turned this issue into a discussion.

I was talking to @diogob and explaining him about a possible scenario where the aggregate would be needed:

Scenario:

We want to pipe through some functions. We will pass an initial input (x) to the pipe and we have the following requirements:

• dfA receives x as input and returns { a } => dfA(x)<{ a }>
• dfB does not need an input and returns { b } => dfB()<{ b }>
• dfC needs output a and b as inputs to return its output => dfC({ a, b })<{ c: a + b }>
• dfD needs output b as input and initial x to return d => dfD({ b, x })<{ d: x + b }>
• They all must run in sequence => dfA -> dfB -> dfC -> dfD to result in DomainFunction<{ a, b, c, d }>

How to solve that with the current combinators?

It is possible currently!

Premises

• Every DF receives an input and an environment, but maybe the naming is not ideal. Let's think about them like so:
  • input: something that changes depending on the caller. It is repeated when called by combinators that run in parallel such as collect, all, and merge
  • environment: something that never changes in compositions, regardless if they are parallel or sequential
• We can make one DF transparently forward it's input by merging it with an identity DF.

Here is an identity implementation:

const identity  = (input?: any) => ({ success:true, data:input, inputErrors: [], errors: [], environmentErrors: []}) as any;

Solution

To solve the proposed scenario under the given premises this is a possible solution to the problem:

// First we change the structure of our dfs a lil bit so the first input `x` becomes environment:
const envSchema = z.object({ x: z.number() });

const dfA = mdf(z.any(), envSchema)((_, { x }) => ({ a: x * 2 }));
const dfB = mdf()(() => ({ b: 5 }));
const dfC = mdf(z.object({ a: z.number(), b: z.number() }))(({ a, b }) => ({
  c: a + b,
}));
const dfD = mdf(
  z.object({ b: z.number() }),
  envSchema
)(({ b }, { x }) => ({ d: x + b }));

Now, with the help of identity we can do the following:

const data = map(
  sequence(dfA, merge(dfB, identity), merge(dfC, identity), dfD),
  mergeObjects
);

What should we add to the library?

We can do that composition with our current methods, but should we add more combinators to avoid the boilerplate?

I'd export identity to be used that way... maybe even some sort of passthrough:

const identity  = (input?: any) => ({ success:true, data:input, inputErrors: [], errors: [], environmentErrors: []}) as any;
const passthrough = <T extends DomainFunction<Record<string, unknown>>>(df: T) => merge(df, identity)

const data = map(
  sequence(dfA, passthrough(dfB), passthrough(dfC), dfD),
  mergeObjects
);

What do you guys think?

[gustavoguichard]

@diogob we could also use a Map to keep the order, what do you think?

[gustavoguichard]

The deal we should pay attention to is to have access to early outputs on later dfs

[gustavoguichard]

@diogob , order of keys is guaranteed since ECMA2015 so I think it is safe to assume we can use simple objects.
However, if the keys are "number-like" the order will be altered.

So basically this:

const functions = {
  '3': () => console.log('Third function'),
  '2': () => console.log('Second function'),
  '1': () => console.log('First function'),
};

Object.values(functions).forEach((func) => func());

Will be logged as:

First function
Second function
Third function

But is seems to be pretty hard (maybe impossible) to create a mapped type off a Map so it might become a problem.
What do you think about using plain objects and do docs and runtime checks to avoid people to assign numbers as keys?

[diogob]

Sounds good to me.

I'm happy not having any runtime checks and just document the behaviour. If the user wants to use numbered keys to specify an arbitrary input order that should be allowed.

[gustavoguichard]

FWIW from this discussion we came up with collectSequence

[diogob]

@gustavoguichard rereading the original post here, it seems that the upcoming tap would fit the bill for what @jam-fran was asking

[gustavoguichard]

yeah! but it also may be solved with the - already implemented - new map with original inputs on the handler. Depending on the case.

const add = composable((a: number, b: number) => a + b)
const myFn = map(
  add,
  (result, a, b) => `${a} + ${b} is equal to ${result}`
)