How to represent a Middleware

[magne4000]

Not sure of the actual format that those should use, but for now, let's take hattip way of doing middlewares

const poweredBy = async (ctx) => {
  const response = await ctx.next();
  response.headers.set("X-Powered-By", "Hattip");
  return response;
};

cc @cyco130 @mindplay-dk @brillout

[magne4000]

Could we adapt express middleware to universal ones?

I think so.

Express middlewares can edit req or res before calling next() (they also can after, but it shouldn't be supported because even in full express environment it's undefined behavior).

Hattip, and some others, can edit res AFTER calling context.next(), and edit req or context BEFORE or AFTER calling context.next().

What does this this imply?

It implies that, given any express middleware, res will be modified in definition order.
Let's take an example:

const app = express();

app.use(middleware_a());
app.use(middleware_b());
app.use(middleware_c());

And as an example for the rest of this comment, let's assume the following behavior:

middleware_a:
	-> Define Response.something()
	-> add header A to Response
	-> next()
		middleware_b:
			-> use Response.something() // Succeed!
			-> add header B to Response if header A is present
			-> context.next()
				middleware_c:
					-> context.next()

In this case, modifications applied to res in middleware_a are visible by middleware_b and middleware_c;

But in the case of hattip-like middlewares, res is modified in reverse order.
Let's adapt the example:

import { compose } from "@hattip/compose";

compose(
  adaptExpressToHattip(middleware_a),
  adaptExpressToHattip(middleware_b),
  adaptExpressToHattip(middleware_c),
);

Here there are multiple cases:

adaptExpressToHattip always call context.next() before the middleware

If middleware_b relies on modifications provided by middleware_a, they will not be present because middleware are executed in reverse order:

middleware_a:
	-> context.next()
		middleware_b:
			-> context.next()
				middleware_c:
					-> context.next()
					<- Response
			-> Trying to use Response.something() // Fails

adaptExpressToHattip always call context.next() after the middleware

This requires to craft a fake Response that the middleware can use to properly work.
In this case, middlewares are executed in definition order, but a few things need to be accounted for.

Let's again adapt the example:

middleware_a:
	-> Create fake Response
	-> Define Response.something()
	-> context.next()
		middleware_b:
			-> Create fake Response
			-> Trying to use Response.something() // Succeed!
			-> context.next()
				middleware_c:
					-> Create fake Response
					-> context.next()
					<- Response
			<- Response + add header B if header A is present // Fails, not as intended
	<- Response + add header A

In this example, header B is not added to the Response, unlike in real express environment.

Possible solution

One solution would be to store express-like middlewares modifications in the Context temporarily.

middleware_a:
	-> Create fake Response
	-> Define Response.something()
	-> add header A to Response
	-> add Response to Context
	-> context.next()
		middleware_b:
			-> Create fake Response from Context
			-> Trying to use Response.something() // Succeed!
			-> add header B to Response if header A is present // Succeed!
			-> add Response to Context
			-> context.next()
				middleware_c:
					-> Create fake Response from Context
					-> context.next()
					<- Response
			<- Response
	<- Response

I still need to test that, but it seems promising.

Should we adapt other server middlewares to universal ones?

I really don't think that's necessary, unless it's trivial (i.e. if the middleware system is already close to the universal one, why not).

TODO

How to handle next('route')?

[nitedani]

Maybe something like this:
using web streams https://github.com/nitedani/universal-handler/blob/main/adapters/expressToHattip.ts
using node streams https://github.com/nitedani/universal-handler/blob/main/adapters/expressToHattipNode.ts

usage: https://github.com/nitedani/universal-handler/blob/main/hattip-entry.ts

Can we assume that express middlewares are only used on node? Or we want to adapt them to serverless, for example cloudflare workers? It should be possible for middlewares that don't use node builtins, but not for others.

[magne4000]

I was starting to test things with on-change package. Creating a proper Proxy for Response was my next step, glad to see you already have a solid base for it!

Can we assume that express middlewares are only used on node? Or we want to adapt them to serverless, for example cloudflare workers? It should be possible for middlewares that don't use node builtins, but not for others

I wouldn't assume anything. We can do what h3 did, i.e. use unenv:

import {
  IncomingMessage as NodeIncomingMessage,
  ServerResponse as NodeServerResponse,
} from "unenv/runtime/node/http/index"

We just need to be smart and not bundle those when not needed.

I really like the non-node adapter, we should focus on not relying on node default module as much as possible IMO, but if it's not possible, either use node native modules or unenv ones.

Also using expressToAnything adapters should comes with a warning, specifying that for performance sensitive apps, it should be avoided and a proper universal solution should be used instead (I imagine that we could port well know express middlewares ourselves).
And packages requiring unenv should comes with an added warning that says it could bloat the bundle.

[magne4000]

Before adapting the Proxy even further, I think we should focus on checking if we can adapt express middlewares to other servers, still using hattip interfaces as intermediary, that way we should be able to have a pretty good idea of most of the limitations we'll face.

EDIT: related discussion for MVP: #6

[magne4000]

I stumbled upon a recurring issue while adapting Bati's code to universal middlewares and handlers.

Some servers (mostly fastify by default and express with some middlewares) are consuming and parsing the request's body, making it hard to then create a proper Request object.

I adapted @hattip/adapter-node to fix part of this issue into a new (and first) @universal-middleware/express. Here is a simplified typing of the adapter:

type RequestAdapter = (req: IncomingMessage) => Request;

To properly fix the aforementioned issue, we'll probably have to change this signature to:

type RequestAdapter = (req: IncomingMessage, body?: BodyInit | null) => Request;

This way, if the body is already consumed by the server, we have a way to give it back to the Request.

[mindplay-dk]

Some servers (mostly fastify by default and express with some middlewares) are consuming and parsing the request's body

The eternal problem with streams in middleware - we have this with every language and platform.

No idea if this is useful, but in some of my own experiments, I came up with the idea of modeling the body of an HTTP message as a function rather than an actual buffer. It sounds odd, but hear me out. 😄

So let's say we have the most basic conception of handlers and middleware:

type Handler = (request: Request) => Response | Promise<Response>;

type Middleware = (handler: Handler) => Handler;

The idea I kicked around is to model e.g. the response body not as a stream or a buffer, but as a function that writes the stream or buffer - something like:

type Writer = (write: Write) => Promise<void>;

type Write = {
  (data: string, encoding?: BufferEncoding): void;
  (data: Buffer): void;
  (data: ReadableStream): void;
};

So in the response model, you'd have a write: Writer property instead of the traditional body property.

Now, in middleware that wants to modify the response body, instead of replacing the body or altering it's contents, you replace the write function - deferring the actual creation of the body contents until the framework is actually ready to write, ensuring the operation won't be repeated, which may not be possible.

This enables a kind of middleware pattern for the writer itself - your replacement Writer function can pass a different Write function to the writer it is replacing - enabling it to incrementally process the incoming data, preserving read-once/write-once semantics and avoiding the "everything is in memory" problem. (for example, you can have a parent middleware incrementally writing strings to a long JSON response, and the next middleware could be compressing those strings into binary chunks, on the fly.)

Similarly, you could probably model the request body as a read function instead - I'm not sure, I never got that far in my experiments, and it seemed impractical for most uses. The only time it would matter is for uploads (being read-once) and the whole "function being passed a function" is arguably a bit difficult to grasp compared with just having a buffer or at least a stream, so there is a cognitive complexity tradeoff here. (also, you wouldn't be writing to the request body - so you don't have the same mutability issue as with the response body...)

Note that there is a technical complexity tradeoff as well, as this approach can defer stream creation errors - for example, if something fails to read a file or render a template, the error won't happen while processing the middleware stack, but instead may happen while the framework is trying to emit the response.

There are tradeoffs for sure. But it's the only answer I've seen to the read-once and "everything in memory" problems - most frameworks seem to to suffer from one or both of these. It wouldn't be great if a universal middleware creates a "lowest common denominator" problem. Unless maybe the idea is that universal middleware isn't expected to work for all types of middleware - but this limitation would create a cognitive complexity problem in itself, "can I implement this as universal middleware or not", since these limitations are never obvious from looking at just the API or types, superficially.

[magne4000]

Now that I finished the MVP in Bati, I'll summarize my thoughts here.

What changes where made in Bati's codebase

Servers for which universal handlers (and middlewares) where applied:

• Express
• Fastify
• Hono
• Hattip
• h3

For all those servers, I managed to use Handler as defined in #1:

• All returning Response with ReadableStream body
• Can manipulate Context, and pass it on to further Handlers and Middlewares
• Adapters for each servers are just a few lines each. I had to create @universal-middleware/express for Fastify and Express (which is for now a simple patch on top of @hattip/adpater-node. There is room for optimization)

The only middlewares I needed to implement are ones that modify the Context and not the Response. More on that on next comments.

[magne4000]

What should Middlewares actually look like?

Middlewares can edit the Context. This can be done at any moment (i.e. before or after the Response).
They can also edit the Response. How should we represent those 2 concepts under the same roof?

Currently my proposals are:

Pattern 1

This is mostly an adaption of how @hattip/compose works.

// Should be extendable by user
interface Context extends Record<string | number | symbol, unknown>> {
  next: () => Promise<Response>;
}

interface Middleware {
  (request: Request, context: Context): Response | void | Promise<Response> | Promise<void>;
}

async function myMiddleware1(request: Request, context: Context) {
  // do things
  const response = await context.next();

  // do things
  // If `context.next()` is called, a Response MUST be returned
  return response;
}

async function myMiddleware2(request: Request, context: Context) {
  context.user = { ... };
  // Here we should be able to return nothing, as we just modified the context in no particular order
  return;
}

Pattern 2

I slightly prefer this one, as this keep the Context on its own, making it actually universal.

// Should be extendable by user
interface Context extends Record<string | number | symbol, unknown> {
}

interface Middleware {
  (request: Request, context: Context, next: () => Promise<Response>): Response | void | Promise<Response> | Promise<void>;
}

async function myMiddleware1(request: Request, context: Context, next: () => Promise<Response>) {
  // do things
  const response = await next();

  // do things
  // If `next()` is called, a Response MUST be returned
  return response;
}

async function myMiddleware2(request: Request, context: Context, next: () => Promise<Response>) {
  context.user = { ... };
  // Here we should be able to return nothing, as we just modified the context in no particular order
  return;
}

Pattern 3

Taken from @mindplay-dk example. This one has a different approach than the previous ones.

// Should be extendable by user
interface Context extends Record<string | number | symbol, unknown> {
}

interface Middleware {
  (handler: Handler): Handler;
}

async function myMiddleware1(handler: Handler) {
  return (request: Request, context: Context) => {
    // do things
    const response = await handler(request, context);

    // do things
	return response;
  }
}

async function myMiddleware2(handler: Handler) {
  return (request: Request, context: Context) => {
    context.user = { ... };
	return handler;
  }
}

This pattern is simple enough, but it needs to be tested further, because I have the feeling that for some servers, it would require a custom middleware composition layer (akin to @hattip/compose). I could just be biased but the fact that I only tested Pattern 1 though.

Pattern 4

Extracted from a discussion below. The idea would be to either only care about Middlewares that do not return anything (only updating the Context), or define 2 distinct kind a Middlewares.

// Should be extendable by user
interface Context extends Record<string | number | symbol, unknown> {
}

// Those kind of middleware do not care about the Response
interface BeforeMiddleware {
  (request: Request, context: Context): void | Promise<void>;
}

// Those kind of middleware interact with the Response
interface WrappingMiddleware {
  // Any kind of pattern (like the ones in 1., 2. or 3.) could be valid here
  // They should always return a Response
  (request: Request, context: Context, next: () => Promise<Response>): Response | Promise<Response>;
}

// BeforeMiddleware
async function myMiddleware1(request: Request, context: Context) {
  context.user = { ... };
}

// WrappingMiddleware
async function myMiddleware2(request: Request, context: Context, next: () => Promise<Response>) {
  const response = await next();

  // do things with response..., then
  return response;
}

If we limit our scope to BeforeMiddleware only, we could simplify things a lot. But it could be hard to make this limitation apparent, as it does not exist currently on any Server that I'm aware of. This would change the focus of this project a little bit, (more focus on Handlers, with minimal support for Middlewares), but this still seem really useful for app/framework authors, like Auth.js for instance, as it still provide a unique interface to both update the Context and provide a Response.
On the other hand, if we go choose this direction and keep this dual definition of Middlewares, I find it a bit simpler to explain, but it now makes designing a Middleware a bit more difficult than currently, as it forces users to make this distinction that is new to them.

Pattern 5

Inspired by webroute's middlewares.
Mostly like pattern 2, but instead of relying on a next() function, rely on returning a Response handler.

// Should be extendable by user
interface Context extends Record<string | number | symbol, unknown> {
}

type Awaitable<T> = T | Promise<T>;

interface Middleware {
  (request: Request, context: Context): Awaitable<Response> | Awaitable<void> | ((response: Response) => Awaitable<Response>);
}

async function myMiddleware(request: Request, context: Context) {
  // If we return a response handler, it will be called AFTER the handler has executed an returned a response.
  return (response: Response) => {
    // manipule the response before returning it to user-land
    response.headers.set('Cache-Control', '...');
    
    return response;
  };
}

[magne4000]

I kind of like it, but I'm also generally suspicious of anything that has two or more mutually exclusive modes.

@mindplay-dk This made me think of another potential approach. We could split the middlewares representations into distinct ones:

// Returns either a Response or nothing. This could be used as guards (like auth guard for instance)
export interface UniversalGuardMiddleware<Context extends Universal.Context> {
  (
    request: Request,
    context: Context,
  ):
    | Awaitable<Response>
    | Awaitable<void>;
}

// Returns either a new Context or nothing. Only type-safe way to edit the Context
export interface UniversalContextMiddleware<Context extends Universal.Context, Result extends Universal.Context> {
  (
    request: Request,
    context: Context,
  ):
    | Awaitable<Result>
    | Awaitable<void>;
}

// Returns either a Response handler or nothing. Used to edit the Response
export interface UniversalResponseMiddleware<Context extends Universal.Context> {
  (
    request: Request,
    context: Context,
  ):
    | ((response: Response) => Awaitable<Response | void>)
    | Awaitable<void>;
}

export interface UniversalHandler<Context extends Universal.Context> {
  (request: Request, context: Context): Awaitable<Response>;
}

// This would come with helpers: `createHandler`, `createGuardMiddleware`, `createContextMiddleware`, `createResponseMiddleware`

EDIT: we could even expand the idea, and propose helpers like createNextMiddleware (like Pattern 2) that could also be supported by universal middleware 🤔

[mindplay-dk]

I actually went down that exact road many years ago when I participated in work on the PSR standard for HTTP models.

I'm sure this is something you've already explored, but the idea that keeps coming back to my mind is... why even split the request from the context in the first place? It kind of seems like that's the root cause of these "problems" or "difficult questions". (or "complexity")

I mean, the request and the context models are inherently 1:1, right? They are never created or used independently - it doesn't really make sense to have one without the other, which, at least to my mind, strongly indicates they should probably be one model?

Likely solution, make the context a property of the request, e.g. using a generic type, something like Request<TContext>.

Then you wouldn't need to choose between different modes - rather than each function needing to decide between returning a response or a (partial) context, instead make it mandatory for every function to decide exactly the request and context it wants to return. (You could still allow partial context, I suppose, but it doesn't seem all that necessary - it's always nice to be explicit, and it's easy enough to spread and merge objects in JS.)

This is sort of the route we went with the PSR standard, where attributes are largely synonymous with what we call context here - it's just data you can associate with the request, if you want to make it available to other middleware. (though attributes are untyped in PHP, since it doesn't have generic types.)

This way, there are fewer choices to make - you only have one mode/flavor/variant of handlers and middleware to consider, and they all consistently adhere to the exact same contract. This should lead to fewer if-statements and complexity overall.

The request and context are never passed or received without the other - it might "feel good" or "look nice" to separate them, but it really seems like context is conceptually "extra information about the request", and so why would it be separate object from the request? 🤔

[magne4000]

I'm sure this is something you've already explored, but the idea that keeps coming back to my mind is... why even split the request from the context in the first place? It kind of seems like that's the root cause of these "problems" or "difficult questions". (or "complexity")

I was putting this solution aside because I thought that representing the Context as its own entity would make things easier/less "invent a new standard"-ish, but realistically both solutions are standardizing new things in one way or another.
I'll play a bit with something like UniversalRequest<TContext> to see what it looks like.
I even think that something like UniversalRequest<unknown> have the same meaning as standard Request, thus making authoring middlewares and handlers that do not use Context simpler.

Then you wouldn't need to choose between different modes - rather than each function needing to decide between returning a response or a (partial) context, instead make it mandatory for every function to decide exactly the request and context it wants to return.

So if you edit the Context you return a new UniversalRequest<NewContext> for instance?

(You could still allow partial context, I suppose, but it doesn't seem all that necessary - it's always nice to be explicit, and it's easy enough to spread and merge objects in JS.)

Not fond of partial context either, I prefer to keep it simple.

The request and context are never passed or received without the other - it might "feel good" or "look nice" to separate them, but it really seems like context is conceptually "extra information about the request", and so why would it be separate object from the request? 🤔

I was thinking about this all morning, and I'm less and less confident that keeping Request and Context separated is the right solution. Whatever we choose, it will be some kind of new standard (sigh), the major goal is to make it universal, and as easy to understand as possible.

[mindplay-dk]

So if you edit the Context you return a new UniversalRequest<NewContext> for instance?

Well, ideally it would be inferred from a return type somehow.

This is another thing that occupies my mind whenever I'm on this topic, is that, ideally, middleware (and handlers) should be able to have explicit context requirements, as well as middleware (but not handlers) having an implicit context return type... such that middleware and handlers can specify their context requirements - and whatever the resulting return context type, this gets somehow added to the type as well, so that they can be chained by a middleware running.

But from everything I've been able to figure on this subject, that's probably only possible with a factory function (wrapping or "constructing" the middleware function) or via some sort chained use function like you have in your framework.

But maybe you've touched on this already, and maybe that's not possible within the scope of a universal middleware? But if it's not possible to have the required input/output context be part of the type, then how would it work when it gets applied to a use function in a framework? The required and returned context needs to be present in the middleware type, somehow, I think? Otherwise there's no way you can write something like a use chain that type-checks universal middleware... is there? idk 😅

[magne4000]

I tested moving the Context inside the Request directly, and this gave me some interesting feedback.

As you mentioned, we need a way to represent the Context given to a Middleware, and also the Context that it could have modified. Moving the Context inside the Request cannot help with that, and without wrappers functions or chained middlewares, with current pattern it's impossible to represent.
Thus, the only sensible way to represent a modified Context (without extending the scope of this repo) is by doing what webroute did, i.e. return the new Context.
With this change, I can represent the Middleware as such:

export interface UniversalMiddleware<
  InContext extends Universal.Context = Universal.Context,
  OutContext extends Universal.Context = Universal.Context,
> {
  (
    request: Request,
    context: InContext,
  ):
    | Awaitable<Response>
    | Awaitable<void>
    | Awaitable<OutContext>
    | ((response: Response) => Awaitable<Response>);
}

We could split that into several sub types (and sub helpers), but the idea remains the same.

I also noticed that moving the Context onto the Request made it a little bit more confusing converting between a standard Request and a potential UniversalRequest<Context>.

One of the difficulty about the Context, is being able to ensure a single reference of it exist at a time. Attaching the Context to the Request directly, makes it more complex to (re)create a Request (for instance when converting between Express Request and standard Request). If we separate them, we do not need to worry about not dropping the context while doing conversions.

Lastly, I'd like to be able to use own Server utils to retrieve the context when possible.
For instance, if you use Hono, make use of get to retrieve the Context (with correct typing). Keeping with this Hono example, for this to work, we need to store the Context with the set method. I found out that having a Context separated from the Request made it simpler to ensure only one instance of the Context actually existed, as only the Server representation itself keeps a reference to the Context.

All of this tends to make me think that having an internal representation of Request + Context (either by having Context inside Request or Context + Request beside each other under a new { request, context } object) adds complexity without adding actual benefits. I hope this makes sense.

[magne4000]

Caching Request and Response transformations

When adapting for different servers, we currently need to, for instance, convert an IncomingMessage to a Request (and back). Same goes for ServerResponse and Response.
This have a cost and some limitations, and we probably either need to be able to maintain both original and derived objects in real time (1) or have a way to cache the transformation somehow (2).

The biggest limitation regarding IncomingMessage right now is that we can only consume the body once. So if we do not cache the transformed Request, we're unable to ensure that we can compute it properly a second time.

In Bati's MVP, I chose solution 2 when implementing @universal-middleware/express, mostly because it required very few line of code to adapt from @hattip/adapter-node.
In this case, IncomingMessage is converted to Request, and this new Request is store on a property on the IncomingMessage. If this property then exists, we can return the previously computed Request.

This works well until one adds a non-universal middleware in the mix, that modifies the IncomingMessage. I would gladly just ignore this and recommend users to just only use universal middlewares, but realistically, it will not be the case.
So how do we account for that?

Solution 1 could perhaps fix this, with the help of Proxy objects. This would still require the user to wrap all their non-universal middlewares in a function to replace IncomingMessage by its Proxy.

We could also adapt solution 2 to be smarter. When we create the Request cached property, we can also store a hash of all the values used to create the Request. Then, when we're required to convert IncomingMessage to Request again, recompute the hash. If they are the same, reuse cache, otherwise, edit the cached Request.

There are probably other limitations that I did not stumble upon, but problem seems broad enough that fixing the issue for IncomingMessage -> Request will probably fix the issue for most other cases that we can have.

[mindplay-dk]

That's not the action of modifying the Context that makes it a Middleware, more that it's a function that does not return a Response.

That sounds weird. How is something a handler, if it is not essentially a request => response function?

What would it be doing instead?

Are you talking about these awkward middleware standards like Express, e.g. (request, response, next) => void, where middleware relies on a "fingers crossed" call to next, and the request and response are implicitly passed for you? These are extremely fragile in terms of types, since types can't guarantee you're actually calling next. (Besides being extremely confusing...)

[magne4000]

That sounds weird. How is something a handler, if it is not essentially a request => response function?

We're saying the same thing (I think). Handlers are request => response, everything else is then a middleware (?).

[mindplay-dk]

Maybe? I was mostly thrown off by your remark that middleware is "a function that does not return a Response" - the handler inside the middleware needs to return a response, either implicitly or explicitly, but a response needs to be procured, otherwise it's not a part of the resulting, composed handler, right? 🙂

[magne4000]

I was mostly thrown off by your remark that middleware is "a function that does not return a Response"

Ah I see the confusion. It should have been taken the other way around. If a function does not return a Response, it must be a Middleware (and only if we agree that's even a valid pattern to return nothing).

I think that the discussion now loops back to What should Middlewares actually look like?, with a few more possibilities (that I'll update directly in the original post original post updated with pattern 4).

[magne4000]

We could probably make use of Request.clone to be able to have multiple body consumers.

[magne4000]

Caching computed transformations

Not only Request and Response transformations needs to be cached, but also transformations on their properties.

For instance, several middlewares could want to parse request.headers.get("cookies"), or parse new URL(request.url).

Currently, the ecosystem does this caching in 2 ways.
The establish old players like Express and Fastify have their own Request and Response representation, and they (themselves or through middlewares) store computed informations directly on those representations.
The other more recent players, like Hono and h3, have a their set of tools to handle this and hide their inner working from the user.

The custom set of tools seems like a good thing to have anyway for universal Handlers and Middlewares, so the second solution seems like a good fit IMO.

This would require us to adapt what Hono or h3 does, and make it universal, i.e. work with generic Request or Response.

I have 2 ideas for this:
We can either store cached computed properties directly on Request and Response, through the use of symbols, or store them on the Context, in the same manner.
I slightly prefer the first solution, because it allows to omit the Context in all the tool, i.e.:

// On Request
const cookies = getCookies(request);

// On Context
const cookies = getCookies(request, context);

My only concern is that there could be potential limitations when writing directly to Request and Response, but it's easy to test.

[magne4000]

Should we have our own @universal-middleware/compose or only rely on what the server recommends

@universal-middleware/compose (akin to @hattip/compose) would be a way to combine multiple middlewares, and inject then onto a server in a single call.
This could be what we need at some point, if we want full control regarding the order of execution for instance. See #4

I haven't stumble yet on a use case that would require our own composition layer, so for now this seem superfluous.
But we can use this discussion to list some use cases that are really complex or impossible to implement on a specific server that would be solved by an in-house composition layer (@brillout perhaps?).

[brillout]

I guess it's only needed if we want to make the server no-code. (But ejectable, aka the Stem vision.)

Actually, I'm realizing that we only need decentral ordering for no-code server. I've edited #4.

[brillout]

Actually, I'm thinking maybe we can even do no-code server without any in-house composition. It indeed seems simpler to just use whatever the server recommends as you suggest. It also leads to a more idiomatic ejected code.