Discussion: Consider ESM mirrors for widely used packages

[kettanaito]

Hi 👋

I would like to bring up the topic of creating official ESM mirrors for widely used packages like cookie.

Motivation

• Eases propagation of changes to dependents that are ESM-only or dual-published.
• Has relatively low cost for self-contained packages like cookie. I'm not proposing this for every package.
• Encourages the use of ESM and brings the entire ecosystem forward. If not giants like cookie, then who?

This proposal originates from my Twitter post.

Execution

You can choose to have a standalone repo or keep the ESM mirror within the same repository. I leave that decision to you.

I would be more than happy to discuss this and help with organizing the repo to provision ESM publishing. Thank you.

[kettanaito]

A maintenance cost has been brought up on Twitter, and to that I advise to dual-publish. The same source, different build targets. This requires little-to-none additional maintenance cost when it comes to bug fixes or new features. Compared to a designated mirror, which likely requires someone to backport changes.

Note that I don't imply dual-publishing to the same namespace on npm. Only that it may be more beneficial to keep the same source. You can publish any directory with package.json in it, afaik.

[wesleytodd]

I have a lot of thoughts on this, and only 7 minutes before I have a meeting. I just wanted to drop my first comment:

I think there are a few decisions we need to make on this first before we decide how we are going to support it:

1. Do we want to migrate things to ESM? IMO we should not to this today. We will be supporting too many major version lines at once for a few years, and once we migrate it makes back porting fixes more difficult than necessary (you will never cleanly cherry pick changes to imports). IMO we should wait to do this until we are EOL on Express v4.
2. Do we want to strive for common tooling across the packages? We have already broken this consistency a bit with cookie being migrated to TS. IMO it would be pretty nice if at least all of the packages within the orgs had consistency within the org. But that is a very soft opinion of mine. My main concern is that we dont go all in different directions and make it a maintennce nightmare or hard for new contributors.
3. and I ran out of time...I will write more later.

[ljharb]

I see no benefit in shipping native ESM in any form until it becomes widely usable everywhere (namely, after node’s require ESM feature is shipping in all supported release lines, for many years)

a dual package creates tons of problems and complexity, and ESM-only is user-hostile until the time i previously referred to.

[UlisesGascon]

+1 on the Wes' comment. IMO maybe we are not in a good time for this, but I don't discard to revisit this again in the future 🤔

[kettanaito]

Thanks everyone for sharing their thoughts.

Do we want to migrate things to ESM? IMO we should not to this today.

No, we don't. This proposal is not about adopting ESM. It's about mirroring the existing code to ESM. You can keep the package source as-is, and use a bundler to output an ESM build.

IMO it would be pretty nice if at least all of the packages within the orgs had consistency within the org.

You are the right to decide here, but I strongly advise TypeScript.

I see no benefit in shipping native ESM in any form until it becomes widely usable everywhere

@ljharb, this is catch 22. How do we expect ESM to get widely usable if we don't want to adopt it before it is? IMO, packages like cookie is exactly the surface that should adopt ESM at the frontier. cookie is a widely usable package, and if it adopts ESM, suddenly, ESM is a bit more widely usable, don't you agree?

Let's make it abundantly clear that we are waiting on no one here. Developers cannot adopt ESM because they are blocked by third party packages. Third parties cannot adopt ESM because they are blocked by packages like cookie (👋 that's me). And packages like cookie cannot adopt ESM because . . . the adoption is low? That doesn't compute.

But I digress. Once again, this proposal is not about adopting ESM. It's about giving the ESM consumers a way to consume your packages. If community-maintained mirror packages can do that, I don't see why you shouldn't provide a more maintained and stable version of that. You can keep CJS if you so choose, but it doesn't mean not providing ESM options. Those aren't mutually exclusive.

[ljharb]

If it is good enough, it will get wide usage on its own. If it needs to be helped or pushed or forced, that indicates that it shouldnt get wide usage.

CJS is usable from within ESM via every tool I’m aware of. It’s simply not a practical or common constraint that native ESM is needed.

Mirror packages just increase the risk of supply chain security attack vectors, as well as maintenance burden, for a dubious benefit.

[kettanaito]

If it is good enough, it will get wide usage on its own. If it needs to be helped or pushed or forced, that indicates that it shouldnt get wide usage.

That is a rather odd position to have, but if my arguments didn't convince you, so be it.

It’s simply not a practical or common constraint that native ESM is needed.

It's not a practical and common constraint because there are packages that prevent it from becoming a practical and common constraint. I struggle to see how build-less JavaScript isn't something everybody would want. The same code that I can just run anywhere. No bundlers, no requires. I suppose that's just me then.

Mirror packages just increase the risk of supply chain security attack vectors, as well as maintenance burden, for a dubious benefit.

Have you considered what I proposed then? Minor maintenance cost, about the same supply chain risk as the package has right now. Would love to hear your opinion on this.

[ljharb]

Buildless is inherently slower (ESM has the waterfall problem any time a remote URL is used, which includes browsers) and means you can’t apply the many optimizations bundlers and build tools are best at.

Dual publishing creates a hazard if there’s anything in the package that depends on identity, or has state. This is possible to do safely but is very easy to mess up.

[kettanaito]

We can use tshy to do this safely. Afaik, cookie has no state so it's a perfect package to try this on. I really just want to know if you see this as a viable direction or I should invest time into creating mirrors, making the entire ecosystem less safe and rigid. I'd rather we found a proper solution to this.

Meanwhile, I will also explore createRequire and see how far that will get me.

[ljharb]

It can’t be automated and tshy simply can’t prevent this problem - it certainly does a good job automating the rest though.