Error: Worker terminated due to reaching memory limit: JS heap out of memory #875

WINOFFRG · 2023-04-03T04:55:38Z

Hi!

While building I am getting this error

DTS Build start
node:events:491
      throw er; // Unhandled 'error' event
      ^

Error [ERR_WORKER_OUT_OF_MEMORY]: Worker terminated due to reaching memory limit: JS heap out of memory
    at new NodeError (node:internal/errors:399:5)
    at [kOnExit] (node:internal/worker:277:26)
    at Worker.<computed>.onexit (node:internal/worker:199:20)
Emitted 'error' event on Worker instance at:
    at [kOnExit] (node:internal/worker:277:12)
    at Worker.<computed>.onexit (node:internal/worker:199:20) {
  code: 'ERR_WORKER_OUT_OF_MEMORY'
}

Node.js v18.14.2

I feel there is some issue with my tsconfig.json Here is the link to the current tsconfig which causes this issue
https://github.com/WINOFFRG/limeplay/blob/c7b4342dc74cdbded43c0c39101eb8dd10bd98ac/packages/limeplay-core/tsconfig.json

Please check the root tsconfig.json as well! If I don't extend the root config to package config ./tsconfig.json to packages/limeplay-core/tsconfig.json with the below config

{
	"compilerOptions": {
		"target": "ESNext",
		"useDefineForClassFields": true,
		"lib": ["DOM", "DOM.Iterable", "ESNext"],
		"allowJs": true,
		"checkJs": false,
		"skipLibCheck": true,
		"esModuleInterop": false,
		"allowSyntheticDefaultImports": true,
		"strict": false,
		"forceConsistentCasingInFileNames": true,
		"module": "ESNext",
		"moduleResolution": "Node",
		"resolveJsonModule": true,
		"isolatedModules": false,
		"noEmit": true,
		"noEmitOnError": true,
		"jsx": "react-jsx",
		"declaration": true,
		"emitDecoratorMetadata": true,
		"experimentalDecorators": true,
		"noImplicitAny": true,
		"preserveConstEnums": true,
		"typeRoots": ["./src/types"]
	},
	"include": ["src", "../../@types"],
	"exclude": ["node_modules"]
}

The build succeeds but when extended with the config shared above in the URL, It fails. Here is the link to error from Actions

https://github.com/WINOFFRG/limeplay/actions/runs/4589372907/jobs/8104219650#step:4:224

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

The text was updated successfully, but these errors were encountered:

juniorwmr · 2023-04-25T14:34:54Z

Same here.

anhntcv · 2023-05-17T02:13:46Z

I have the same problem

Achaak · 2023-05-17T13:43:29Z

{
	"compilerOptions": {
		"target": "ESNext",
		"useDefineForClassFields": true,
		"lib": ["DOM", "DOM.Iterable", "ESNext"],
		"allowJs": true,
		"checkJs": false,
		"skipLibCheck": true,
		"esModuleInterop": false,
		"allowSyntheticDefaultImports": true,
		"strict": false,
		"forceConsistentCasingInFileNames": true,
		"module": "ESNext",
		"moduleResolution": "Node",
		"resolveJsonModule": true,
		"isolatedModules": false,
		"noEmit": true,
		"noEmitOnError": true,
		"jsx": "react-jsx",
		"declaration": true,
		"emitDecoratorMetadata": true,
		"experimentalDecorators": true,
		"noImplicitAny": true,
		"preserveConstEnums": true,
		"typeRoots": ["./src/types"]
	},
	"include": ["src", "../../@types"],
	"exclude": ["node_modules"]
}

I have the same probleme but when I tested your tsconfig, it works.
And I tested to merge with mine and the problem was "typeRoots": [] which was missing
When I add this empty array in my configuration, it works.

anhntcv · 2023-05-18T08:29:42Z

thank you @Achaak , I tried with "typeRoots": [] and it fixed the error. RAM reaches 4GB threshold when build.

see egoist/tsup#875

kysley · 2023-08-01T16:33:49Z

Wow! Thanks @Achaak. This also fixed the problem in our pipeline 🤯

coopbri · 2023-08-31T01:38:43Z

Unfortunately, I am still encountering this issue despite the workarounds above. I have a React component library with ~40 components and a single entry point (though I tested multiple entry points). The builds (ESM and CJS in my case) are fast, but the *.d.ts generation eventually OOMs after ~10 minutes. I also made sure I don't have any cyclic dependencies.

# PR Overview ## Why Are We Making This Change? Both builds are failing ## How Is The Codebase Changing? – bump version to deploy – removed generation .d.ts files to handle out-of-memory issue on eas build – add preparation script before deploy ## Notes - [Issue OOM](egoist/tsup#875) https://expo.dev/accounts/hero24 <img width="1286" alt="Screenshot 2023-10-27 at 15 12 23" src="https://github.com/Helpdor/frontend/assets/4072250/0de326af-00e9-455f-9a76-bfe4b5cbe0c5">

9vfQbg7z4ajrGQxR · 2023-10-30T13:40:32Z

same like @coopbri

antonio-ivanovski · 2023-11-06T15:02:45Z

Have the same issue on TS only library (not React involved), and not even that large. The DTS step in order to be successful has pulled 14GB memory from my machine, this is unacceptable for CI runners.

What I have tried but it didn't work:

Only generate ESM or CJS
Emit declarations only
Setting typeRoots to empty array (same amount of time + build error because it didn't find the types)

Only thing that worked was downgrading to 6.6 as suggested.

@egoist @sxzz Do you have any idea what could be the issue here? Providing minimal reproducing example may be harder since it mostly happens in medium to large codebases.

antonio-ivanovski · 2023-11-06T16:59:33Z

Downgrading to 6.6 made me remove the top-level async logic from my tsup.config.js. This made my build work, but then I bumped the tsup version to 7.2 and by having no top-level async in the config, the memory requirement went down from 14GB to 4GB, bit more manageable by CI.

Looking more into the issues, found #1018 that is somewhat expected to solve this, but not sure since running the build, it still take around 4GB.

Probably part of my issue is (please confirm if true) that I am being lazy to define actual entry points so I wrote a method that fetches all the .ts files in the source and defines those as entry points. It also uses these entries to generate the package.json#exports. Reducing the set of entries to the actual entries reduces the memory footprint to ~2,5GB.

For reference, the build on tsup 6.6 takes <500MB with 1/10th of the time.

antonio-ivanovski · 2023-11-07T10:24:17Z

After bit of debugging decided to time the hooks execution times. As suspected the rollup dts plugin is the culprit with the hook for options that in my case took ~30s to execute. Will investigate further to find what exactly is the issue.

millerized · 2023-12-13T16:14:42Z

As an extra data point (same as @coopbri) I can confirm that something in v6.7.0 has created a performance regression -- both in speed and memory usage (OOM).

I tried v8.0.1 today and the problem still persists and seems worse now that my codebase is a little larger. Seems like this is most likely due to entry point count.

Probably part of my issue is (please confirm if true) that I am being lazy to define actual entry points so I wrote a method that fetches all the .ts files in the source and defines those as entry points.

@ToTeTo I wouldn't say this is your issue because I have mine setup to do tsup src/**/**/index.ts .. etc which essentially creates an entry point for every component in my library (30). I do this intentionally to avoid huge barrel file exports because of perf issues in user land.

nolde · 2024-06-03T09:00:29Z

I have been using onSuccess hook to just use tsc to compile the types, while tsup takes care of code. This took any memory issues away, and allowed me to build .d.ts files for all internal files, allowing deep lib reference with exports. It fixed my memory issues during build.

// tsup.config.ts
import { copyFile } from 'node:fs/promises'
import { exec } from 'node:child_process'
import { promisify } from 'node:util'

import glob from 'tiny-glob'
import { defineConfig } from 'tsup'

const pexec = promisify(exec)

export default defineConfig({
  cjsInterop: true,
  clean: true,
  entry: ['src/**/*.ts', '!src/**/*.test.ts'],
  format: ['cjs', 'esm'],
  shims: true,
  sourcemap: false,
  splitting: true,
  target: 'node20',
  //
  async onSuccess () {
    try {
      await pexec('tsc --emitDeclarationOnly --declaration')
      const files = await glob('dist/**/*.d.ts')
      await Promise.all(files.map(file => copyFile(file, file.replace('.d.ts', '.d.mts')))) // or to `.d.cjs` for "type": "module" projects
    } catch (err) {
      console.error()
      console.error('Typescript compilation error:')
      console.error()
      console.error(err.stdout)
      throw err
    }
  }
})

Don't forget to add "outDir": "./dist" to your tsconfig.

tysonclugg · 2024-06-11T02:02:47Z

As per #920 (comment) this may have been fixed upstream in Node v21 and above. Can someone please test and confirm if this is the case?

faithfulojebiyi · 2024-09-28T00:25:02Z

I have been using onSuccess hook to just use tsc to compile the types, while tsup takes care of code. This took any memory issues away, and allowed me to build .d.ts files for all internal files, allowing deep lib reference with exports. It fixed my memory issues during build.

// tsup.config.ts
import { copyFile } from 'node:fs/promises'
import { exec } from 'node:child_process'
import { promisify } from 'node:util'

import glob from 'tiny-glob'
import { defineConfig } from 'tsup'

const pexec = promisify(exec)

export default defineConfig({
  cjsInterop: true,
  clean: true,
  entry: ['src/**/*.ts', '!src/**/*.test.ts'],
  format: ['cjs', 'esm'],
  shims: true,
  sourcemap: false,
  splitting: true,
  target: 'node20',
  //
  async onSuccess () {
    try {
      await pexec('tsc --emitDeclarationOnly --declaration')
      const files = await glob('dist/**/*.d.ts')
      await Promise.all(files.map(file => copyFile(file, file.replace('.d.ts', '.d.mts')))) // or to `.d.cjs` for "type": "module" projects
    } catch (err) {
      console.error()
      console.error('Typescript compilation error:')
      console.error()
      console.error(err.stdout)
      throw err
    }
  }
})

Don't forget to add "outDir": "./dist" to your tsconfig.

Have you had any issues where ot generated wrong types

bengry added a commit to bengry/typedash that referenced this issue May 18, 2023

build: reduce build memory usage

80dcf2e

see egoist/tsup#875

bnwebdev mentioned this issue Oct 30, 2023

SC-2587: New TestFlight release (0.1.2) Hero app build failing Hero24Oy/frontend#47

Merged

antonio-ivanovski mentioned this issue Nov 6, 2023

tsup build failing with ERR_WORKER_OUT_OF_MEMORY #920

Open

antonio-ivanovski mentioned this issue Nov 7, 2023

Upgrading from tsup 6.6.2 makes memory usage during DTS spike #867

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: Worker terminated due to reaching memory limit: JS heap out of memory #875

Error: Worker terminated due to reaching memory limit: JS heap out of memory #875

WINOFFRG commented Apr 3, 2023 •

edited by polar-sh bot

Loading

juniorwmr commented Apr 25, 2023

anhntcv commented May 17, 2023

Achaak commented May 17, 2023

anhntcv commented May 18, 2023

kysley commented Aug 1, 2023

coopbri commented Aug 31, 2023

9vfQbg7z4ajrGQxR commented Oct 30, 2023

antonio-ivanovski commented Nov 6, 2023

antonio-ivanovski commented Nov 6, 2023 •

edited

Loading

antonio-ivanovski commented Nov 7, 2023

millerized commented Dec 13, 2023 •

edited

Loading

nolde commented Jun 3, 2024

tysonclugg commented Jun 11, 2024

faithfulojebiyi commented Sep 28, 2024

Error: Worker terminated due to reaching memory limit: JS heap out of memory #875

Error: Worker terminated due to reaching memory limit: JS heap out of memory #875

Comments

WINOFFRG commented Apr 3, 2023 • edited by polar-sh bot Loading

Upvote & Fund

juniorwmr commented Apr 25, 2023

anhntcv commented May 17, 2023

Achaak commented May 17, 2023

anhntcv commented May 18, 2023

kysley commented Aug 1, 2023

coopbri commented Aug 31, 2023

9vfQbg7z4ajrGQxR commented Oct 30, 2023

antonio-ivanovski commented Nov 6, 2023

antonio-ivanovski commented Nov 6, 2023 • edited Loading

antonio-ivanovski commented Nov 7, 2023

millerized commented Dec 13, 2023 • edited Loading

nolde commented Jun 3, 2024

tysonclugg commented Jun 11, 2024

faithfulojebiyi commented Sep 28, 2024

WINOFFRG commented Apr 3, 2023 •

edited by polar-sh bot

Loading

antonio-ivanovski commented Nov 6, 2023 •

edited

Loading

millerized commented Dec 13, 2023 •

edited

Loading