Error: mathematics con

[jianghumu]

Error: mathematics con
0|app | at module.exports.__wbindgen_error_new (/home/chatgpt-serve/node_modules/@dqbd/tiktoken/tiktoken_bg.cjs:410:17)
0|app | at null. (wasm://wasm/00b5f812:1:168780)
0|app | at module.exports.encoding_for_model (/home/chatgpt-serve/node_modules/@dqbd/tiktoken/tiktoken_bg.cjs:177:14)

Is it because the lack of calling enc.free(); to release it?

[dqbd]

Hello, @jianghumu!

I believe that is the case, afaict WASM becomes unreliable after saturating the available memory. Will address the issue with memory leaks in the future, see #35, but in the meantime, consider freeing the encoder after doing the work.

[dqbd]

This should be addressed with never versions (although it does have some issues with lost support in other environments #46). It is still recommended to manually call free() when the encoder is not needed anymore, as FinalizationRegistry is not a bullet-proof and memory leaks might still happen (eg. when instantiating in a loop within a single event loop).

Closing for now! Feel free to reopen if needed :)