Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilize Node Workers to parallelize files #141

Open
JoelYoung01 opened this issue Feb 7, 2024 · 1 comment
Open

Utilize Node Workers to parallelize files #141

JoelYoung01 opened this issue Feb 7, 2024 · 1 comment

Comments

@JoelYoung01
Copy link

Hello! I was poking at the code and I noticed the use of Promises to run the migrate file work per file and was thinking it could be even further optimized if the process was updated to use Node Workers.

I saw in the contributing guidelines that it was recommended to open an issue for some discussion first, so here we are. Anyone have any thoughts on this?

@JoelYoung01
Copy link
Author

JoelYoung01 commented Feb 8, 2024

I played with this idea for an hour or two, but ran into a lot of issues trying to serialize the data from the main to the worker threads.

I'd really be interested to hear if anyone with more experience with worker threads in Node thinks of this idea and it's feasibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant