Feature: worker pooling #27

Activity

1. TimvanScherpenzeel commented on Jun 30, 2018

   

   TimvanScherpenzeel

   on Jun 30, 2018

   More actions

   @developit recently added this gist that explores the idea of a worker pool: https://gist.github.com/developit/65a2212731f6b00a8aaa55d70c594f5c

   I was wondering @developit, is this something you are planning to add to the repo at some stage or should it be seen more like a rough sketch or like a standalone extension of the library?

   It looks like an attempt to solve the TODO's mentioned in workerize:
   

   workerize/src/index.js

   Lines 2 to 4 in 683631f

   	/** TODO:
   	* - pooling (+ load balancing by tracking # of open calls)
   	* - queueing (worth it? sortof free via postMessage already)
2. jzyzxx commented on Jun 24, 2021

   

   jzyzxx

   on Jun 24, 2021

   More actions

   Hello,

   It would be great if I could create a pool of workers. That way, parallelizing hot paths could be completely abstracted using workerize.

   Considerations:

   • To implement worker pooling, a worker may have to know when another worker is busy, so it can handle take over handling new requests until becomes busy itself
   • Since workers independently manage memory, they would all have to run the same code, and there would have to be a way to run a function on all workers simultaneously in order to share state

   It will be interesting to see what we will need to do for this. Would you accept a PR?

   Thanks for considering!

   Good idea!