Enable semantics similar to java.lang.Thread?

[alexandergunnarson]

I've been having ridiculous problems with trying to get this to work in a non-headachy way, not through any fault of your own. This is a great library and I appreciate your work! But I'm using ClojureScript (a compile-to-JS language) and due to its fancy use of Google Closure and other such things, I'm not able to get Workers to work in an easy way (I can use them on the web just fine with ClojureScript). I'd love to have this library support (in addition to the existing WebWorkers semantics, because I suppose some people want that) java.lang.Thread-esque semantics like this:

var fnICanRunOnAnyThread = function () {
  if (Thread.isMainThread) {
    console.log("Hello world from main thread!");
    console.log("I can also do UI things!");
    ReactNative.WhateverComponent.whateverUIFunctionWorkersCantCallWithoutError();
  } else {
    console.log("Hello world from worker (non-main) thread!");
    console.log("I can't do UI things but that's okay cause these semantics are awesome!");
  }
}

var myFn = function () {
  new Thread(fnICanRunOnAnyThread);
}

I'm thinking that under the hood Thread might use the native bridge to spin up a JS VM and load all the JS code that the current context has access to (yes, this part would be hard), and then inject a boolean Thread.isMainThread = true (I'm not sure if this is possible but it if it is, it seems like it would be simple and it would be dang nice).

Thoughts on feasibility?
I'd love to contribute to help make this possible.

[seantempesta]

Hey Alex. I'm also using Clojurescript, and it's working fine for me. Is there anything I can help you with (besides writing a new interface)?

[alexandergunnarson]

Thanks so much for commenting and being so helpful @seantempesta ! Glad to know there's someone out there using ClojureScript with React Native and WebWorkers :D Basically for now I'm trying to get it to work with Figwheel to work with it (I'll figure out the production part later, if there is indeed anything additional to figure out). I'd like to use it with the CLJS Servant library (for WebWorkers in the browser) do the following currently (new semantics aside):

(ns arbitrary)

(def Worker (.-Worker (js/require "react-native-workers")))

(defn run-this-on-whatever-thread []
  (if Worker.onWorkerThread ; this doesn't exist — is there something similar to it?
    (println "in worker thread")
    (println "in main thread")))

(defn spawn-thread []
  ; this doesn't work, but would work with the Servant library
  (js/Worker. "/target/path/to/arbitrary-compiled-by-figwheel.js")
  ; This is where the Servant library would call `.onmessage` for the Worker, but
  ; .onmessage (and the equivalent (.addEventListener <Worker> "message" <fn>))
  ; isn't a property of Worker in this (React Native) library
  )

I've tried using a separate worker.js file which requires the "/target/path/to/arbitrary-compiled-by-figwheel.js", but that was a HUGE headache with Google Closure and involved fiddling with things that shouldn't need fiddling. The main thing is that I want the Worker to have the same context as the caller (the main thread). How are you making things work with your setup?

[seantempesta]

I haven't figured out a way to get the worker to run in figwheel. I'm compiling the worker to a separate worker.js file using :simple cljs compilation mode, then building a bundle using the react-native packager:

/usr/local/bin/node --expose-gc --max_old_space_size=4096 "./node_modules/react-native/local-cli/cli.js" bundle --entry-file ./worker.js --platform ios --dev false --reset-cache true --bundle-output "ios/worker.jsbundle"

and am loading it by a manual patch to WorkerManager.m:

   NSURL *workerURL = [[NSBundle mainBundle] URLForResource: @"worker" withExtension:@"jsbundle"];
    //NSURL *workerURL = [[RCTBundleURLProvider sharedSettings] jsBundleURLForBundleRoot:name fallbackResource:nil];

Here's a rough version of my code (let me know if you want the full version?):

worker.js

(ns project.worker)
(def Self (.-self (js/require "react-native-workers"))

(defn on-message [transit-message]
  (let [message (t/read tr transit-message)
        operation (first message)
        args (first (rest message))]
    (info "WORKER: Message received:" (str operation))
    (case operation
      :subscribe (receive-subscription args)
      :dispatch (receive-dispatch args)
      :dispatch-sync (receive-dispatch-sync args))))

;; init and send a message to the main thread when ready
(aset Self "onmessage" on-message)
(let [ready-transit-m (t/write tw [:worker-ready])]
  (.postMessage Self ready-transit-m)
  (info "WORKER: Ready"))

And then in my normal project.ios.core file:

(defonce Worker (.-Worker (js/require "react-native-workers")))
(defonce worker (atom nil))                                 ;; worker ref stored here
(defonce worker-ready? (r/atom false))             

(defn on-message [transit-message ready-fn]
  (let [message (t/read tr transit-message)
        operation (first message)
        args (first (rest message))]
    (trace "MAIN: Received message from worker" operation)
    (case operation
      :worker-ready (ready-fn)
      :worker-logging (receive-worker-logging args)
      :reaction-results (receive-reaction-results args))))

(defn init-worker [worker-file ready-fn]
  (debug "MAIN: Starting worker")
  (let [worker-init (Worker. worker-file)]
    (aset worker-init "onmessage" #(on-message % ready-fn))
    (reset! worker worker-init)))

;; Start the worker
(init-worker "worker.js" (fn [] (reset! worker-ready? true))))

So you might be thinking that this would be a horrible dev environment? It would be if I had to re-compile, bundle and restart the simulator on every change, so I just wrote an abstraction library that passes the calls directly whatever functions the worker is using (in the case, re-frame subscribe, dispatch and dispatch-sync):

;;;;;;;  API ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; When in production pass all re-frame requests to the worker
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defonce use-worker? (atom false))

(defn dispatch [dispatch-v]
  (if @use-worker?
    (worker-utils/dispatch dispatch-v)
    (re-frame.core/dispatch dispatch-v)))

(defn dispatch-sync [dispatch-v]
  (if @use-worker?
    (worker-utils/dispatch-sync dispatch-v)
    (re-frame.core/dispatch-sync dispatch-v)))

(defn subscribe [sub-v]
  (if @use-worker?
    (worker-utils/subscribe sub-v)
    (re-frame.core/subscribe sub-v)))

(defn init-worker [worker-file ready-fn]
  (reset! use-worker? true)
  (worker-utils/init-worker worker-file ready-fn))

I don't know if you're using re-frame, but this worked tremendously well for me. I just keep all of my heavy computations in the subscriptions and handlers and the main thread just renders the results.

[alexandergunnarson]

Thanks for your detailed and helpful reply @seantempesta ! Yes, I'm using re-frame (with Reagent). I think it's a really great idea to pass (most of) the dispatches to a worker! That makes the separation of concerns really clean and has the potential to drastically improve performance (provided that it's not a super simple dispatch like a an assoc to app-db that's needed in sub-ms time for rendering purposes). Also a great idea to use Transit for inter-thread (inter VM) communication!

However... what do you do when you modify code in development/Figwheel that is used by the worker thread, whether exclusive to it or shared with the main thread? I guess what you were saying is that since you haven't gotten workers to run in Figwheel, then you'd probably just set the use-worker? atom to false and run everything on the main thread, so your changes would be reflected automatically and you wouldn't have to rebuild every time. In that case either you might run the computationally expensive code on the main thread (less than ideal, because your UI wouldn't behave as responsively as it would with the workers enabled and you wouldn't always be able to pin the problem on the lack of workers), or you might simply choose not run computationally expensive code period, just spoof the results on the main thread, and test it on a different instance of React Native entirely (probably an even worse option).

I mean, it's great to hear that it runs in ClojureScript! Don't get me wrong about that. I think your solution is the most elegant one possible given the constraints of Figwheel and the fiddly nature of workers (which require a separate JS VM and a worker-specific .js entry point) as compared to e.g. Java threads which share context. It still just seems far less fluid than would be optimal for rapid development. What do you think?

[alexandergunnarson]

So I did some digging around and saw that react-native-workers uses the React Native JSBundleLoader class in Android to load the worker .js file(s) (I don't really know Objective-C super well so I haven't looked at that side of things, but I assume it works roughly the same way). Since it ultimately works the same way as WebWorkers (that is, to create a different Thread, you have to load a .js file), I'm thinking of this as a rough pseudocode solution:

import { Worker } from "react-native-workers"

static class JSGlobalThreadpool {
  const threads = new HashMap<String, HashSet<Worker>>()
  private int size = 0
  int maxThreads = 0 // disables WebWorkers by default
  private boolean isDestroyed = false

  add(String pathToJSFile) {
    if (size >= maxThreads)
      throw new Error("Attempted to initialize more than max threads (" + maxThreads + ") from path '" + pathToJSFile + "'")
    } else {
      threads.addDeep(pathToJSFile, new Worker(pathToJSFile))
      size ++
    }
  }

  destroy() {
    foreach thread in threads { thread.terminate() }
    isDestroyed = true
  }

  isDestroyed() { return isDestroyed }

  remove(String pathToJSFile) { <remove and destroy a thread matching the path, if present> }

  getFirstAvailable(String pathToJSFile) { return <first available/non-busy JS worker matching the filename> }
}

class Thread {
  // more convenient (and less feasible) constructor
  constructor(Function f) {
    String thisPath = <somehow determine what this code's JS bundle file path is... maybe have a global like global.thisPath which is added to the top of the code within the bundle?>
    this(thisPath, f)
  }

  // less convenient (and more feasible) constructor
  constructor(String pathToJSFile, Function f) {
    Worker chosenWorker = JSGlobalThreadpool.getFirstAvailable(pathToJSFile)
    if (chosenWorker == null) JSGlobalThreadpool.add(pathToJSFile)
    chosenWorker.postMessage(serializeFunction(f))
  }

  serializeFunction(Function f) {
    return <maybe a string which is evaled by the worker, with some kind of security to guarantee to the worker that the function comes from a secure source>
  }
}

What do you think, @devfd ?

Meanwhile... for Figwheel's sake @seantempesta we could have some sort of additional service/middleman (or maybe modify Re-Natal or something?) which, every time the bundle is changed, outputs a JS worker bundle which at the top says something like var isWorker = true or global.isWorker = true, which variable can then be used to dispatch on to prevent execution of UI-specific (or worker-specific) code as detailed in my first post on this thread.

[seantempesta]

@alexandergunnarson: These are all good questions (most of which I don't have answers to). However, I finally got around to packaging the code I've been using into a library. I think you might find it useful.

I put an example in there that runs re-frame normally during development (where it's okay if it's slower) and then uses the worker in production. That seems to be the easiest way to use this library with chrome debugging ATM.

https://github.com/seantempesta/cljsrn-re-frame-workers

[alexandergunnarson]

That's awesome @seantempesta ! Thanks for sharing. I'll take a look at what you wrote and try it out. Your solution is probably the closest we ClojureScript people are going to get until the pseudocode I wrote (along with the extra service/middleman I mentioned) is implemented, if it ever is. I think I'm going to at least try implementing it sometime soon.