Allow the configuration to change automatically at runtime

`#load_config` is now called whenever a worker completes
a job, allowing a configuration loader to modify
the config each time.

Some configuration strategies (like the default
config YAML file loader) may want to return the same
value every time until explicitly asked to reload.
It is the loader's responsibility to cache the value
they return. They can implement a `#reset!` method,
which will be called when the HUP signal is received.

Modified the config file specs to demonstrate that
repeated calls to `#load_config` are harmless and do
not reload the configuration from disk.

README.md

@@ -113,7 +113,7 @@ SIGNALS
 
 The pool manager responds to the following signals:
 
-* `HUP`   - reload the config file, reload logfiles, restart all workers.
+* `HUP`   - reset config loader (reload the config file), reload logfiles, restart all workers.
 * `QUIT`  - gracefully shut down workers (via `QUIT`) and shutdown the manager
   after all workers are done.
 * `INT`   - gracefully shut down workers (via `QUIT`) and immediately shutdown manager
@@ -155,6 +155,14 @@ Resque::Pool.config_loader = lambda {|env|
 end
 ```
 
+The configuration loader's `#call` method will be invoked every time a worker
+completes a job. This allows the configuration to constantly change, for example,
+to scale the number of workers up/down based on different conditions.
+If the response is generally static, the loader may want to cache the value it
+returns. It can optionally implement a `#reset!` method, which will be invoked
+when the HUP signal is received, allowing the loader to flush its cache, or
+perform any other re-initialization.
+
 Other Features
 --------------
 

lib/resque/pool.rb

@@ -110,6 +110,11 @@ def load_config
       @config = config_loader.call(environment)
     end
 
+    def reset_config
+      config_loader.reset! if config_loader.respond_to?(:reset!)
+      load_config
+    end
+
     def environment
       if defined? RAILS_ENV
         RAILS_ENV
@@ -176,8 +181,8 @@ def handle_sig_queue!
         log "#{signal}: sending to all workers"
         signal_all_workers(signal)
       when :HUP
-        log "HUP: reload config file and reload logfiles"
-        load_config
+        log "HUP: reset configuration and reload logfiles"
+        reset_config
         Logging.reopen_logs!
         log "HUP: gracefully shutdown old children (which have old logfiles open)"
         if term_child
@@ -280,6 +285,7 @@ def join
         break if handle_sig_queue! == :break
         if sig_queue.empty?
           master_sleep
+          load_config
           maintain_worker_count
         end
         procline("managing #{all_pids.inspect}")

lib/resque/pool/file_or_hash_loader.rb

@@ -11,7 +11,11 @@ def initialize(filename_or_hash=nil)
   end
 
   def call(environment)
-    load_config_from_file(environment)
+    @config ||= load_config_from_file(environment)
+  end
+
+  def reset!
+    @config = nil
   end
 
   private

spec/resque_pool_spec.rb

@@ -184,13 +184,17 @@ module Rails; end
 
       File.open(config_file_path, "w"){|f| f.write "changed: 1"}
       subject.config.keys.should == ["orig"]
+      subject.load_config
+      subject.config.keys.should == ["orig"]
     end
 
     it "should reload the changes on HUP signal" do
       subject.config.keys.should == ["orig"]
 
       File.open(config_file_path, "w"){|f| f.write "changed: 1"}
       subject.config.keys.should == ["orig"]
+      subject.load_config
+      subject.config.keys.should == ["orig"]
 
       simulate_signal subject, :HUP
 
@@ -212,22 +216,30 @@ module Rails; end
   end
 
   it "should reset the config loader on HUP" do
-    custom_loader = double(call: Hash.new)
+    custom_loader = double(call: Hash.new, reset!: true)
 
     pool = no_spawn(Resque::Pool.new(custom_loader))
     custom_loader.should have_received(:call).once
 
     pool.sig_queue.push :HUP
     pool.handle_sig_queue!
+    custom_loader.should have_received(:reset!)
     custom_loader.should have_received(:call).twice
   end
 
   it "can be a lambda" do
     RAILS_ENV = "fake"
+    count = 1
     pool = no_spawn(Resque::Pool.new(lambda {|env|
-      {env.reverse => 1}
+      {env.reverse => count}
     }))
     pool.config.should == {"ekaf" => 1}
+
+    count = 3
+    pool.sig_queue.push :HUP
+    pool.handle_sig_queue!
+
+    pool.config.should == {"ekaf" => 3}
   end
 end