Populate conflicting ports

Quoted from issue internally by @thegedge:

Whenever toxiproxy.json is changed such that a new proxy is added somewhere in the middle of the list, it gets assigned a port number for a pre-existing proxy. Toxiproxy.populate cannot find this proxy and tries to create it, but someone is already bound to that port and we end up with a 500 from toxiproxy. This is currently solved with a dev down; dev up, which will reset the toxiproxy state completely so things work.

The patch looks like this:

From 0ed9b7dba66ce062d9b1db8786b1b387076fbbd3 Mon Sep 17 00:00:00 2001
From: Jason Gedge <jason.gedge@shopify.com>
Date: Mon, 10 Apr 2017 18:31:35 -0400
Subject: [PATCH] Destroy all proxies when receiving 500 from toxiproxy

 eagerlib/shopify/bootstrap.rb | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/eagerlib/shopify/bootstrap.rb b/eagerlib/shopify/bootstrap.rb
index 7405fe55..1ad3aea 100644
--- a/eagerlib/shopify/bootstrap.rb
+++ b/eagerlib/shopify/bootstrap.rb
@@ -57,7 +57,12 @@ def self.run(use_cache = false)
         want_data = MessagePack.dump(pid: railgun_pid, config: config)
         return if have_data == want_data # comparing msgpack'd data
 
-        ::Toxiproxy.populate(config)
+        begin
+          ::Toxiproxy.populate(config)
+        rescue Net::HTTPFatalError
+          ::Toxiproxy.all.each(&:destroy)
+          ::Toxiproxy.populate(config)
+        end
         File.binwrite(LAST_POPULATE_MARKER, want_data)
       end
 

It seems like we should prevent this issue upstream.

Start with the reported Toxiproxy.populate behavior and the example workaround in eagerlib/shopify/bootstrap.rb, then trace how adding a proxy in the middle of toxiproxy.json affects existing port assignments. Reproduce the conflict and determine how population should handle it; done means the configuration can be updated without a 500 or requiring a full dev reset.