Add load balancing to bridge
We rehearsed a load-balanced bridge installation in #40091 (closed). Now let's do it for the production bridge. To reduce risk, we plan to do a staged upgrade using a secondary bridge.
We still do not have a permanent solution to the onion key rotation issue. The current plan is to periodically reset LastRotatedOnionKey in the state file of all tor instances.
-
ask sysadmin team to reduce TTL for snowflake.torproject.net to 60 seconds tpo/tpa/team#40594 (closed) -
copy user accounts to staging bridge #40091 (comment 2768855) -
install new staging bridge (installation guide) -
refresh the LastRotatedOnionKey line in the state file of the production bridge and restart tor -
back up identity and onion keys from production bridge -
copy identity and onion keys from production bridge to the staging bridge -
copy HTTPS TLS keys and certificates from the production bridge to the staging bridge -
test HTTPS of staging bridge using curl --connect-to -
test tor bootstrap on staging bridge using local broker and proxy, and temporary domain name #40091 (comment 2770360) -
switch DNS for snowflake.torproject.net to point to staging bridge tpo/tpa/team#40598 (closed) -
monitor for a day, and be ready to switch DNS back to production if connections fail on the staging bridge -
disable and mask tor@default instance on production bridge -
install load balancing configuration on production bridge installation guide -
test HTTPS of production bridge using curl --connect-to -
test tor bootstrap on production bridge using local broker and proxy, and temporary domain name #40091 (comment 2770360) -
switch DNS for snowflake.torproject.net to point back to production bridge tpo/tpa/team#40602 (closed) -
monitor for 2 days, and be ready to switch DNS back to staging if connections fail on the production bridge -
ask sysadmin team to restore TTL for snowflake.torproject.net to normal tpo/tpa/team#40595 (closed) -
shut down staging bridge -
post new instructions to Snowflake Bridge Installation Guide and Snowflake Bridge Survival Guide
References