Skip to content

Add Distributed Snowflake Server Support

shelikhoorequested to merge
shelikhoo/snowflake:dev-mubroker into main

This changeset adds Distributed Snowflake Server Support that allows more than one Snowflake Servers to share the same proxy pool.

Things to look at:

  • It adds 3 metrics for rejected proxies polls, proxies polls with relay URL extensions, and those without. (Both file-based and prometheus based are added) Please confirm it works with our existing infrastructure, and nothing will blow up.
  • It will allow graceful upgrade by allowing the broker to update first with support for existing clients and proxies to give a chance for proxy operators to update their proxies.
  • Broker sends the proxy the relay URL so that the proxy can connect to a secondary snowflake server.
  • The proxy checks if this URL has an allowed host according to a pattern and whether the HTTPS connection requirement is set.
  • The proxy sends its allowed URL pattern to the broker. The broker rejects proxies that are too restrictive.

Extra Changes;

  • db1db4cb : Fixed desynchronized comment and behavior for log interval

See Also: #40129 (closed) #28651 (comment 2787394)

Edited by shelikhoo

Merge request reports