RudderStack makes it easier for you to add more nodes to your Kubernetes cluster, to process the events in a timely and effective manner. This guide shows you how to effectively configure your Helm charts to trigger a horizontal scale up/down, based on the version of RudderStack you are using.
RudderStack Open-source Version
Each RudderStack node consists of a backend pod and a PostgreSQL pod. In case a RudderStack cluster is not able to process the events, it is advisable to add more more nodes to it. By changing the backendReplicaCount in the Helm configuration, you can trigger horizontal scale up/down. If you set this value to any positive number, helm charts will create as many RudderStack nodes.
values.yaml file can be found here.There are, however, a couple of challenges in scaling up or down this way.
Event Ordering
In a single-node cluster, all events are routed to the same RudderStack node. This guarantees the ordering of all events it has received for a given user. In case of a multi-node cluster, since the events of a same user can go to different nodes, the event ordering is not guaranteed by the cluster.
.png "image 17")
Event Ordering While Scaling Up from 1 Node to 2 NodesScale Down
If you scale down, all the pending events in the terminated node will not be delivered to the destination. You will need to make sure that all the events in a node are drained before terminating it.
.png "image 65")
Scaling Down from 2 Nodes to 1 NodeRudderStack Enterprise Version
The Enterprise version of RudderStack makes horizontal scaling seamless, while maintaining the event ordering and guaranteeing the delivery of all events.
Event Ordering
RudderStack Enterprise has a proxy service that implements a routing algorithm. It forwards a given set of user events to the same RudderStack node every time. Each node then guarantees the event ordering while delivering to the specified destinations. This proxy service is highly available and scalable.
.png "image 27")
Event Ordering in RudderStack EnterpriseScale Down
RudderStack Enterprise includes a separate process named Jobs DB Migrator. This process is responsible for moving all pending events from the existing nodes to the new nodes after scaling up or down. It makes sure all the events are migrated to the new nodes and the event ordering is maintained during and after the migration process.
.png "image 55")
Scaling Down in RudderStack EnterpriseContact us
For queries on any of the sections covered in this guide, you can contact us or start a conversation in our Slack community.