When you run pull queries, it’s often the case that you need your data to remain available for querying even if one server fails. Because ksqlDB supports clustering, it can remain highly available to support pull queries on replicas of your data, even in the face of partial cluster failures.
High availability is turned off by default, but you can enable it with the following server configuration parameters. These parameters must be turned on for all nodes in your ksqlDB cluster.
ksql.streams.num.standby.replicasto a value greater than
In addition, make sure that all of your nodes identify as part of the same
cluster by setting
to the same value.
In Confluent Cloud, ksqlDB clusters with 8 or 12 CSUs are automatically configured for high availability. High availability can't be enabled for Confluent Cloud clusters with fewer than 8 CSUs.
Because ksqlDB replicates data between its servers asynchronously, you may want
to bound the potential staleness that your query will tolerate. You can control
this per pull query by using the
parameter. For instance, a value of
10,000 means that results of pull queries
forwarded to servers whose current offset is more than
behind the end offset of the changelog topic are rejected.
Compatability with Authentication¶
authentication.skip.paths config with both
This enables ksqlDB cluster instances to communicate without authenticating between each other.
This configuration prevents the following error.
For security reasons, this configuration is recommended to block the traffic from outside your cluster to those endpoints.