You can develop event streaming applications by using the KSQL user interface provided by Confluent Control Center. In Control Center, you can create Apache Kafka® topics and develop persistent queries in the KSQL query editor. When you install Confluent Platform, KSQL Server is integrated with Control Center by default, and you can configure Control Center to interact with other KSQL Server instances that run on separate hosts.
Configuration Settings for KSQL and Control Center¶
Set up the integration between KSQL and Control Center by assigning properties in the KSQL Server and Control Center configuration files.
- By default, the KSQL Server configuration file is installed at
<path-to-confluent>/etc/ksqldb/ksql-server.properties. - By default, the Control Center configuration file is installed
at
<path-to-confluent>/etc/confluent-control-center/control-center.properties.
Secure Communication with KSQL Server¶
When you use Basic authentication in KSQL, Control Center allows passing credentials as part of the KSQL Server URL in Control Center configuration.
1 2 | |
You can set up KSQL Server to communicate securely with other components in Confluent Platform. For more information, see Configure Security for KSQL.
Network Configuration for KSQL and Control Center¶
These are the configuration settings that you assign to set up network connectivity between KSQL and Control Center.
- In the KSQL Server configuration file, set the
listeners property to the IP address of the REST
API endpoint for KSQL Server. Typical values are
http://0.0.0.0:8088andhttp://localhost:8088. - In the Control Center configuration file, set the
confluent.controlcenter.ksql.<ksql-cluster-name>.urlproperty to the hostnames and listener ports for the KSQL cluster specified by<ksql-cluster-name>. This setting specifies how Control Center communicates with KSQL Server for regular HTTP requests. For more information, see Control Center Configuration Reference. - If KSQL Server communicates over an internal DNS that is not
externally resolvable or routeable, set the
confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.urlproperty in the Control Center configuration file. This setting specifies how the browser communicates with KSQL Server for websocket requests. For more information, see Control Center Configuration Reference.
When KSQL Server and Confluent Control Center run on the same host, you can use the default configuration defined by Confluent Platform setup.
When KSQL and Control Center run on different hosts¶
If KSQL Server and Control Center run on different hosts, you must specify a configuration that ensures KSQL Server and Control Center can communicate. This is necessary when KSQL Server and Control Center are deployed in the following situations:
- KSQL Server and Control Center run in separate containers.
- They run in separate virtual machines.
- They communicate over a virtual private network (VPN).
- The KSQL Server host publishes a public URL that's different from the private URL for KSQL Server.
Note
When KSQL and Control Center communicate over a virtual private
network (VPN), Control Center proxies your queries, but query
results stream directly from KSQL Server back to your browser without
going through Control Center. Over a VPN, the advertised URL isn't
localhost. Instead, it's the hostname of the remote server.
Assign the following configuration properties to integrate KSQL Server with Control Center when they run on separate hosts.
KSQL Server Configuration¶
In the KSQL Server configuration file, set listeners to bind to all
interfaces:
1 | |
Control Center Configuration¶
In the Control Center configuration file, set
confluent.controlcenter.ksql.<ksql-cluster-name>.url to a list of URLs
for the KSQL Server hosts, which must be reachable from the host that
Control Center is installed on. Replace <ksql-cluster-name> with the
name that Control Center uses to identify the KSQL cluster.
1 | |
Also, set
confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url to the
public IP addresses published by the KSQL Server hosts, which must be a
list of URLs that the browser can resolve through externally available
DNS.
1 | |
The Control Center configuration must match the KSQL Server
listeners values.
Use the curl command to check whether these URLs are reachable.
Depending on your deployment, you may need to check from two different
hosts:
- Check from the host where Control Center is running, which is
relevant for the
confluent.controlcenter.ksql.<ksql-cluster-name>.urlsetting. - Check from the host where the browser is running, which is relevant
for the
confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.urlsetting.
On both hosts, run the following command to confirm that the KSQL Server
cluster is reachable. The hostname value is one of the hosts in the
listed in the confluent.controlcenter.ksql.<ksql-cluster-name>.url and
confluent.controlcenter.ksql.<ksql-cluster-name>.advertised.url
configuration settings.
1 2 | |
Note
You must specify the ports in the KSQL URL settings. For example, if the
public URL is
http://ksql-server-677739697.us-east-1.elb.amazonaws.com:80, be sure
to include port 80, or the Control Center connection to KSQL
Server will fail.
Check Network Connectivity Between KSQL and Control Center¶
Use a web browser to check the configuration of an advertised URL. Make
sure that your browser can reach the info endpoint at
http://<ksql.advertised.url>/info. If the configuration is wrong, and
the browser can't resolve the URL of the KSQL Server host, you'll
receive an error:
Websocket error when communicating with <ksql.advertised.url>.
Check KSQL Server Network Binding¶
If Control Center doesn't connect with your KSQL Server instance, check the network binding on the KSQL Server host:
1 | |
If your KSQL server is bound to localhost only, your output should
resemble:
1 2 | |
If 0.0.0.0 isn't listed, KSQL Server isn't accepting external
connections. In the ksql-server.properties file, set
listeners=http://0.0.0.0:8088 and restart KSQL Server.