{"id":2973,"date":"2026-06-16T01:06:55","date_gmt":"2026-06-16T01:06:55","guid":{"rendered":"https:\/\/sreschool.com\/blog\/?p=2973"},"modified":"2026-06-16T01:06:56","modified_gmt":"2026-06-16T01:06:56","slug":"kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka","status":"publish","type":"post","link":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/","title":{"rendered":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka"},"content":{"rendered":"\n

1. First understand the four layers<\/h2>\n\n\n\n
flowchart TD\n    A[Application \/ Tool \/ Terraform \/ Connector] --> B[Connection Method]\n    B --> B1[Native Kafka Protocol]\n    B --> B2[Kafka REST API]\n    B --> B3[Confluent Cloud Management API]\n    B --> B4[Confluent CLI]\n    B --> B5[Managed Connectors]\n    B --> B6[Cluster Linking]\n\n    B1 --> C[Network Path]\n    B2 --> C\n    B3 --> C\n\n    C --> C1[Public Internet]\n    C --> C2[Public + IP Filtering]\n    C --> C3[AWS PrivateLink]\n    C --> C4[VPC Peering]\n    C --> C5[AWS Transit Gateway]\n    C --> C6[Private Network Interface]\n\n    C --> D[Authentication]\n    D --> D1[Kafka API Key \/ Secret]\n    D --> D2[Cloud API Key \/ Secret]\n    D --> D3[Schema Registry API Key \/ Secret]\n    D --> D4[OAuth \/ OIDC]\n    D --> D5[mTLS Client Certificate]\n\n    D --> E[Authorization]\n    E --> E1[Kafka ACLs]\n    E --> E2[Confluent RBAC]\n<\/code><\/pre>\n\n\n\n
Confluent Cloud supports native Kafka clients in many languages, including Java, Python, Go, JavaScript, .NET, C\/C++, and others. For normal producer\/consumer workloads, this is usually the best and most standard path. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
2. Method 1 \u2014 Native Kafka protocol<\/h1>\n\n\n\n
This is the direct Kafka access<\/strong> method.<\/p>\n\n\n\n
Your application connects directly to the Kafka bootstrap server using the Kafka protocol.<\/p>\n\n\n\n
Application \/ Pod \/ Service\n        |\n        | Kafka protocol\n        | SASL_SSL \/ mTLS\n        v\nConfluent Kafka bootstrap endpoint :9092\n<\/code><\/pre>\n\n\n\n
Confluent Cloud Kafka clients commonly use:<\/p>\n\n\n\n
bootstrap.servers=<BOOTSTRAP_SERVER>\nsecurity.protocol=SASL_SSL\nsasl.mechanism=PLAIN\nsasl.username=<KAFKA_API_KEY>\nsasl.password=<KAFKA_API_SECRET>\n<\/code><\/pre>\n\n\n\n
The official Confluent Java example uses bootstrap.servers<\/code>, security.protocol=SASL_SSL<\/code>, sasl.mechanism=PLAIN<\/code>, and API key\/secret in the JAAS config. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Use cases<\/h2>\n\n\n\n
Use this for:<\/p>\n\n\n\n
Use case<\/th>Recommended?<\/th><\/tr><\/thead>
Microservices producing\/consuming events<\/td>Yes<\/td><\/tr>
EKS workloads<\/td>Yes<\/td><\/tr>
High-throughput streaming<\/td>Yes<\/td><\/tr>
Long-running consumers<\/td>Yes<\/td><\/tr>
Consumer groups<\/td>Yes<\/td><\/tr>
Exactly-once \/ transactions<\/td>Yes<\/td><\/tr>
Kafka Streams<\/td>Yes<\/td><\/tr>
Low-latency event processing<\/td>Yes<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Example: Java client config<\/h2>\n\n\n\n
bootstrap.servers=pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\nsecurity.protocol=SASL_SSL\nsasl.mechanism=PLAIN\nsasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"KAFKA_API_KEY\" password=\"KAFKA_API_SECRET\";\nclient.id=orders-service\n<\/code><\/pre>\n\n\n\n
Example: Python client<\/h2>\n\n\n\n
from confluent_kafka import Producer\n\nconf = {\n    \"bootstrap.servers\": \"pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\",\n    \"security.protocol\": \"SASL_SSL\",\n    \"sasl.mechanism\": \"PLAIN\",\n    \"sasl.username\": \"KAFKA_API_KEY\",\n    \"sasl.password\": \"KAFKA_API_SECRET\",\n    \"client.id\": \"orders-producer\"\n}\n\nproducer = Producer(conf)\n\nproducer.produce(\n    \"orders\",\n    key=\"order-1001\",\n    value='{\"order_id\": \"1001\", \"status\": \"CREATED\"}'\n)\n\nproducer.flush()\n<\/code><\/pre>\n\n\n\n
Example: Node.js \/ JavaScript<\/h2>\n\n\n\n
const { Kafka } = require(\"@confluentinc\/kafka-javascript\");\n\nconst kafka = new Kafka({\n  kafkaJS: {\n    brokers: [\"pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\"],\n    ssl: true,\n    sasl: {\n      mechanism: \"plain\",\n      username: process.env.KAFKA_API_KEY,\n      password: process.env.KAFKA_API_SECRET\n    }\n  }\n});\n\nconst producer = kafka.producer();\n<\/code><\/pre>\n\n\n\n
Confluent\u2019s JavaScript example also uses brokers, SSL, SASL plain, API key, and API secret. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
3. Method 2 \u2014 Kafka REST API<\/h1>\n\n\n\n
This is Kafka over HTTPS<\/strong>.<\/p>\n\n\n\n
Instead of using a native Kafka client, you call an HTTPS endpoint.<\/p>\n\n\n\n
Application \/ Script \/ Serverless Function\n        |\n        | HTTPS REST API\n        | Basic auth using API key\/secret\n        v\nConfluent Kafka REST endpoint :443\n<\/code><\/pre>\n\n\n\n
Confluent Cloud exposes a Kafka REST endpoint per cluster, usually on port 443<\/code>, while the native Kafka bootstrap server uses port 9092<\/code>. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Use cases<\/h2>\n\n\n\n
Use Kafka REST API when:<\/p>\n\n\n\n
Use case<\/th>Why REST helps<\/th><\/tr><\/thead>
Serverless functions<\/td>Easier than keeping Kafka connections<\/td><\/tr>
Shell scripts<\/td>Simple curl<\/code> works<\/td><\/tr>
Third-party integration<\/td>Partner does not need Kafka client<\/td><\/tr>
Restricted firewall<\/td>HTTPS 443 may be easier<\/td><\/tr>
Low-throughput producers<\/td>Simple and acceptable<\/td><\/tr>
CI\/CD automation<\/td>Create\/manage topics or produce test events<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Confluent says the Kafka REST API is useful when native Kafka clients are not available, such as serverless functions, CI\/CD pipelines, shell scripts, or restricted environments where only HTTPS is allowed. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
REST API produce example<\/h2>\n\n\n\n
First encode your Kafka API key and secret:<\/p>\n\n\n\n
echo -n \"KAFKA_API_KEY:KAFKA_API_SECRET\" | base64\n<\/code><\/pre>\n\n\n\n
Then produce a record:<\/p>\n\n\n\n
curl -X POST \\\n  -H \"Content-Type: application\/json\" \\\n  -H \"Authorization: Basic <BASE64_API_KEY_AND_SECRET>\" \\\n  \"https:\/\/pkc-xxxxx.ap-northeast-1.aws.confluent.cloud\/kafka\/v3\/clusters\/lkc-xxxxx\/topics\/orders\/records\" \\\n  -d '{\n    \"key\": {\n      \"type\": \"STRING\",\n      \"data\": \"order-1001\"\n    },\n    \"value\": {\n      \"type\": \"JSON\",\n      \"data\": {\n        \"order_id\": \"1001\",\n        \"status\": \"CREATED\"\n      }\n    }\n  }'\n<\/code><\/pre>\n\n\n\n
Confluent\u2019s REST API examples use Authorization: Basic <BASE64-encoded-key-and-secret><\/code> and the \/kafka\/v3\/clusters\/<cluster-id>\/topics\/<topic-name>\/records<\/code> endpoint. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
When not to use REST<\/h2>\n\n\n\n
Avoid Kafka REST API for:<\/p>\n\n\n\n
Very high-throughput streaming\nHeavy consumer group workloads\nKafka Streams\nExactly-once processing\nLarge-scale event pipelines\nLong-running low-latency consumers\n<\/code><\/pre>\n\n\n\n
For those, use the native Kafka protocol.<\/p>\n\n\n\n
\n\n\n\n
4. Method 3 \u2014 Confluent Cloud Management API<\/h1>\n\n\n\n
This is not the normal data streaming path<\/strong>.<\/p>\n\n\n\n
It is used to manage Confluent resources.<\/p>\n\n\n\n
Terraform \/ CI\/CD \/ Platform Automation\n        |\n        | HTTPS\n        | Cloud API key\/secret\n        v\nConfluent Cloud Management API\n<\/code><\/pre>\n\n\n\n
Use this for:<\/p>\n\n\n\n
Create environments\nCreate Kafka clusters\nCreate service accounts\nCreate API keys\nCreate topics\nManage ACLs\nManage RBAC\nManage connectors\nManage networking\nFetch metrics\n<\/code><\/pre>\n\n\n\n
Important distinction:<\/p>\n\n\n\n
API key type<\/th>Used for<\/th><\/tr><\/thead>
Cloud API key<\/strong><\/td>Confluent Cloud management APIs, provisioning, metrics<\/td><\/tr>
Kafka API key<\/strong><\/td>Produce\/consume\/admin against one Kafka cluster<\/td><\/tr>
Schema Registry API key<\/strong><\/td>Schema Registry access<\/td><\/tr>
ksqlDB API key<\/strong><\/td>ksqlDB application access<\/td><\/tr>
Flink API key<\/strong><\/td>Flink region access<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Confluent documents this distinction clearly: Cloud API keys are for management APIs, while resource-specific API keys are for Kafka clusters, Schema Registry, Flink, or ksqlDB. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Terraform provider example<\/h2>\n\n\n\n
terraform {\n  required_providers {\n    confluent = {\n      source  = \"confluentinc\/confluent\"\n      version = \"~> 2.0\"\n    }\n  }\n}\n\nprovider \"confluent\" {\n  cloud_api_key    = var.confluent_cloud_api_key\n  cloud_api_secret = var.confluent_cloud_api_secret\n}\n<\/code><\/pre>\n\n\n\n
Use Terraform for platform\/infrastructure resources. Do not<\/strong> use Terraform as the application data producer\/consumer.<\/p>\n\n\n\n
\n\n\n\n
5. Method 4 \u2014 Confluent CLI<\/h1>\n\n\n\n
This is mostly for humans, testing, debugging, and admin tasks.<\/p>\n\n\n\n
Engineer laptop \/ Bastion \/ CI job\n        |\n        | CLI\n        v\nConfluent Cloud\n<\/code><\/pre>\n\n\n\n
Common commands<\/h2>\n\n\n\n
Login:<\/p>\n\n\n\n
confluent login\n<\/code><\/pre>\n\n\n\n
List environments:<\/p>\n\n\n\n
confluent environment list\n<\/code><\/pre>\n\n\n\n
Select environment:<\/p>\n\n\n\n
confluent environment use <env-id>\n<\/code><\/pre>\n\n\n\n
List Kafka clusters:<\/p>\n\n\n\n
confluent kafka cluster list\n<\/code><\/pre>\n\n\n\n
Use a Kafka cluster:<\/p>\n\n\n\n
confluent kafka cluster use <lkc-id>\n<\/code><\/pre>\n\n\n\n
Create topic:<\/p>\n\n\n\n
confluent kafka topic create orders --partitions 6\n<\/code><\/pre>\n\n\n\n
Produce test messages:<\/p>\n\n\n\n
confluent kafka topic produce orders\n<\/code><\/pre>\n\n\n\n
Consume test messages:<\/p>\n\n\n\n
confluent kafka topic consume orders --from-beginning\n<\/code><\/pre>\n\n\n\n
Confluent\u2019s CLI tutorial shows creating API keys, creating a topic, producing messages, and consuming them from the beginning. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
6. Method 5 \u2014 Managed connectors<\/h1>\n\n\n\n
This is an indirect Kafka connection<\/strong>.<\/p>\n\n\n\n
Your application does not connect to Kafka. A connector does it for you.<\/p>\n\n\n\n
flowchart LR\n    A[Database \/ S3 \/ Salesforce \/ API \/ Snowflake] --> B[Confluent Connector]\n    B --> C[Kafka Topic]\n    C --> D[Consumer Application]\n\n    E[Kafka Topic] --> F[Sink Connector]\n    F --> G[Data Warehouse \/ Search \/ Storage \/ External API]\n<\/code><\/pre>\n\n\n\n
Source connector<\/h2>\n\n\n\n
Writes data into Kafka.<\/p>\n\n\n\n
Examples:<\/p>\n\n\n\n
PostgreSQL \u2192 Kafka\nMySQL CDC \u2192 Kafka\nS3 \u2192 Kafka\nHTTP API \u2192 Kafka\nSalesforce \u2192 Kafka\n<\/code><\/pre>\n\n\n\n
Sink connector<\/h2>\n\n\n\n
Reads data from Kafka.<\/p>\n\n\n\n
Examples:<\/p>\n\n\n\n
Kafka \u2192 Snowflake\nKafka \u2192 Elasticsearch\nKafka \u2192 S3\nKafka \u2192 BigQuery\nKafka \u2192 HTTP endpoint\n<\/code><\/pre>\n\n\n\n
Authentication<\/h2>\n\n\n\n
A connector needs Kafka permissions. Confluent says connector configuration must include either an API key\/secret or a service account ID, and connector service accounts require ACLs such as DESCRIBE, CREATE, READ, and WRITE depending on the connector type. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Example service account setup:<\/p>\n\n\n\n
confluent iam service-account create orders-sink-connector \\\n  --description \"Service account for Orders sink connector\"\n<\/code><\/pre>\n\n\n\n
Create Kafka API key for connector service account:<\/p>\n\n\n\n
confluent api-key create \\\n  --resource lkc-xxxxx \\\n  --service-account sa-xxxxx\n<\/code><\/pre>\n\n\n\n
Sink connector ACL example:<\/p>\n\n\n\n
confluent kafka acl create \\\n  --allow \\\n  --service-account sa-xxxxx \\\n  --operations read \\\n  --topic orders\n<\/code><\/pre>\n\n\n\n
confluent kafka acl create \\\n  --allow \\\n  --service-account sa-xxxxx \\\n  --operations read \\\n  --consumer-group connect-lcc-\n<\/code><\/pre>\n\n\n\n
Use connectors when you want to avoid writing custom producer\/consumer code.<\/p>\n\n\n\n
\n\n\n\n
7. Method 6 \u2014 Schema Registry access<\/h1>\n\n\n\n
Schema Registry is separate from Kafka broker access.<\/p>\n\n\n\n
You need it when using:<\/p>\n\n\n\n
Avro\nProtobuf\nJSON Schema\nSchema compatibility rules\nSchema evolution\nContract-first event design\n<\/code><\/pre>\n\n\n\n
flowchart LR\n    A[Producer App] --> B[Schema Registry]\n    A --> C[Kafka Topic]\n    C --> D[Consumer App]\n    D --> B\n<\/code><\/pre>\n\n\n\n
Kafka client config:<\/p>\n\n\n\n
bootstrap.servers=pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\nsecurity.protocol=SASL_SSL\nsasl.mechanism=PLAIN\nsasl.username=<KAFKA_API_KEY>\nsasl.password=<KAFKA_API_SECRET>\n<\/code><\/pre>\n\n\n\n
Schema Registry config:<\/p>\n\n\n\n
schema.registry.url=https:\/\/psrc-xxxxx.ap-northeast-1.aws.confluent.cloud\nbasic.auth.credentials.source=USER_INFO\nbasic.auth.user.info=<SCHEMA_REGISTRY_API_KEY>:<SCHEMA_REGISTRY_API_SECRET>\n<\/code><\/pre>\n\n\n\n
Confluent Cloud provides one Schema Registry per environment, and you commonly need one API key\/secret pair for the Kafka cluster and another for Schema Registry. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
8. Method 7 \u2014 Cluster Linking<\/h1>\n\n\n\n
Cluster Linking connects Kafka cluster to Kafka cluster.<\/p>\n\n\n\n
flowchart LR\n    A[Source Kafka Cluster] -->|Cluster Link| B[Destination Kafka Cluster]\n    B --> C[Mirror Topics]\n    B --> D[Consumers]\n<\/code><\/pre>\n\n\n\n
Use it for:<\/p>\n\n\n\n
Disaster recovery\nRegion-to-region replication\nCloud migration\nHybrid Kafka to Confluent Cloud migration\nMulti-cloud event sharing\nLow-downtime Kafka migration\n<\/code><\/pre>\n\n\n\n
Confluent describes Cluster Linking as a way to mirror data directly between clusters across regions, clouds, lines of business, or organizations, with support for mirroring topics, consumer offsets, and ACLs. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Example:<\/p>\n\n\n\n
confluent kafka link create tokyo-sydney \\\n  --source-bootstrap-server pkc-source.ap-northeast-1.aws.confluent.cloud:9092 \\\n  --source-cluster lkc-source \\\n  --source-api-key <SOURCE_API_KEY> \\\n  --source-api-secret <SOURCE_API_SECRET>\n<\/code><\/pre>\n\n\n\n
Create mirror topic:<\/p>\n\n\n\n
confluent kafka mirror create orders.tokyo \\\n  --link tokyo-sydney\n<\/code><\/pre>\n\n\n\n
For DR, Confluent recommends creating the DR cluster in another region\/cloud, enabling consumer offset sync and ACL sync, and creating mirror topics for the primary topics. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
9. Authentication methods<\/h1>\n\n\n\n
9.1 Kafka API key and secret<\/h2>\n\n\n\n
This is the most common method.<\/p>\n\n\n\n
AuthN: Kafka API key + secret\nProtocol: SASL_SSL\nMechanism: PLAIN\nAuthorization: ACLs or RBAC\n<\/code><\/pre>\n\n\n\n
Best for:<\/p>\n\n\n\n
Standard applications\nEKS workloads\nKafka clients\nSimple production setup\nService-to-service access\n<\/code><\/pre>\n\n\n\n
Recommended pattern:<\/p>\n\n\n\n
One service account per app\nOne API key per app\/environment\nACL\/RBAC least privilege\nStore secret in AWS Secrets Manager\nSync to Kubernetes using External Secrets Operator\nRotate regularly\n<\/code><\/pre>\n\n\n\n
Confluent recommends service account API keys for production and avoiding user account API keys except for development\/testing. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
9.2 OAuth \/ OIDC<\/h2>\n\n\n\n
This avoids long-lived Kafka API secrets.<\/p>\n\n\n\n
AuthN: OAuth token\nProtocol: SASL_SSL\nMechanism: OAUTHBEARER\nAuthorization: RBAC \/ ACL through identity pool\n<\/code><\/pre>\n\n\n\n
Example config:<\/p>\n\n\n\n
security.protocol=SASL_SSL\nsasl.mechanism=OAUTHBEARER\nsasl.oauthbearer.token.endpoint.url=https:\/\/idp.example.com\/oauth2\/token\nsasl.oauthbearer.client.id=orders-service\nsasl.oauthbearer.client.secret=<client-secret>\nsasl.oauthbearer.scope=kafka:read kafka:write\nsasl.oauthbearer.extensions=logicalCluster=lkc-xxxxx,identityPoolId=pool-xxxxx\n<\/code><\/pre>\n\n\n\n
Confluent Cloud supports OAuth\/OIDC for Kafka clients and documents common parameters such as sasl.mechanism=OAUTHBEARER<\/code>, token endpoint URL, client ID, client secret, scope, and extensions for logical cluster and identity pool. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Best for:<\/p>\n\n\n\n
Enterprise identity integration\nCentralized IdP\nShort-lived credentials\nLarge organization governance\nZero-trust style workload identity\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
9.3 mTLS<\/h2>\n\n\n\n
This uses client certificates.<\/p>\n\n\n\n
AuthN: Client certificate\nProtocol: TLS \/ mTLS\nAuthorization: RBAC \/ ACL through certificate identity pool\n<\/code><\/pre>\n\n\n\n
Confluent Cloud supports mTLS authentication by letting you upload your own Certificate Authority; Confluent brokers validate client certificates against that CA. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Best for:<\/p>\n\n\n\n
Strong machine identity\nCertificate-based enterprise security\nRegulated environments\nPKI-based organizations\nLong-lived infrastructure services\n<\/code><\/pre>\n\n\n\n
mTLS is powerful, but operationally heavier because you must manage:<\/p>\n\n\n\n
CA lifecycle\nClient certificate issuance\nCertificate rotation\nRevocation\nIdentity pool mapping\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
10. Authorization methods<\/h1>\n\n\n\n
Authentication answers:<\/p>\n\n\n\n
Who are you?\n<\/code><\/pre>\n\n\n\n
Authorization answers:<\/p>\n\n\n\n
What are you allowed to do?\n<\/code><\/pre>\n\n\n\n
Confluent supports both ACLs and RBAC. Service accounts represent applications that access Confluent Cloud programmatically, and permissions can be assigned through ACLs and role bindings. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
10.1 ACLs<\/h2>\n\n\n\n
ACLs are Kafka-style permissions.<\/p>\n\n\n\n
Examples:<\/p>\n\n\n\n
Allow service account to WRITE to topic orders\nAllow service account to READ from topic orders\nAllow service account to READ consumer group orders-service\nAllow service account to DESCRIBE topic\n<\/code><\/pre>\n\n\n\n
Confluent describes ACLs as a way to secure access to Kafka resources and data; a principal only has permissions for resources granted to it. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Producer ACL example:<\/p>\n\n\n\n
confluent kafka acl create \\\n  --allow \\\n  --service-account sa-orders-prod \\\n  --operations write,describe \\\n  --topic orders\n<\/code><\/pre>\n\n\n\n
Consumer ACL example:<\/p>\n\n\n\n
confluent kafka acl create \\\n  --allow \\\n  --service-account sa-orders-consumer-prod \\\n  --operations read,describe \\\n  --topic orders\n<\/code><\/pre>\n\n\n\n
confluent kafka acl create \\\n  --allow \\\n  --service-account sa-orders-consumer-prod \\\n  --operations read \\\n  --consumer-group orders-consumer-group\n<\/code><\/pre>\n\n\n\n
10.2 RBAC<\/h2>\n\n\n\n
RBAC is higher-level and role-based.<\/p>\n\n\n\n
Examples:<\/p>\n\n\n\n
DeveloperRead\nDeveloperWrite\nDeveloperManage\nCloudClusterAdmin\nEnvironmentAdmin\nOrganizationAdmin\nMetricsViewer\nResourceOwner\n<\/code><\/pre>\n\n\n\n
Confluent Cloud predefined RBAC roles can be scoped to organizations, environments, clusters, and Kafka resources, and roles such as DeveloperRead, DeveloperWrite, DeveloperManage, CloudClusterAdmin, EnvironmentAdmin, and OrganizationAdmin are documented. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Best practice:<\/p>\n\n\n\n
Use RBAC for human\/platform roles.\nUse ACLs or resource-scoped RBAC for application data-plane access.\nDo not give OrganizationAdmin to apps.\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
11. Network connectivity methods<\/h1>\n\n\n\n
11.1 Public internet endpoint<\/h2>\n\n\n\n
EKS Pod \/ App \/ Laptop\n        |\n        | Internet\n        v\nConfluent Cloud public Kafka endpoint\n<\/code><\/pre>\n\n\n\n
Use cases:<\/p>\n\n\n\n
Development\nSimple production\nLow security restriction environments\nQuick onboarding\n<\/code><\/pre>\n\n\n\n
Pros:<\/p>\n\n\n\n
Easy\nFast setup\nNo private network complexity\nWorks from anywhere with credentials\n<\/code><\/pre>\n\n\n\n
Cons:<\/p>\n\n\n\n
Internet-reachable endpoint\nDepends fully on auth\/RBAC\/ACLs\nMay not satisfy strict enterprise security\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
11.2 Public endpoint + IP filtering<\/h2>\n\n\n\n
Confluent IP Filtering allows inbound requests only from trusted source networks using IP groups\/CIDR blocks; by default, publicly networked Confluent resources are reachable from any source IP. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
EKS NAT Gateway static IP\n        |\n        | Allowed CIDR only\n        v\nConfluent Cloud public endpoint\n<\/code><\/pre>\n\n\n\n
Use cases:<\/p>\n\n\n\n
Public endpoint acceptable\nSource NAT IP is stable\nSmall number of known egress IPs\n<\/code><\/pre>\n\n\n\n
For EKS, this usually means:<\/p>\n\n\n\n
Private subnets\nNAT Gateway with Elastic IP\nConfluent IP filter allows NAT EIP\n<\/code><\/pre>\n\n\n\n
Limitation:<\/p>\n\n\n\n
If clients come from dynamic IPs, IP filtering becomes painful.\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
11.3 AWS PrivateLink<\/h2>\n\n\n\n
flowchart LR\n    A[EKS Pods in AWS VPC] --> B[AWS Interface VPC Endpoint]\n    B --> C[AWS PrivateLink]\n    C --> D[Confluent Cloud Kafka Cluster]\n<\/code><\/pre>\n\n\n\n
This is often the best enterprise design for AWS + EKS.<\/p>\n\n\n\n
Confluent says AWS PrivateLink provides one-way secure connection access from your VPC to Confluent Cloud and adds protection against data exfiltration. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Use cases:<\/p>\n\n\n\n
Production EKS workloads\nNo public Kafka endpoint\nStrict security\/compliance\nPrivate traffic path\nFinancial\/enterprise workloads\n<\/code><\/pre>\n\n\n\n
Pros:<\/p>\n\n\n\n
Private network path\nNo public internet exposure\nStrong security posture\nGood for EKS\n<\/code><\/pre>\n\n\n\n
Cons:<\/p>\n\n\n\n
More setup\nDNS configuration required\nUsually cluster-type\/plan dependent\nNeed VPC endpoint management\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
11.4 VPC peering \/ VNet peering<\/h2>\n\n\n\n
Your VPC <---- Peering ----> Confluent Cloud network\n<\/code><\/pre>\n\n\n\n
Use cases:<\/p>\n\n\n\n
Private routing\nLarge internal network\nOlder private connectivity patterns\n<\/code><\/pre>\n\n\n\n
Pros:<\/p>\n\n\n\n
Private connectivity\nNative routing\n<\/code><\/pre>\n\n\n\n
Cons:<\/p>\n\n\n\n
CIDR overlap issues\nRouting complexity\nTransitive routing limitations\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
11.5 AWS Transit Gateway<\/h2>\n\n\n\n
Multiple VPCs\n    |\n    v\nAWS Transit Gateway\n    |\n    v\nConfluent Cloud\n<\/code><\/pre>\n\n\n\n
Confluent Cloud supports private connectivity options including AWS PrivateLink, Azure Private Link, VPC\/VNet peering, and AWS Transit Gateway. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Use cases:<\/p>\n\n\n\n
Many VPCs\nCentralized enterprise networking\nShared services VPC\nMulti-account AWS setup\n<\/code><\/pre>\n\n\n\n
Best when many EKS clusters or AWS accounts need Kafka access.<\/p>\n\n\n\n
\n\n\n\n
11.6 Private Network Interface<\/h2>\n\n\n\n
Confluent Private Network Interface enables private communication between your AWS resources and Confluent Cloud clusters through Elastic Network Interfaces in your AWS account. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Use cases:<\/p>\n\n\n\n
AWS-native private access\nEnterprise\/Freight cluster designs\nPrivate networking with ENI model\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
12. Ports and firewall requirements<\/h1>\n\n\n\n
Very important:<\/p>\n\n\n\n
Endpoint<\/th>Port<\/th>Protocol<\/th><\/tr><\/thead>
Kafka bootstrap<\/td>9092<\/code><\/td>SASL_SSL or mTLS<\/td><\/tr>
Kafka REST endpoint<\/td>443<\/code><\/td>HTTPS<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Confluent\u2019s connectivity testing guide says Kafka clusters use port 9092<\/code> for bootstrap and port 443<\/code> for Kafka REST, and that Terraform Provider \/ Kafka REST require access to port 443<\/code>. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
Test Kafka bootstrap:<\/p>\n\n\n\n
nc -zv pkc-xxxxx.ap-northeast-1.aws.confluent.cloud 9092\n<\/code><\/pre>\n\n\n\n
Test REST endpoint:<\/p>\n\n\n\n
nc -zv pkc-xxxxx.ap-northeast-1.aws.confluent.cloud 443\n<\/code><\/pre>\n\n\n\n
Test TLS\/SNI:<\/p>\n\n\n\n
openssl s_client \\\n  -servername pkc-xxxxx.ap-northeast-1.aws.confluent.cloud \\\n  -connect pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\n<\/code><\/pre>\n\n\n\n
Confluent notes that Kafka broker hosts do not respond to ping<\/code>, so use tools like nc<\/code>, openssl<\/code>, or telnet<\/code> instead. (Confluent Documentation<\/a>)<\/p>\n\n\n\n
\n\n\n\n
13. EKS recommended architecture<\/h1>\n\n\n\n
For your EKS case, I would recommend this order.<\/p>\n\n\n\n
Short-term practical design<\/h2>\n\n\n\n
flowchart LR\n    A[EKS Pod] --> B[NAT Gateway Elastic IP]\n    B --> C[Confluent Public Endpoint]\n    C --> D[Kafka Cluster]\n\n    E[AWS Secrets Manager] --> F[External Secrets Operator]\n    F --> G[Kubernetes Secret]\n    G --> A\n<\/code><\/pre>\n\n\n\n
Use:<\/p>\n\n\n\n
Native Kafka client\nSASL_SSL\nKafka API key\/secret\nService account per app\nACL least privilege\nNAT Gateway static EIP\nConfluent IP Filtering if possible\nSecrets stored in AWS Secrets Manager\nExternal Secrets Operator sync to Kubernetes\n<\/code><\/pre>\n\n\n\n
Best enterprise design<\/h2>\n\n\n\n
flowchart LR\n    A[EKS Private Subnet Pod] --> B[AWS PrivateLink \/ VPC Endpoint]\n    B --> C[Confluent Private Endpoint]\n    C --> D[Confluent Kafka Cluster]\n\n    E[Terraform Cloud Agent in VPC] --> B\n    F[Admin Bastion \/ VPN] --> B\n<\/code><\/pre>\n\n\n\n
Use:<\/p>\n\n\n\n
PrivateLink\nNative Kafka protocol\nService account API key OR OAuth\/OIDC OR mTLS\nACL\/RBAC least privilege\nTerraform Cloud Agent inside VPC if Terraform must reach private resources\n<\/code><\/pre>\n\n\n\n
This matches your earlier EKS API endpoint discussion: if workloads\/tools must reach private endpoints, run the automation agent inside the private network.<\/p>\n\n\n\n
\n\n\n\n
14. Kubernetes example for EKS app<\/h1>\n\n\n\n
Secret<\/h2>\n\n\n\n
apiVersion: v1\nkind: Secret\nmetadata:\n  name: confluent-kafka-orders\n  namespace: orders-prod\ntype: Opaque\nstringData:\n  KAFKA_BOOTSTRAP_SERVERS: \"pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092\"\n  KAFKA_API_KEY: \"<api-key>\"\n  KAFKA_API_SECRET: \"<api-secret>\"\n<\/code><\/pre>\n\n\n\n
Deployment env vars<\/h2>\n\n\n\n
apiVersion: apps\/v1\nkind: Deployment\nmetadata:\n  name: orders-service\n  namespace: orders-prod\nspec:\n  replicas: 3\n  selector:\n    matchLabels:\n      app: orders-service\n  template:\n    metadata:\n      labels:\n        app: orders-service\n    spec:\n      containers:\n        - name: orders-service\n          image: myrepo\/orders-service:1.0.0\n          env:\n            - name: KAFKA_BOOTSTRAP_SERVERS\n              valueFrom:\n                secretKeyRef:\n                  name: confluent-kafka-orders\n                  key: KAFKA_BOOTSTRAP_SERVERS\n            - name: KAFKA_API_KEY\n              valueFrom:\n                secretKeyRef:\n                  name: confluent-kafka-orders\n                  key: KAFKA_API_KEY\n            - name: KAFKA_API_SECRET\n              valueFrom:\n                secretKeyRef:\n                  name: confluent-kafka-orders\n                  key: KAFKA_API_SECRET\n<\/code><\/pre>\n\n\n\n
Production improvement:<\/p>\n\n\n\n
Do not manually create Kubernetes Secrets.\nStore secret in AWS Secrets Manager.\nSync using External Secrets Operator.\nRotate Confluent API key periodically.\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
15. Which method to use when?<\/h1>\n\n\n\n
Requirement<\/th>Best method<\/th><\/tr><\/thead>
EKS microservice produces\/consumes events<\/td>Native Kafka protocol<\/td><\/tr>
High throughput<\/td>Native Kafka protocol<\/td><\/tr>
Long-running consumer group<\/td>Native Kafka protocol<\/td><\/tr>
Serverless function sends occasional events<\/td>Kafka REST API<\/td><\/tr>
Shell script sends test event<\/td>Kafka REST API or CLI<\/td><\/tr>
CI\/CD creates topics\/ACLs<\/td>Terraform Provider or Kafka REST Admin API<\/td><\/tr>
Platform team provisions Confluent resources<\/td>Terraform + Confluent Cloud API<\/td><\/tr>
Human debugging<\/td>Confluent CLI<\/td><\/tr>
Database to Kafka<\/td>Managed source connector<\/td><\/tr>
Kafka to Snowflake\/S3\/Elastic<\/td>Managed sink connector<\/td><\/tr>
Multi-region replication<\/td>Cluster Linking<\/td><\/tr>
Migration from old Kafka to new Kafka<\/td>Cluster Linking<\/td><\/tr>
Schema governance<\/td>Schema Registry<\/td><\/tr>
Enterprise identity, no long-lived Kafka secrets<\/td>OAuth\/OIDC<\/td><\/tr>
Certificate-based machine identity<\/td>mTLS<\/td><\/tr>
Strict private AWS access<\/td>AWS PrivateLink<\/td><\/tr>
Many AWS VPCs\/accounts<\/td>AWS Transit Gateway<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n
16. Recommended standard for your organization<\/h1>\n\n\n\n
For a production EKS + Confluent Cloud setup, I would define standards like this:<\/p>\n\n\n\n
Application standard<\/h2>\n\n\n\n
Connection method:\nNative Kafka protocol\n\nNetwork:\nPrivateLink preferred\nPublic + IP Filtering acceptable as interim\n\nAuthentication:\nService account Kafka API key\/secret initially\nOAuth\/OIDC or mTLS for mature enterprise identity\n\nAuthorization:\nLeast-privilege ACLs or scoped RBAC\n\nSecrets:\nAWS Secrets Manager + External Secrets Operator\n\nObservability:\nConsumer lag, delivery error rate, auth failures, broker connection errors\n<\/code><\/pre>\n\n\n\n
Platform\/IaC standard<\/h2>\n\n\n\n
Tool:\nTerraform Provider for Confluent\n\nAuthentication:\nCloud API key\/secret stored in Terraform Cloud sensitive variables or Vault\n\nNetwork:\nTerraform execution must have access to Confluent Cloud API and Kafka REST endpoint if managing Kafka resources\n\nOwnership:\nPlatform team owns environments, clusters, networking, service accounts, ACL\/RBAC patterns\nApplication teams own topic usage and client configuration\n<\/code><\/pre>\n\n\n\n
Human\/debug standard<\/h2>\n\n\n\n
Tool:\nConfluent CLI or kcat\n\nAuthentication:\nTemporary\/user-scoped access where possible\n\nAccess:\nVia VPN\/bastion\/private network if cluster is private\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
17. Simple final mental model<\/h1>\n\n\n\n
Native Kafka Protocol\n= Best for applications and real streaming.\n\nKafka REST API\n= Best for HTTPS-only, serverless, scripts, light integrations.\n\nConfluent Cloud Management API\n= Best for provisioning and automation.\n\nConfluent CLI\n= Best for humans, debugging, and testing.\n\nManaged Connectors\n= Best when moving data between Kafka and external systems.\n\nSchema Registry\n= Best for governed Avro\/Protobuf\/JSON Schema events.\n\nCluster Linking\n= Best for Kafka-to-Kafka replication, migration, DR, and multi-region.\n<\/code><\/pre>\n\n\n\n
My recommended answer for your team:<\/p>\n\n\n\n
\n
For application workloads on EKS, the preferred method is direct Kafka data-plane access using the native Kafka protocol over SASL_SSL or mTLS. The application should authenticate using a Confluent service-account-based Kafka API key\/secret, OAuth\/OIDC, or mTLS, and should be authorized using least-privilege ACLs or scoped RBAC.<\/p>\n\n\n\n
Confluent Cloud APIs and Terraform should be used for control-plane automation such as creating clusters, topics, service accounts, API keys, ACLs, RBAC bindings, and networking. Kafka REST API is useful for HTTPS-based integrations, serverless workloads, scripts, and CI\/CD cases where native Kafka clients are not practical.<\/p>\n\n\n\n
For production EKS, the strongest network design is AWS PrivateLink between the EKS VPC and Confluent Cloud, with secrets managed through AWS Secrets Manager and synced into Kubernetes through External Secrets Operator.<\/p>\n<\/blockquote>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
1. First understand the four layers Confluent Cloud supports native Kafka clients in many languages, including Java, Python, Go, JavaScript, .NET, C\/C++, and others. For normal producer\/consumer… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2973","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\nKafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School\" \/>\n<meta property=\"og:description\" content=\"1. First understand the four layers Confluent Cloud supports native Kafka clients in many languages, including Java, Python, Go, JavaScript, .NET, C\/C++, and others. For normal producer\/consumer...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/\" \/>\n<meta property=\"og:site_name\" content=\"SRE School\" \/>\n<meta property=\"article:published_time\" content=\"2026-06-16T01:06:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-06-16T01:06:56+00:00\" \/>\n<meta name=\"author\" content=\"Rajesh Kumar\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rajesh Kumar\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/\"},\"author\":{\"name\":\"Rajesh Kumar\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/#\\\/schema\\\/person\\\/0ffe446f77bb2589992dbe3a7f417201\"},\"headline\":\"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka\",\"datePublished\":\"2026-06-16T01:06:55+00:00\",\"dateModified\":\"2026-06-16T01:06:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/\"},\"wordCount\":1555,\"commentCount\":0,\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/\",\"url\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/\",\"name\":\"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/#website\"},\"datePublished\":\"2026-06-16T01:06:55+00:00\",\"dateModified\":\"2026-06-16T01:06:56+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/#\\\/schema\\\/person\\\/0ffe446f77bb2589992dbe3a7f417201\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/\",\"name\":\"SRESchool\",\"description\":\"Master SRE. Build Resilient Systems. Lead the Future of Reliability\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/#\\\/schema\\\/person\\\/0ffe446f77bb2589992dbe3a7f417201\",\"name\":\"Rajesh Kumar\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g\",\"caption\":\"Rajesh Kumar\"},\"sameAs\":[\"http:\\\/\\\/sreschool.com\\\/blog\"],\"url\":\"https:\\\/\\\/sreschool.com\\\/blog\\\/author\\\/admin\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/","og_locale":"en_US","og_type":"article","og_title":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School","og_description":"1. First understand the four layers Confluent Cloud supports native Kafka clients in many languages, including Java, Python, Go, JavaScript, .NET, C\/C++, and others. For normal producer\/consumer...","og_url":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/","og_site_name":"SRE School","article_published_time":"2026-06-16T01:06:55+00:00","article_modified_time":"2026-06-16T01:06:56+00:00","author":"Rajesh Kumar","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rajesh Kumar","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/#article","isPartOf":{"@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/"},"author":{"name":"Rajesh Kumar","@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201"},"headline":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka","datePublished":"2026-06-16T01:06:55+00:00","dateModified":"2026-06-16T01:06:56+00:00","mainEntityOfPage":{"@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/"},"wordCount":1555,"commentCount":0,"inLanguage":"en","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/","url":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/","name":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka - SRE School","isPartOf":{"@id":"https:\/\/sreschool.com\/blog\/#website"},"datePublished":"2026-06-16T01:06:55+00:00","dateModified":"2026-06-16T01:06:56+00:00","author":{"@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201"},"breadcrumb":{"@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sreschool.com\/blog\/kafka-complete-guide-ways-to-connect-authenticate-and-use-confluent-kafka\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sreschool.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Kafka Complete Guide: Ways to Connect, Authenticate, and Use Confluent Kafka"}]},{"@type":"WebSite","@id":"https:\/\/sreschool.com\/blog\/#website","url":"https:\/\/sreschool.com\/blog\/","name":"SRESchool","description":"Master SRE. Build Resilient Systems. Lead the Future of Reliability","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sreschool.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en"},{"@type":"Person","@id":"https:\/\/sreschool.com\/blog\/#\/schema\/person\/0ffe446f77bb2589992dbe3a7f417201","name":"Rajesh Kumar","image":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f901a4f2929fa034a291a8363d589791d5a3c1f6a051c22e744acb8bfc8e022a?s=96&d=mm&r=g","caption":"Rajesh Kumar"},"sameAs":["http:\/\/sreschool.com\/blog"],"url":"https:\/\/sreschool.com\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/2973","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/comments?post=2973"}],"version-history":[{"count":1,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/2973\/revisions"}],"predecessor-version":[{"id":2974,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/posts\/2973\/revisions\/2974"}],"wp:attachment":[{"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/media?parent=2973"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/categories?post=2973"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sreschool.com\/blog\/wp-json\/wp\/v2\/tags?post=2973"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}