1. First understand the four layers
flowchart TD
A[Application / Tool / Terraform / Connector] --> B[Connection Method]
B --> B1[Native Kafka Protocol]
B --> B2[Kafka REST API]
B --> B3[Confluent Cloud Management API]
B --> B4[Confluent CLI]
B --> B5[Managed Connectors]
B --> B6[Cluster Linking]
B1 --> C[Network Path]
B2 --> C
B3 --> C
C --> C1[Public Internet]
C --> C2[Public + IP Filtering]
C --> C3[AWS PrivateLink]
C --> C4[VPC Peering]
C --> C5[AWS Transit Gateway]
C --> C6[Private Network Interface]
C --> D[Authentication]
D --> D1[Kafka API Key / Secret]
D --> D2[Cloud API Key / Secret]
D --> D3[Schema Registry API Key / Secret]
D --> D4[OAuth / OIDC]
D --> D5[mTLS Client Certificate]
D --> E[Authorization]
E --> E1[Kafka ACLs]
E --> E2[Confluent RBAC]
Confluent Cloud supports native Kafka clients in many languages, including Java, Python, Go, JavaScript, .NET, C/C++, and others. For normal producer/consumer workloads, this is usually the best and most standard path. (Confluent Documentation)
2. Method 1 — Native Kafka protocol
This is the direct Kafka access method.
Your application connects directly to the Kafka bootstrap server using the Kafka protocol.
Application / Pod / Service
|
| Kafka protocol
| SASL_SSL / mTLS
v
Confluent Kafka bootstrap endpoint :9092
Confluent Cloud Kafka clients commonly use:
bootstrap.servers=<BOOTSTRAP_SERVER>
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.username=<KAFKA_API_KEY>
sasl.password=<KAFKA_API_SECRET>
The official Confluent Java example uses bootstrap.servers, security.protocol=SASL_SSL, sasl.mechanism=PLAIN, and API key/secret in the JAAS config. (Confluent Documentation)
Use cases
Use this for:
| Use case | Recommended? |
|---|---|
| Microservices producing/consuming events | Yes |
| EKS workloads | Yes |
| High-throughput streaming | Yes |
| Long-running consumers | Yes |
| Consumer groups | Yes |
| Exactly-once / transactions | Yes |
| Kafka Streams | Yes |
| Low-latency event processing | Yes |
Example: Java client config
bootstrap.servers=pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="KAFKA_API_KEY" password="KAFKA_API_SECRET";
client.id=orders-service
Example: Python client
from confluent_kafka import Producer
conf = {
"bootstrap.servers": "pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092",
"security.protocol": "SASL_SSL",
"sasl.mechanism": "PLAIN",
"sasl.username": "KAFKA_API_KEY",
"sasl.password": "KAFKA_API_SECRET",
"client.id": "orders-producer"
}
producer = Producer(conf)
producer.produce(
"orders",
key="order-1001",
value='{"order_id": "1001", "status": "CREATED"}'
)
producer.flush()
Example: Node.js / JavaScript
const { Kafka } = require("@confluentinc/kafka-javascript");
const kafka = new Kafka({
kafkaJS: {
brokers: ["pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092"],
ssl: true,
sasl: {
mechanism: "plain",
username: process.env.KAFKA_API_KEY,
password: process.env.KAFKA_API_SECRET
}
}
});
const producer = kafka.producer();
Confluent’s JavaScript example also uses brokers, SSL, SASL plain, API key, and API secret. (Confluent Documentation)
3. Method 2 — Kafka REST API
This is Kafka over HTTPS.
Instead of using a native Kafka client, you call an HTTPS endpoint.
Application / Script / Serverless Function
|
| HTTPS REST API
| Basic auth using API key/secret
v
Confluent Kafka REST endpoint :443
Confluent Cloud exposes a Kafka REST endpoint per cluster, usually on port 443, while the native Kafka bootstrap server uses port 9092. (Confluent Documentation)
Use cases
Use Kafka REST API when:
| Use case | Why REST helps |
|---|---|
| Serverless functions | Easier than keeping Kafka connections |
| Shell scripts | Simple curl works |
| Third-party integration | Partner does not need Kafka client |
| Restricted firewall | HTTPS 443 may be easier |
| Low-throughput producers | Simple and acceptable |
| CI/CD automation | Create/manage topics or produce test events |
Confluent says the Kafka REST API is useful when native Kafka clients are not available, such as serverless functions, CI/CD pipelines, shell scripts, or restricted environments where only HTTPS is allowed. (Confluent Documentation)
REST API produce example
First encode your Kafka API key and secret:
echo -n "KAFKA_API_KEY:KAFKA_API_SECRET" | base64
Then produce a record:
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Basic <BASE64_API_KEY_AND_SECRET>" \
"https://pkc-xxxxx.ap-northeast-1.aws.confluent.cloud/kafka/v3/clusters/lkc-xxxxx/topics/orders/records" \
-d '{
"key": {
"type": "STRING",
"data": "order-1001"
},
"value": {
"type": "JSON",
"data": {
"order_id": "1001",
"status": "CREATED"
}
}
}'
Confluent’s REST API examples use Authorization: Basic <BASE64-encoded-key-and-secret> and the /kafka/v3/clusters/<cluster-id>/topics/<topic-name>/records endpoint. (Confluent Documentation)
When not to use REST
Avoid Kafka REST API for:
Very high-throughput streaming
Heavy consumer group workloads
Kafka Streams
Exactly-once processing
Large-scale event pipelines
Long-running low-latency consumers
For those, use the native Kafka protocol.
4. Method 3 — Confluent Cloud Management API
This is not the normal data streaming path.
It is used to manage Confluent resources.
Terraform / CI/CD / Platform Automation
|
| HTTPS
| Cloud API key/secret
v
Confluent Cloud Management API
Use this for:
Create environments
Create Kafka clusters
Create service accounts
Create API keys
Create topics
Manage ACLs
Manage RBAC
Manage connectors
Manage networking
Fetch metrics
Important distinction:
| API key type | Used for |
|---|---|
| Cloud API key | Confluent Cloud management APIs, provisioning, metrics |
| Kafka API key | Produce/consume/admin against one Kafka cluster |
| Schema Registry API key | Schema Registry access |
| ksqlDB API key | ksqlDB application access |
| Flink API key | Flink region access |
Confluent documents this distinction clearly: Cloud API keys are for management APIs, while resource-specific API keys are for Kafka clusters, Schema Registry, Flink, or ksqlDB. (Confluent Documentation)
Terraform provider example
terraform {
required_providers {
confluent = {
source = "confluentinc/confluent"
version = "~> 2.0"
}
}
}
provider "confluent" {
cloud_api_key = var.confluent_cloud_api_key
cloud_api_secret = var.confluent_cloud_api_secret
}
Use Terraform for platform/infrastructure resources. Do not use Terraform as the application data producer/consumer.
5. Method 4 — Confluent CLI
This is mostly for humans, testing, debugging, and admin tasks.
Engineer laptop / Bastion / CI job
|
| CLI
v
Confluent Cloud
Common commands
Login:
confluent login
List environments:
confluent environment list
Select environment:
confluent environment use <env-id>
List Kafka clusters:
confluent kafka cluster list
Use a Kafka cluster:
confluent kafka cluster use <lkc-id>
Create topic:
confluent kafka topic create orders --partitions 6
Produce test messages:
confluent kafka topic produce orders
Consume test messages:
confluent kafka topic consume orders --from-beginning
Confluent’s CLI tutorial shows creating API keys, creating a topic, producing messages, and consuming them from the beginning. (Confluent Documentation)
6. Method 5 — Managed connectors
This is an indirect Kafka connection.
Your application does not connect to Kafka. A connector does it for you.
flowchart LR
A[Database / S3 / Salesforce / API / Snowflake] --> B[Confluent Connector]
B --> C[Kafka Topic]
C --> D[Consumer Application]
E[Kafka Topic] --> F[Sink Connector]
F --> G[Data Warehouse / Search / Storage / External API]
Source connector
Writes data into Kafka.
Examples:
PostgreSQL → Kafka
MySQL CDC → Kafka
S3 → Kafka
HTTP API → Kafka
Salesforce → Kafka
Sink connector
Reads data from Kafka.
Examples:
Kafka → Snowflake
Kafka → Elasticsearch
Kafka → S3
Kafka → BigQuery
Kafka → HTTP endpoint
Authentication
A connector needs Kafka permissions. Confluent says connector configuration must include either an API key/secret or a service account ID, and connector service accounts require ACLs such as DESCRIBE, CREATE, READ, and WRITE depending on the connector type. (Confluent Documentation)
Example service account setup:
confluent iam service-account create orders-sink-connector \
--description "Service account for Orders sink connector"
Create Kafka API key for connector service account:
confluent api-key create \
--resource lkc-xxxxx \
--service-account sa-xxxxx
Sink connector ACL example:
confluent kafka acl create \
--allow \
--service-account sa-xxxxx \
--operations read \
--topic orders
confluent kafka acl create \
--allow \
--service-account sa-xxxxx \
--operations read \
--consumer-group connect-lcc-
Use connectors when you want to avoid writing custom producer/consumer code.
7. Method 6 — Schema Registry access
Schema Registry is separate from Kafka broker access.
You need it when using:
Avro
Protobuf
JSON Schema
Schema compatibility rules
Schema evolution
Contract-first event design
flowchart LR
A[Producer App] --> B[Schema Registry]
A --> C[Kafka Topic]
C --> D[Consumer App]
D --> B
Kafka client config:
bootstrap.servers=pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.username=<KAFKA_API_KEY>
sasl.password=<KAFKA_API_SECRET>
Schema Registry config:
schema.registry.url=https://psrc-xxxxx.ap-northeast-1.aws.confluent.cloud
basic.auth.credentials.source=USER_INFO
basic.auth.user.info=<SCHEMA_REGISTRY_API_KEY>:<SCHEMA_REGISTRY_API_SECRET>
Confluent Cloud provides one Schema Registry per environment, and you commonly need one API key/secret pair for the Kafka cluster and another for Schema Registry. (Confluent Documentation)
8. Method 7 — Cluster Linking
Cluster Linking connects Kafka cluster to Kafka cluster.
flowchart LR
A[Source Kafka Cluster] -->|Cluster Link| B[Destination Kafka Cluster]
B --> C[Mirror Topics]
B --> D[Consumers]
Use it for:
Disaster recovery
Region-to-region replication
Cloud migration
Hybrid Kafka to Confluent Cloud migration
Multi-cloud event sharing
Low-downtime Kafka migration
Confluent describes Cluster Linking as a way to mirror data directly between clusters across regions, clouds, lines of business, or organizations, with support for mirroring topics, consumer offsets, and ACLs. (Confluent Documentation)
Example:
confluent kafka link create tokyo-sydney \
--source-bootstrap-server pkc-source.ap-northeast-1.aws.confluent.cloud:9092 \
--source-cluster lkc-source \
--source-api-key <SOURCE_API_KEY> \
--source-api-secret <SOURCE_API_SECRET>
Create mirror topic:
confluent kafka mirror create orders.tokyo \
--link tokyo-sydney
For DR, Confluent recommends creating the DR cluster in another region/cloud, enabling consumer offset sync and ACL sync, and creating mirror topics for the primary topics. (Confluent Documentation)
9. Authentication methods
9.1 Kafka API key and secret
This is the most common method.
AuthN: Kafka API key + secret
Protocol: SASL_SSL
Mechanism: PLAIN
Authorization: ACLs or RBAC
Best for:
Standard applications
EKS workloads
Kafka clients
Simple production setup
Service-to-service access
Recommended pattern:
One service account per app
One API key per app/environment
ACL/RBAC least privilege
Store secret in AWS Secrets Manager
Sync to Kubernetes using External Secrets Operator
Rotate regularly
Confluent recommends service account API keys for production and avoiding user account API keys except for development/testing. (Confluent Documentation)
9.2 OAuth / OIDC
This avoids long-lived Kafka API secrets.
AuthN: OAuth token
Protocol: SASL_SSL
Mechanism: OAUTHBEARER
Authorization: RBAC / ACL through identity pool
Example config:
security.protocol=SASL_SSL
sasl.mechanism=OAUTHBEARER
sasl.oauthbearer.token.endpoint.url=https://idp.example.com/oauth2/token
sasl.oauthbearer.client.id=orders-service
sasl.oauthbearer.client.secret=<client-secret>
sasl.oauthbearer.scope=kafka:read kafka:write
sasl.oauthbearer.extensions=logicalCluster=lkc-xxxxx,identityPoolId=pool-xxxxx
Confluent Cloud supports OAuth/OIDC for Kafka clients and documents common parameters such as sasl.mechanism=OAUTHBEARER, token endpoint URL, client ID, client secret, scope, and extensions for logical cluster and identity pool. (Confluent Documentation)
Best for:
Enterprise identity integration
Centralized IdP
Short-lived credentials
Large organization governance
Zero-trust style workload identity
9.3 mTLS
This uses client certificates.
AuthN: Client certificate
Protocol: TLS / mTLS
Authorization: RBAC / ACL through certificate identity pool
Confluent Cloud supports mTLS authentication by letting you upload your own Certificate Authority; Confluent brokers validate client certificates against that CA. (Confluent Documentation)
Best for:
Strong machine identity
Certificate-based enterprise security
Regulated environments
PKI-based organizations
Long-lived infrastructure services
mTLS is powerful, but operationally heavier because you must manage:
CA lifecycle
Client certificate issuance
Certificate rotation
Revocation
Identity pool mapping
10. Authorization methods
Authentication answers:
Who are you?
Authorization answers:
What are you allowed to do?
Confluent supports both ACLs and RBAC. Service accounts represent applications that access Confluent Cloud programmatically, and permissions can be assigned through ACLs and role bindings. (Confluent Documentation)
10.1 ACLs
ACLs are Kafka-style permissions.
Examples:
Allow service account to WRITE to topic orders
Allow service account to READ from topic orders
Allow service account to READ consumer group orders-service
Allow service account to DESCRIBE topic
Confluent describes ACLs as a way to secure access to Kafka resources and data; a principal only has permissions for resources granted to it. (Confluent Documentation)
Producer ACL example:
confluent kafka acl create \
--allow \
--service-account sa-orders-prod \
--operations write,describe \
--topic orders
Consumer ACL example:
confluent kafka acl create \
--allow \
--service-account sa-orders-consumer-prod \
--operations read,describe \
--topic orders
confluent kafka acl create \
--allow \
--service-account sa-orders-consumer-prod \
--operations read \
--consumer-group orders-consumer-group
10.2 RBAC
RBAC is higher-level and role-based.
Examples:
DeveloperRead
DeveloperWrite
DeveloperManage
CloudClusterAdmin
EnvironmentAdmin
OrganizationAdmin
MetricsViewer
ResourceOwner
Confluent Cloud predefined RBAC roles can be scoped to organizations, environments, clusters, and Kafka resources, and roles such as DeveloperRead, DeveloperWrite, DeveloperManage, CloudClusterAdmin, EnvironmentAdmin, and OrganizationAdmin are documented. (Confluent Documentation)
Best practice:
Use RBAC for human/platform roles.
Use ACLs or resource-scoped RBAC for application data-plane access.
Do not give OrganizationAdmin to apps.
11. Network connectivity methods
11.1 Public internet endpoint
EKS Pod / App / Laptop
|
| Internet
v
Confluent Cloud public Kafka endpoint
Use cases:
Development
Simple production
Low security restriction environments
Quick onboarding
Pros:
Easy
Fast setup
No private network complexity
Works from anywhere with credentials
Cons:
Internet-reachable endpoint
Depends fully on auth/RBAC/ACLs
May not satisfy strict enterprise security
11.2 Public endpoint + IP filtering
Confluent IP Filtering allows inbound requests only from trusted source networks using IP groups/CIDR blocks; by default, publicly networked Confluent resources are reachable from any source IP. (Confluent Documentation)
EKS NAT Gateway static IP
|
| Allowed CIDR only
v
Confluent Cloud public endpoint
Use cases:
Public endpoint acceptable
Source NAT IP is stable
Small number of known egress IPs
For EKS, this usually means:
Private subnets
NAT Gateway with Elastic IP
Confluent IP filter allows NAT EIP
Limitation:
If clients come from dynamic IPs, IP filtering becomes painful.
11.3 AWS PrivateLink
flowchart LR
A[EKS Pods in AWS VPC] --> B[AWS Interface VPC Endpoint]
B --> C[AWS PrivateLink]
C --> D[Confluent Cloud Kafka Cluster]
This is often the best enterprise design for AWS + EKS.
Confluent says AWS PrivateLink provides one-way secure connection access from your VPC to Confluent Cloud and adds protection against data exfiltration. (Confluent Documentation)
Use cases:
Production EKS workloads
No public Kafka endpoint
Strict security/compliance
Private traffic path
Financial/enterprise workloads
Pros:
Private network path
No public internet exposure
Strong security posture
Good for EKS
Cons:
More setup
DNS configuration required
Usually cluster-type/plan dependent
Need VPC endpoint management
11.4 VPC peering / VNet peering
Your VPC <---- Peering ----> Confluent Cloud network
Use cases:
Private routing
Large internal network
Older private connectivity patterns
Pros:
Private connectivity
Native routing
Cons:
CIDR overlap issues
Routing complexity
Transitive routing limitations
11.5 AWS Transit Gateway
Multiple VPCs
|
v
AWS Transit Gateway
|
v
Confluent Cloud
Confluent Cloud supports private connectivity options including AWS PrivateLink, Azure Private Link, VPC/VNet peering, and AWS Transit Gateway. (Confluent Documentation)
Use cases:
Many VPCs
Centralized enterprise networking
Shared services VPC
Multi-account AWS setup
Best when many EKS clusters or AWS accounts need Kafka access.
11.6 Private Network Interface
Confluent Private Network Interface enables private communication between your AWS resources and Confluent Cloud clusters through Elastic Network Interfaces in your AWS account. (Confluent Documentation)
Use cases:
AWS-native private access
Enterprise/Freight cluster designs
Private networking with ENI model
12. Ports and firewall requirements
Very important:
| Endpoint | Port | Protocol |
|---|---|---|
| Kafka bootstrap | 9092 | SASL_SSL or mTLS |
| Kafka REST endpoint | 443 | HTTPS |
Confluent’s connectivity testing guide says Kafka clusters use port 9092 for bootstrap and port 443 for Kafka REST, and that Terraform Provider / Kafka REST require access to port 443. (Confluent Documentation)
Test Kafka bootstrap:
nc -zv pkc-xxxxx.ap-northeast-1.aws.confluent.cloud 9092
Test REST endpoint:
nc -zv pkc-xxxxx.ap-northeast-1.aws.confluent.cloud 443
Test TLS/SNI:
openssl s_client \
-servername pkc-xxxxx.ap-northeast-1.aws.confluent.cloud \
-connect pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092
Confluent notes that Kafka broker hosts do not respond to ping, so use tools like nc, openssl, or telnet instead. (Confluent Documentation)
13. EKS recommended architecture
For your EKS case, I would recommend this order.
Short-term practical design
flowchart LR
A[EKS Pod] --> B[NAT Gateway Elastic IP]
B --> C[Confluent Public Endpoint]
C --> D[Kafka Cluster]
E[AWS Secrets Manager] --> F[External Secrets Operator]
F --> G[Kubernetes Secret]
G --> A
Use:
Native Kafka client
SASL_SSL
Kafka API key/secret
Service account per app
ACL least privilege
NAT Gateway static EIP
Confluent IP Filtering if possible
Secrets stored in AWS Secrets Manager
External Secrets Operator sync to Kubernetes
Best enterprise design
flowchart LR
A[EKS Private Subnet Pod] --> B[AWS PrivateLink / VPC Endpoint]
B --> C[Confluent Private Endpoint]
C --> D[Confluent Kafka Cluster]
E[Terraform Cloud Agent in VPC] --> B
F[Admin Bastion / VPN] --> B
Use:
PrivateLink
Native Kafka protocol
Service account API key OR OAuth/OIDC OR mTLS
ACL/RBAC least privilege
Terraform Cloud Agent inside VPC if Terraform must reach private resources
This matches your earlier EKS API endpoint discussion: if workloads/tools must reach private endpoints, run the automation agent inside the private network.
14. Kubernetes example for EKS app
Secret
apiVersion: v1
kind: Secret
metadata:
name: confluent-kafka-orders
namespace: orders-prod
type: Opaque
stringData:
KAFKA_BOOTSTRAP_SERVERS: "pkc-xxxxx.ap-northeast-1.aws.confluent.cloud:9092"
KAFKA_API_KEY: "<api-key>"
KAFKA_API_SECRET: "<api-secret>"
Deployment env vars
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-service
namespace: orders-prod
spec:
replicas: 3
selector:
matchLabels:
app: orders-service
template:
metadata:
labels:
app: orders-service
spec:
containers:
- name: orders-service
image: myrepo/orders-service:1.0.0
env:
- name: KAFKA_BOOTSTRAP_SERVERS
valueFrom:
secretKeyRef:
name: confluent-kafka-orders
key: KAFKA_BOOTSTRAP_SERVERS
- name: KAFKA_API_KEY
valueFrom:
secretKeyRef:
name: confluent-kafka-orders
key: KAFKA_API_KEY
- name: KAFKA_API_SECRET
valueFrom:
secretKeyRef:
name: confluent-kafka-orders
key: KAFKA_API_SECRET
Production improvement:
Do not manually create Kubernetes Secrets.
Store secret in AWS Secrets Manager.
Sync using External Secrets Operator.
Rotate Confluent API key periodically.
15. Which method to use when?
| Requirement | Best method |
|---|---|
| EKS microservice produces/consumes events | Native Kafka protocol |
| High throughput | Native Kafka protocol |
| Long-running consumer group | Native Kafka protocol |
| Serverless function sends occasional events | Kafka REST API |
| Shell script sends test event | Kafka REST API or CLI |
| CI/CD creates topics/ACLs | Terraform Provider or Kafka REST Admin API |
| Platform team provisions Confluent resources | Terraform + Confluent Cloud API |
| Human debugging | Confluent CLI |
| Database to Kafka | Managed source connector |
| Kafka to Snowflake/S3/Elastic | Managed sink connector |
| Multi-region replication | Cluster Linking |
| Migration from old Kafka to new Kafka | Cluster Linking |
| Schema governance | Schema Registry |
| Enterprise identity, no long-lived Kafka secrets | OAuth/OIDC |
| Certificate-based machine identity | mTLS |
| Strict private AWS access | AWS PrivateLink |
| Many AWS VPCs/accounts | AWS Transit Gateway |
16. Recommended standard for your organization
For a production EKS + Confluent Cloud setup, I would define standards like this:
Application standard
Connection method:
Native Kafka protocol
Network:
PrivateLink preferred
Public + IP Filtering acceptable as interim
Authentication:
Service account Kafka API key/secret initially
OAuth/OIDC or mTLS for mature enterprise identity
Authorization:
Least-privilege ACLs or scoped RBAC
Secrets:
AWS Secrets Manager + External Secrets Operator
Observability:
Consumer lag, delivery error rate, auth failures, broker connection errors
Platform/IaC standard
Tool:
Terraform Provider for Confluent
Authentication:
Cloud API key/secret stored in Terraform Cloud sensitive variables or Vault
Network:
Terraform execution must have access to Confluent Cloud API and Kafka REST endpoint if managing Kafka resources
Ownership:
Platform team owns environments, clusters, networking, service accounts, ACL/RBAC patterns
Application teams own topic usage and client configuration
Human/debug standard
Tool:
Confluent CLI or kcat
Authentication:
Temporary/user-scoped access where possible
Access:
Via VPN/bastion/private network if cluster is private
17. Simple final mental model
Native Kafka Protocol
= Best for applications and real streaming.
Kafka REST API
= Best for HTTPS-only, serverless, scripts, light integrations.
Confluent Cloud Management API
= Best for provisioning and automation.
Confluent CLI
= Best for humans, debugging, and testing.
Managed Connectors
= Best when moving data between Kafka and external systems.
Schema Registry
= Best for governed Avro/Protobuf/JSON Schema events.
Cluster Linking
= Best for Kafka-to-Kafka replication, migration, DR, and multi-region.
My recommended answer for your team:
For application workloads on EKS, the preferred method is direct Kafka data-plane access using the native Kafka protocol over SASL_SSL or mTLS. The application should authenticate using a Confluent service-account-based Kafka API key/secret, OAuth/OIDC, or mTLS, and should be authorized using least-privilege ACLs or scoped RBAC.
Confluent Cloud APIs and Terraform should be used for control-plane automation such as creating clusters, topics, service accounts, API keys, ACLs, RBAC bindings, and networking. Kafka REST API is useful for HTTPS-based integrations, serverless workloads, scripts, and CI/CD cases where native Kafka clients are not practical.
For production EKS, the strongest network design is AWS PrivateLink between the EKS VPC and Confluent Cloud, with secrets managed through AWS Secrets Manager and synced into Kubernetes through External Secrets Operator.
