EventSmartKafka — Final Release

EventSmartKafka is an open-source custom Kafka Consumer Sender Adapter for SAP BTP Integration Suite / SAP Cloud Integration, built with SAP ADK.

Part ISAP BTP – CI – How I Built an Open-Source Enterprise Custom Kafka Adapter from Scratch – Part IPart IISAP BTP – CI – Building Open-Source Custom Adapter EventSmartKafka from Zero with SAP ADK – Part IIPart IIISAP BTP – CI – EventSmartKafka – Custom Kafka Adapter From .ESA – Validated Runtime – Part IIIPart IV SAP BTP – CI – EventSmartKafka – Custom Kafka Adapter From .ESA v1.1.0 – Validated Runtime – Part IV

After multiple development iterations and validation cycles, EventSmartKafka has reached its final release.

The adapter provides support for:

Apache KafkaSAP Cloud Connector (TCP/SOCKS5)Cloud and On-Premise connectivityPLAINTEXTSASL/PLAINSCRAM-SHA-256SCRAM-SHA-512SASL_SSLMutual TLS (mTLS)Apache Avro to JSON/XML conversionSchema Registry integrationMessage recovery by timestampMultiple consumer modesKafka Producer support

The Consumer component has been validated through extensive long-running tests totaling more than 6.5 million processed messages, with zero processing failures.

The Producer component has also been successfully validated in both local and on-premise environments, covering all supported authentication methods, message delivery scenarios, automatic topic creation, and continuous operation for more than 20 consecutive hours.

In addition, deployment lifecycle testing confirmed that deleting an existing adapter, installing a newer ESA version, and redeploying dependent iFlows can be performed successfully without runtime interruptions or deployment issues.

Cloud to Cloud – SALS – TLS – PLAIN

Cloud to on-premise – None – No Authentication:

Cloud to on-premise – SALS – No TLS – Basic Authentication:

Cloud to on-premise – SALS – TLS – Scram 512 and Private Certificate – SALS_SSL:

Cloud to On-premise – mTLS – SSL – Private KEY Certificate.

Cloud Connector – TCP – SOCKS5:

Kafka Local:

Compose in case that you want to test for free, just download the kafka local:

services:
sdia-kafka:
image: apache/kafka:4.1.2
container_name: sdia-kafka
hostname: sdia-kafka
user: root
ports:
– “19108:19108”
– “19109:19109”
– “19110:19110”
– “19111:19111”
volumes:
– ./certs:/etc/kafka/secrets
– kafka-data:/tmp/kafka-logs
environment:
KAFKA_NODE_ID: “1”
KAFKA_PROCESS_ROLES: “broker,controller”
KAFKA_LISTENERS: “CONTROLLER://0.0.0.0:9093,INTERNAL://0.0.0.0:9092,NONEEXT://0.0.0.0:19108,SASLEXT://0.0.0.0:19109,SASLSSLEXT://0.0.0.0:19110,MTLSEXT://0.0.0.0:19111”
KAFKA_ADVERTISED_LISTENERS: “INTERNAL://sdia-kafka:9092,NONEEXT://localhost:19108,SASLEXT://localhost:19109,SASLSSLEXT://localhost:19110,MTLSEXT://localhost:19111”
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: “CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,NONEEXT:PLAINTEXT,SASLEXT:SASL_PLAINTEXT,SASLSSLEXT:SASL_SSL,MTLSEXT:SSL”
KAFKA_INTER_BROKER_LISTENER_NAME: “INTERNAL”
KAFKA_CONTROLLER_LISTENER_NAMES: “CONTROLLER”
KAFKA_CONTROLLER_QUORUM_VOTERS: “1@sdia-kafka:9093”
KAFKA_MESSAGE_MAX_BYTES: “26214400”
KAFKA_REPLICA_FETCH_MAX_BYTES: “26214400”
KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: “LogAppendTime”

# SASL
KAFKA_SASL_ENABLED_MECHANISMS: “PLAIN,SCRAM-SHA-256,SCRAM-SHA-512”
KAFKA_LISTENER_NAME_SASLEXT_SASL_ENABLED_MECHANISMS: “PLAIN,SCRAM-SHA-256,SCRAM-SHA-512”
KAFKA_LISTENER_NAME_SASLEXT_PLAIN_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.plain.PlainLoginModule required username=”sdia” password=”SDIABRASIL” user_sdia=”SDIABRASIL”;’
KAFKA_LISTENER_NAME_SASLEXT_SCRAM___SHA___256_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.scram.ScramLoginModule required;’
KAFKA_LISTENER_NAME_SASLEXT_SCRAM___SHA___512_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.scram.ScramLoginModule required;’
KAFKA_LISTENER_NAME_SASLSSLEXT_SASL_ENABLED_MECHANISMS: “PLAIN,SCRAM-SHA-256,SCRAM-SHA-512”
KAFKA_LISTENER_NAME_SASLSSLEXT_PLAIN_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.plain.PlainLoginModule required username=”USER” password=”ADD THE PASSWORD” user_sdia=”ADD THE USER”;’
KAFKA_LISTENER_NAME_SASLSSLEXT_SCRAM___SHA___256_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.scram.ScramLoginModule required;’
KAFKA_LISTENER_NAME_SASLSSLEXT_SCRAM___SHA___512_SASL_JAAS_CONFIG: ‘org.apache.kafka.common.security.scram.ScramLoginModule required;’

# SSL — definindo os paths diretamente, bypassa a condição do script
# que só ativa quando ADVERTISED_LISTENERS contém “SSL://” literal
KAFKA_SSL_KEYSTORE_LOCATION: “/etc/kafka/secrets/kafka.server.keystore.p12”
KAFKA_SSL_KEYSTORE_PASSWORD: “changeit”
KAFKA_SSL_KEYSTORE_TYPE: “PKCS12”
KAFKA_SSL_KEY_PASSWORD: “changeit”
KAFKA_SSL_TRUSTSTORE_LOCATION: “/etc/kafka/secrets/kafka.server.truststore.p12”
KAFKA_SSL_TRUSTSTORE_PASSWORD: “ADD THE PASSWORD”
KAFKA_SSL_TRUSTSTORE_TYPE: “PKCS12”
KAFKA_SSL_CLIENT_AUTH: “none”
KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: “”

# mTLS — sobrescreve client.auth só para MTLSEXT
KAFKA_LISTENER_NAME_MTLSEXT_SSL_CLIENT_AUTH: “required”

KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: “1”
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: “1”
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: “1”
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: “0”
KAFKA_AUTO_CREATE_TOPICS_ENABLE: “true”
KAFKA_NUM_PARTITIONS: “1”

volumes:
kafka-data:

After you install dockers and kafka local, is free, you can use the setup .ps1 below, execute via Powershell, just change the topics in the way you want or not, those are using – SDIA

The script generate the uses, password, topics, all parameters and others

# ============================================================================
# EventSmartKafka — Local Kafka Setup
# Portas: 19108 PLAINTEXT | 19109 SASL | 19110 SASL+TLS | 19111 mTLS
# PRE-REQUISITO: certificados em .certs gerados pelo gerar-certs.ps1
# ============================================================================

$container = “sdia-kafka”

Set-Location C:SAPCPI.CustomAdapterkafka-local-demo

# Verifica se os certs existem
if (-not (Test-Path “.certskafka.server.keystore.p12”)) {
Write-Host “ERRO: certs nao encontrados. Rode primeiro: .gerar-certs.ps1” -ForegroundColor Red
exit 1
}

# —————————————————————————-
# 1 — Derrubar e subir
# —————————————————————————-
Write-Host “Derrubando instancia anterior…” -ForegroundColor Yellow
docker compose down -v
Start-Sleep -Seconds 3

Write-Host “Subindo Kafka…” -ForegroundColor Yellow
docker compose up -d

# —————————————————————————-
# 2 — Aguardar broker (porta PLAINTEXT 19108)
# —————————————————————————-
Write-Host “Aguardando broker…” -ForegroundColor Cyan
$ready = $false
for ($i = 1; $i -le 30; $i++) {
docker exec $container /opt/kafka/bin/kafka-broker-api-versions.sh –bootstrap-server localhost:19108 *> $null
if ($LASTEXITCODE -eq 0) { $ready = $true; break }
Write-Host ” Tentativa $i/30 — aguardando 5s…”
Start-Sleep -Seconds 5
}
if (-not $ready) {
Write-Host “ERRO: Kafka nao ficou pronto.” -ForegroundColor Red
docker logs $container –tail 20
exit 1
}
Write-Host “Kafka pronto!” -ForegroundColor Green

# —————————————————————————-
# 3 — Properties (referenciando certs dentro do container)
# —————————————————————————-
Write-Host “”
Write-Host “Criando properties…” -ForegroundColor Yellow

# SASL PLAIN (porta 19109)
@”
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
“@ | Out-File “.plain.properties” -Encoding ASCII

# SASL SCRAM-256 (porta 19109)
@”
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
“@ | Out-File “.scram256.properties” -Encoding ASCII

# SASL SCRAM-512 (porta 19109)
@”
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
“@ | Out-File “.scram512.properties” -Encoding ASCII

# SASL_SSL PLAIN (porta 19110)
@”
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.p12
ssl.truststore.password=changeit
ssl.truststore.type=PKCS12
ssl.endpoint.identification.algorithm=
“@ | Out-File “.sslplain.properties” -Encoding ASCII

# SASL_SSL SCRAM-256 (porta 19110)
@”
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.p12
ssl.truststore.password=changeit
ssl.truststore.type=PKCS12
ssl.endpoint.identification.algorithm=
“@ | Out-File “.ssl256.properties” -Encoding ASCII

# SASL_SSL SCRAM-512 (porta 19110)
@”
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=”CHANGE THE USER” password=”CHANGE THE PASSWORD”;
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.p12
ssl.truststore.password=changeit
ssl.truststore.type=PKCS12
ssl.endpoint.identification.algorithm=
“@ | Out-File “.ssl512.properties” -Encoding ASCII

# mTLS (porta 19111) — usa o keystore do servidor como truststore do cliente (mesmo CA)
@”
security.protocol=SSL
ssl.keystore.location=/etc/kafka/secrets/sdia-kafka-client.p12
ssl.keystore.password=changeit
ssl.keystore.type=PKCS12
ssl.key.password=changeit
ssl.truststore.location=/etc/kafka/secrets/kafka.server.truststore.p12
ssl.truststore.password=changeit
ssl.truststore.type=PKCS12
ssl.endpoint.identification.algorithm=
“@ | Out-File “.mtls.properties” -Encoding ASCII

docker cp .plain.properties ${container}:/tmp/plain.properties
docker cp .scram256.properties ${container}:/tmp/scram256.properties
docker cp .scram512.properties ${container}:/tmp/scram512.properties
docker cp .sslplain.properties ${container}:/tmp/sslplain.properties
docker cp .ssl256.properties ${container}:/tmp/ssl256.properties
docker cp .ssl512.properties ${container}:/tmp/ssl512.properties
docker cp .mtls.properties ${container}:/tmp/mtls.properties

# Copia tambem o cert do cliente para dentro do container (necessario para mTLS topics)
docker cp .certssdia-kafka-client.p12 ${container}:/etc/kafka/secrets/sdia-kafka-client.p12

# —————————————————————————-
# 4 — Usuarios SCRAM (via porta SASL_PLAINTEXT 19109)
# —————————————————————————-
Write-Host “”
Write-Host “Criando usuarios SCRAM…” -ForegroundColor Yellow
docker exec $container /opt/kafka/bin/kafka-configs.sh –bootstrap-server localhost:19109 –command-config /tmp/plain.properties –alter –add-config ‘SCRAM-SHA-256=[iterations=8192,password=SDIABRASIL]’ –entity-type users –entity-name sdia
docker exec $container /opt/kafka/bin/kafka-configs.sh –bootstrap-server localhost:19109 –command-config /tmp/plain.properties –alter –add-config ‘SCRAM-SHA-512=[iterations=8192,password=CHANGE THE PASSWORD]’ –entity-type users –entity-name sdia
docker exec $container /opt/kafka/bin/kafka-configs.sh –bootstrap-server localhost:19109 –command-config /tmp/plain.properties –describe –entity-type users –entity-name sdia

# —————————————————————————-
# 5 — Verificar SSL (porta 19110)
# —————————————————————————-
Write-Host “”
Write-Host “Verificando SASL_SSL 19110…” -ForegroundColor Yellow
docker exec $container /opt/kafka/bin/kafka-broker-api-versions.sh –bootstrap-server localhost:19110 –command-config /tmp/sslplain.properties
Write-Host “SSL OK se apareceu lista acima.” -ForegroundColor Cyan

# —————————————————————————-
# 6 — Topicos PLAINTEXT 19108
# —————————————————————————-
Write-Host “”
Write-Host “Topicos 19108 PLAINTEXT…” -ForegroundColor Yellow
@(
“id00.acme.sales.otc.orders.created.n.s”,
“id13.acme.sales.otc.orders.created.n.s.cj”,
“id14.acme.sales.otc.orders.created.n.s.cj.1mb”,
“id15.acme.sales.otc.orders.created.n.s.cj.2mb”,
“id16.acme.sales.otc.orders.created.n.s.cx”,
“id17.acme.sales.otc.orders.created.n.s.cx.1mb”,
“id18.acme.sales.otc.orders.created.n.s.cx.2mb”
) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19108 –partitions 1 –replication-factor 1
}

# —————————————————————————-
# 7 — Topicos SASL 19109
# —————————————————————————-
Write-Host “”
Write-Host “Topicos 19109 SASL…” -ForegroundColor Yellow
@(“id01.acme.sales.otc.fulfillment.started.s.p”, “id19.acme.sales.otc.orders.created.n.s.cj.53kb”, “id20.acme.sales.otc.orders.created.n.s.cj.53kb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19109 –command-config /tmp/plain.properties –partitions 1 –replication-factor 1
}
@(“id03.acme.sales.otc.orders.updated.s.s.256”, “id21.acme.sales.otc.orders.created.n.s.cj.1mb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19109 –command-config /tmp/scram256.properties –partitions 1 –replication-factor 1
}
@(“id05.acme.sales.otc.shipments.dispatched.s.s.512”, “id22.acme.sales.otc.orders.created.n.s.cj.2mb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19109 –command-config /tmp/scram512.properties –partitions 1 –replication-factor 1
}

# —————————————————————————-
# 8 — Topicos SASL_SSL 19110
# —————————————————————————-
Write-Host “”
Write-Host “Topicos 19110 SASL_SSL…” -ForegroundColor Yellow
@(“id02.acme.sales.otc.fulfillment.started.s.tls.p”, “id23.acme.sales.otc.orders.created.n.s.cj.53kb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19110 –command-config /tmp/sslplain.properties –partitions 1 –replication-factor 1
}
@(“id04.acme.sales.otc.orders.updated.s.tls.s.256”, “id24.acme.sales.otc.orders.created.n.s.cj.1mb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19110 –command-config /tmp/ssl256.properties –partitions 1 –replication-factor 1
}
@(“id06.acme.sales.otc.shipments.dispatched.s.tls.s.512”, “id25.acme.sales.otc.orders.created.n.s.cj.2mb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19110 –command-config /tmp/ssl512.properties –partitions 1 –replication-factor 1
}

# —————————————————————————-
# 9 — Topicos mTLS 19111
# —————————————————————————-
Write-Host “”
Write-Host “Topicos 19111 mTLS…” -ForegroundColor Yellow
@(“id26.acme.sales.otc.orders.created.mtls”, “id27.acme.sales.otc.orders.created.mtls.1mb”, “id28.acme.sales.otc.orders.created.mtls.2mb”) | ForEach-Object {
docker exec $container /opt/kafka/bin/kafka-topics.sh –create –if-not-exists –topic $_ –bootstrap-server localhost:19111 –command-config /tmp/mtls.properties –partitions 1 –replication-factor 1
}

# —————————————————————————-
# 10 — Lista topicos e validacao Python
# —————————————————————————-
Write-Host “”
Write-Host “Topicos criados:” -ForegroundColor Yellow
docker exec $container /opt/kafka/bin/kafka-topics.sh –list –bootstrap-server localhost:19108

Write-Host “”
Write-Host “Validacao Python…” -ForegroundColor Yellow
Set-Location C:SAPCPI.CustomAdapterkafka-local-demoavro
python x.py

Script to generate a local certificate for TLS with CA and mTLS of the local kafka:

# Gera certificados via container Docker — sem openssl local necessario
# Execute ANTES de subir o docker-compose.yml

Set-Location C:SAPCPI.CustomAdapterkafka-local-demo
New-Item -ItemType Directory -Force -Path “.certs” | Out-Null

# Remove certs antigos
Remove-Item “.certs*.p12” -Force -ErrorAction SilentlyContinue
Remove-Item “.certs*.crt” -Force -ErrorAction SilentlyContinue
Remove-Item “.certs*.pem” -Force -ErrorAction SilentlyContinue

Write-Host “Rodando geracao de certs no container…” -ForegroundColor Yellow
docker run –rm `
-v “${PWD}certs:/certs” `
-v “${PWD}gen.sh:/gen.sh” `
apache/kafka:4.1.2 sh /gen.sh

if ($LASTEXITCODE -ne 0) {
Write-Host “ERRO ao gerar certs.” -ForegroundColor Red
exit 1
}

Write-Host “”
Write-Host “Certs gerados em ./certs/:” -ForegroundColor Green
Get-ChildItem .certs

Confluent – Cloud – 1000 + messages

The source code, documentation and release artifacts are available on GitHub.

https://github.com/rhviana/sap-ci-opensource-adapters

Future Projects

Although EventSmartKafka has reached its final release, work continues on new initiatives.

The next project currently under development is SDIA for SAP Cloud Integration, a browser extension focused on integration governance, domain-driven naming conventions, and architecture standardization.

The SDIA architecture was created and published this year as open research under the Creative Commons CC BY 4.0 International license.

The browser extension itself will not be open source or free of charge. A release date has not yet been announced.

SDIA — Semantic Domain Integration Architecture
└── ODCP — SAP Cloud Integration

ODCP — DOI: 10.5281/zenodo.18876594 – [https://lnkd.in/d9icRNyd] – IPCOM000277631D
SDIA — DOI: 10.5281/zenodo.18877635 [https://lnkd.in/dh5MdbjE] – IPCOM000277630D

Update on SAP CITS about process direct and generate a full documentation from package, all in one document.

SAP BTP – CI – CITS Extension – Generate Technical Specifications Directly from the iFlow Designer

Thank you to everyone who followed the series.

Kind regards,

Ricardo Viana
Integration Architecture Independent Solo Researcher
SAP BTP Integration Suite Expert Developer
SAP Press Author – Enterprise Messaging – 2021
Creator of DEIP · SDIA · GDCR · DDCR · ODCP · EDCP · DDCP

#SAPIntegrationSuite #SAPCloudIntegration #Kafka #SAPADK #OpenSource #ApacheKafka #CloudConnector #mTLS #OSGi #EventDrivenArchitecture #SDIA #StressTest #EnterpriseIntegration #SAPSCN

EventSmartKafka has reached its final and stable release. No major new features are planned. Released under the Apache License 2.0, this project is provided as-is, without SLA, commercial support, maintenance, or SAP certification. It is an independent open-source initiative created to help SAP BTP Integration Suite professionals learn how to build custom adapters from the ground up using the SAP Adapter Development Kit (ADK).EventSmartKafka — Final ReleaseEventSmartKafka is an open-source custom Kafka Consumer Sender Adapter for SAP BTP Integration Suite / SAP Cloud Integration, built with SAP ADK. Part ISAP BTP – CI – How I Built an Open-Source Enterprise Custom Kafka Adapter from Scratch – Part IPart IISAP BTP – CI – Building Open-Source Custom Adapter EventSmartKafka from Zero with SAP ADK – Part IIPart IIISAP BTP – CI – EventSmartKafka – Custom Kafka Adapter From .ESA – Validated Runtime – Part IIIPart IV SAP BTP – CI – EventSmartKafka – Custom Kafka Adapter From .ESA v1.1.0 – Validated Runtime – Part IVAfter multiple development iterations and validation cycles, EventSmartKafka has reached its final release.The adapter provides support for:Apache KafkaSAP Cloud Connector (TCP/SOCKS5)Cloud and On-Premise connectivityPLAINTEXTSASL/PLAINSCRAM-SHA-256SCRAM-SHA-512SASL_SSLMutual TLS (mTLS)Apache Avro to JSON/XML conversionSchema Registry integrationMessage recovery by timestampMultiple consumer modesKafka Producer supportThe Consumer component has been validated through extensive long-running tests totaling more than 6.5 million processed messages, with zero processing failures.The Producer component has also been successfully validated in both local and on-premise environments, covering all supported authentication methods, message delivery scenarios, automatic topic creation, and continuous operation for more than 20 consecutive hours.In addition, deployment lifecycle testing confirmed that deleting an existing adapter, installing a newer ESA version, and redeploying dependent iFlows can be performed successfully without runtime interruptions or deployment issues.Cloud to Cloud – SALS – TLS – PLAIN Cloud to on-premise – None – No Authentication:Cloud to on-premise – SALS – No TLS – Basic Authentication:Cloud to on-premise – SALS – TLS – Scram 512 and Private Certificate – SALS_SSL:Cloud to On-premise – mTLS – SSL – Private KEY Certificate.Cloud Connector – TCP – SOCKS5:Kafka Local: Compose in case that you want to test for free, just download the kafka local:services:
sdia-kafka:
image: apache/kafka:4.1.2
container_name: sdia-kafka
hostname: sdia-kafka
user: root
ports:
– “19108:19108”
– “19109:19109”
– “19110:19110”
– “19111:19111”
volumes:
– ./certs:/etc/kafka/secrets
– kafka-data:/tmp/kafka-logs
environment:
KAFKA_NODE_ID: “1”
KAFKA_PROCESS_ROLES: “broker,controller”
KAFKA_LISTENERS: “CONTROLLER://0.0.0.0:9093,INTERNAL://0.0.0.0:9092,NONEEXT://0.0.0.0:19108,SASLEXT://0.0.0.0:19109,SASLSSLEXT://0.0.0.0:19110,MTLSEXT://0.0.0.0:19111”
KAFKA_ADVERTISED_LISTENERS: “INTERNAL://sdia-kafka:9092,NONEEXT://localhost:19108,SASLEXT://localhost:19109,SASLSSLEXT://localhost:19110,MTLSEXT://localhost:19111”
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: “CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,NONEEXT:PLAINTEXT,SASLEXT:SASL_PLAINTEXT,SASLSSLEXT:SASL_SSL,MTLSEXT:SSL”
KAFKA_INTER_BROKER_LISTENER_NAME: “INTERNAL”
KAFKA_CONTROLLER_LISTENER_NAMES: “CONTROLLER”
KAFKA_CONTROLLER_QUORUM_VOTERS: “1@sdia-kafka:9093”
KAFKA_MESSAGE_MAX_BYTES: “26214400”
KAFKA_REPLICA_FETCH_MAX_BYTES: “26214400”
KAFKA_LOG_MESSAGE_TIMESTAMP_TYPE: “LogAppendTime”

# mTLS — sobrescreve client.auth só para MTLSEXT
KAFKA_LISTENER_NAME_MTLSEXT_SSL_CLIENT_AUTH: “required”

volumes:
kafka-data:After you install dockers and kafka local, is free, you can use the setup .ps1 below, execute via Powershell, just change the topics in the way you want or not, those are using – SDIAThe script generate the uses, password, topics, all parameters and others# ============================================================================
# EventSmartKafka — Local Kafka Setup
# Portas: 19108 PLAINTEXT | 19109 SASL | 19110 SASL+TLS | 19111 mTLS
# PRE-REQUISITO: certificados em .certs gerados pelo gerar-certs.ps1
# ============================================================================

$container = “sdia-kafka”

Set-Location C:SAPCPI.CustomAdapterkafka-local-demo

Write-Host “Subindo Kafka…” -ForegroundColor Yellow
docker compose up -d

# Copia tambem o cert do cliente para dentro do container (necessario para mTLS topics)
docker cp .certssdia-kafka-client.p12 ${container}:/etc/kafka/secrets/sdia-kafka-client.p12

Write-Host “”
Write-Host “Validacao Python…” -ForegroundColor Yellow
Set-Location C:SAPCPI.CustomAdapterkafka-local-demoavro
python x.py

Write-Host “”
Write-Host “Setup completo!” -ForegroundColor Green
Write-Host ” certssdia-kafka-client.p12 -> Key Pair CPI Keystore | alias: sdia-kafka | senha: changeit” -ForegroundColor Cyan
Write-Host ” certsca.crt -> Certificate CPI Keystore | alias: sdia-kafka-ca” -ForegroundColor CyanScript to generate a local certificate for TLS with CA and mTLS of the local kafka:# Gera certificados via container Docker — sem openssl local necessario
# Execute ANTES de subir o docker-compose.yml

Set-Location C:SAPCPI.CustomAdapterkafka-local-demo
New-Item -ItemType Directory -Force -Path “.certs” | Out-Null

Write-Host “Rodando geracao de certs no container…” -ForegroundColor Yellow
docker run –rm `
-v “${PWD}certs:/certs” `
-v “${PWD}gen.sh:/gen.sh” `
apache/kafka:4.1.2 sh /gen.sh

if ($LASTEXITCODE -ne 0) {
Write-Host “ERRO ao gerar certs.” -ForegroundColor Red
exit 1
}

Write-Host “”
Write-Host “Certs gerados em ./certs/:” -ForegroundColor Green
Get-ChildItem .certs

Write-Host “”
Write-Host “No CPI Keystore importar:” -ForegroundColor Cyan
Write-Host ” certssdia-kafka-client.p12 -> Key Pair | alias: sdia-kafka | senha: changeit”
Write-Host ” certsca.crt -> Certificate | alias: sdia-kafka-ca”
Write-Host “”
Write-Host “Agora rode: docker compose down -v && docker compose up -d” -ForegroundColor Cyan
Write-Host “Depois rode: .setup-kafka.ps1” -ForegroundColor CyanConfluent – Cloud – 1000 + messagesThe source code, documentation and release artifacts are available on GitHub.https://github.com/rhviana/sap-ci-opensource-adaptersFuture ProjectsAlthough EventSmartKafka has reached its final release, work continues on new initiatives.The next project currently under development is SDIA for SAP Cloud Integration, a browser extension focused on integration governance, domain-driven naming conventions, and architecture standardization.The SDIA architecture was created and published this year as open research under the Creative Commons CC BY 4.0 International license.The browser extension itself will not be open source or free of charge. A release date has not yet been announced.SDIA — Semantic Domain Integration Architecture└── ODCP — SAP Cloud IntegrationODCP — DOI: 10.5281/zenodo.18876594 – [https://lnkd.in/d9icRNyd] – IPCOM000277631DSDIA — DOI: 10.5281/zenodo.18877635 [https://lnkd.in/dh5MdbjE] – IPCOM000277630DUpdate on SAP CITS about process direct and generate a full documentation from package, all in one document.SAP BTP – CI – CITS Extension – Generate Technical Specifications Directly from the iFlow DesignerThank you to everyone who followed the series.Kind regards,Ricardo VianaIntegration Architecture Independent Solo ResearcherSAP BTP Integration Suite Expert DeveloperSAP Press Author – Enterprise Messaging – 2021Creator of DEIP · SDIA · GDCR · DDCR · ODCP · EDCP · DDCP#SAPIntegrationSuite #SAPCloudIntegration #Kafka #SAPADK #OpenSource #ApacheKafka #CloudConnector #mTLS #OSGi #EventDrivenArchitecture #SDIA #StressTest #EnterpriseIntegration #SAPSCN Read More Technology Blog Posts by Members articles

#SAP

#SAPTechnologyblog

Building the ultimate morning dashboard with Gemini 3.6 Flash and Nano Banana

Building the ultimate morning dashboard with Gemini 3.6 Flash and Nano Banana

The AI question every team should ask

Where does AI actually add value in this build, and where is it just hype?

SAP BTP – CI – EventSmartKafka – Custom Kafka Adapter – Part V FINAL