This is more of a bookmark/reminder, but I ran into issues where we weren’t sure if the SIEM tool being used to ingest data from OCI Streaming is actually able to connect to the stream.
I figured there must be a tool for this! And there is. It’s called kcat (previously Kafka Connect). Nice thing I found is that you can find it from the Oracle Linux EPEL repos as well.
Let’s get it installed first in our OCI Oracle Linux 9 VM. You need to navigate to /etc/yum.repos.d/ and enable the oracle-epel-ol9.repo. This is as easy as opening the file and changing the enabled line to be 1:
[opc@stream yum.repos.d]$ cat oracle-epel-ol9.repo [ol9_developer_EPEL] name=Oracle Linux $releasever EPEL Packages for Development ($basearch) baseurl=https://yum$ociregion.$ocidomain/repo/OracleLinux/OL9/developer/EPEL/$basearch/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle gpgcheck=1 enabled=1
After you’re done, run sudo yum update and sudo yum install kcat:
[opc@stream yum.repos.d]$ sudo yum install kcat Last metadata expiration check: 0:06:04 ago on Thu 14 Aug 2025 02:40:50 PM GMT. Dependencies resolved. ===================================================================================================================================== Package Architecture Version Repository Size ===================================================================================================================================== Installing: kcat x86_64 1.7.1-1.el9 ol9_developer_EPEL 43 k Installing dependencies: librdkafka x86_64 1.6.1-102.el9 ol9_appstream 674 k Transaction Summary ===================================================================================================================================== Install 2 Packages Total download size: 717 k Installed size: 2.1 M Is this ok [y/N]: y Downloading Packages: (1/2): kcat-1.7.1-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 (2/2): librdkafka-1.6.1-102.el9.x86_64.rpm 17 MB/s | 674 kB 00:00 ------------------------------------------------------------------------------------------------------------------------------------- Total 16 MB/s | 717 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : librdkafka-1.6.1-102.el9.x86_64 1/2 Installing : kcat-1.7.1-1.el9.x86_64 2/2 Running scriptlet: kcat-1.7.1-1.el9.x86_64 2/2 Verifying : kcat-1.7.1-1.el9.x86_64 1/2 Verifying : librdkafka-1.6.1-102.el9.x86_64 2/2 Installed: kcat-1.7.1-1.el9.x86_64 librdkafka-1.6.1-102.el9.x86_64 Complete!
Now assuming we have a OCI Streaming already setup, you would navigate to your Stream and to Stream Pool to get the Kafka connection settings.
I want to click on the left on the settings to get values to use with kcat.
I will require all the values from the settings. Note the username which will be in form of “tenancy/username/ocid1.of.the.streampool” – it’s not the stream OCID but the stream pool. I also want to note down the name of my stream.
So to summarize what I need:
- User with access to ingest the stream (group and policy)
- Bootstrap server
- SASL Connection String with username (tenancy/domain_if_not_DEFAULT/username/ocid.streampool) & password (user auth_token)
- Security Protocol SASL_SSL
- Security Mechanism PLAIN
- Stream name
Note that you have to enter the domain in the username UNLESS your user is in the Default domain.
After this I’m good to test this with kcat.
First I want to find out what topics there are available in my stream using the -L flag:
[opc@stream ~]$ kcat -b cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username="tfg/OracleIdentityCloudService/simo@tfg.com/ocid1.streampool.oc1.ca-toronto-1.amaaaaaahrvsn4ya4ipj3xcg7aghr3tzjshud4xaaddddga" -X sasl.password="my_auth_token" -L
Metadata for all topics (from broker 0: sasl_ssl://cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092/0):
21 brokers:
broker 5 at br5-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 10 at br10-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 17 at br17-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 6 at br6-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 16 at br16-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 7 at br7-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 12 at br12-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 19 at br19-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 20 at br20-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 11 at br11-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 18 at br18-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 14 at br14-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 0 at cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 (controller)
broker 1 at br1-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 8 at br8-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 13 at br13-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 9 at br9-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 2 at br2-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 3 at br3-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 4 at br4-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
broker 15 at br15-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
1 topics:
topic "sentinel-stream" with 1 partitions:
partition 0, leader 11, replicas: , isrs:
I see I have one topic called “sentinel-stream” so I will use -C and -t flags now to consume that topic.
[opc@stream ~]$ kcat -C -t "sentinel-stream" -b cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username="tfg/OracleIdentityCloudService/simo@tfg.com/ocid1.streampool.oc1.ca-toronto-1.amaaaaaahrvsn4ya4ipj3xcg7aghr3tzjshud4xaaddddga" -X sasl.password="my_auth_token" >> audit.log
% Reached end of topic sentinel-stream [0] at offset 6663
[opc@stream ~]$ head -1 audit.log
{"data":{"additionalDetails":null,"availabilityDomain":"AD1","compartmentId":"ocid1.tenancy.oc1..aaaaaaaaifdc6yaaaa","compartmentName":"simovilmunen","definedTags":null,"eventGroupingId":"49730f57-c653-4ce8-b4c7-8575e070a9d5","eventName":"generateUnifiedAgentConfiguration","freeformTags":null,"identity":{"authType":"instance","callerId":null,"callerName":null,"consoleSessionId":null,"credentials":"ST$eyJraWQiOiJhc3dfeXl6XzE3MTMyMzE2NDYzODciLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJvY2lkMS5pbnN0YW5jZS5vYzEuY2EtdG9yb250by0xLmFuMmc2bGpyaHJ2c240eWM2YWhybXB1bzdsN2lxN29vaXNuampscnJ4a25xZ3JpaGNncjd3cW9jNzJwYSIsIm9wYy1jZXJ0dHlwZSI6Imluc3RhbmNlIiwiaXNzIjoiYXV0aFNlcnZpY2Uub3JhY2xlLmNvbSIsImZwcmludCI6IjI4OkFDOkVDOj
....
Summary
Kcat is useful tool to test connectivity to OCI streaming, maybe you want to validate settings that those are setup correctly or you want to see there are no firewalls blocking the connectivity.