Test you can connect to your OCI Streaming endpoint

This is more of a bookmark/reminder, but I ran into issues where we weren’t sure if the SIEM tool being used to ingest data from OCI Streaming is actually able to connect to the stream.

I figured there must be a tool for this! And there is. It’s called kcat (previously Kafka Connect). Nice thing I found is that you can find it from the Oracle Linux EPEL repos as well.

Let’s get it installed first in our OCI Oracle Linux 9 VM. You need to navigate to /etc/yum.repos.d/ and enable the oracle-epel-ol9.repo. This is as easy as opening the file and changing the enabled line to be 1:

[opc@stream yum.repos.d]$ cat oracle-epel-ol9.repo
[ol9_developer_EPEL]
name=Oracle Linux $releasever EPEL Packages for Development ($basearch)
baseurl=https://yum$ociregion.$ocidomain/repo/OracleLinux/OL9/developer/EPEL/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

After you’re done, run sudo yum update and sudo yum install kcat:

[opc@stream yum.repos.d]$ sudo yum install kcat
Last metadata expiration check: 0:06:04 ago on Thu 14 Aug 2025 02:40:50 PM GMT.
Dependencies resolved.
=====================================================================================================================================
 Package                      Architecture             Version                            Repository                            Size
=====================================================================================================================================
Installing:
 kcat                         x86_64                   1.7.1-1.el9                        ol9_developer_EPEL                    43 k
Installing dependencies:
 librdkafka                   x86_64                   1.6.1-102.el9                      ol9_appstream                        674 k

Transaction Summary
=====================================================================================================================================
Install  2 Packages

Total download size: 717 k
Installed size: 2.1 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): kcat-1.7.1-1.el9.x86_64.rpm                                                                   1.3 MB/s |  43 kB     00:00
(2/2): librdkafka-1.6.1-102.el9.x86_64.rpm                                                            17 MB/s | 674 kB     00:00
-------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                 16 MB/s | 717 kB     00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                             1/1
  Installing       : librdkafka-1.6.1-102.el9.x86_64                                                                             1/2
  Installing       : kcat-1.7.1-1.el9.x86_64                                                                                     2/2
  Running scriptlet: kcat-1.7.1-1.el9.x86_64                                                                                     2/2
  Verifying        : kcat-1.7.1-1.el9.x86_64                                                                                     1/2
  Verifying        : librdkafka-1.6.1-102.el9.x86_64                                                                             2/2

Installed:
  kcat-1.7.1-1.el9.x86_64                                       librdkafka-1.6.1-102.el9.x86_64

Complete!

Now assuming we have a OCI Streaming already setup, you would navigate to your Stream and to Stream Pool to get the Kafka connection settings.

I want to click on the left on the settings to get values to use with kcat.

I will require all the values from the settings. Note the username which will be in form of “tenancy/username/ocid1.of.the.streampool” – it’s not the stream OCID but the stream pool. I also want to note down the name of my stream.

So to summarize what I need:

• User with access to ingest the stream (group and policy)
• Bootstrap server
• SASL Connection String with username (tenancy/domain_if_not_DEFAULT/username/ocid.streampool) & password (user auth_token)
• Security Protocol SASL_SSL
• Security Mechanism PLAIN
• Stream name

Note that you have to enter the domain in the username UNLESS your user is in the Default domain. 

After this I’m good to test this with kcat.

First I want to find out what topics there are available in my stream using the -L flag:

[opc@stream ~]$ kcat -b cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username="tfg/OracleIdentityCloudService/simo@tfg.com/ocid1.streampool.oc1.ca-toronto-1.amaaaaaahrvsn4ya4ipj3xcg7aghr3tzjshud4xaaddddga" -X sasl.password="my_auth_token" -L
Metadata for all topics (from broker 0: sasl_ssl://cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092/0):
 21 brokers:
  broker 5 at br5-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 10 at br10-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 17 at br17-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 6 at br6-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 16 at br16-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 7 at br7-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 12 at br12-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 19 at br19-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 20 at br20-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 11 at br11-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 18 at br18-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 14 at br14-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 0 at cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 (controller)
  broker 1 at br1-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 8 at br8-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 13 at br13-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 9 at br9-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 2 at br2-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 3 at br3-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 4 at br4-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
  broker 15 at br15-cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092
 1 topics:
  topic "sentinel-stream" with 1 partitions:
    partition 0, leader 11, replicas: , isrs:

I see I have one topic called “sentinel-stream” so I will use -C and -t flags now to consume that topic.

[opc@stream ~]$ kcat -C -t "sentinel-stream" -b cell-1.streaming.ca-toronto-1.oci.oraclecloud.com:9092 -X security.protocol=SASL_SSL -X sasl.mechanisms=PLAIN -X sasl.username="tfg/OracleIdentityCloudService/simo@tfg.com/ocid1.streampool.oc1.ca-toronto-1.amaaaaaahrvsn4ya4ipj3xcg7aghr3tzjshud4xaaddddga" -X sasl.password="my_auth_token" >> audit.log
% Reached end of topic sentinel-stream [0] at offset 6663

[opc@stream ~]$ head -1 audit.log
{"data":{"additionalDetails":null,"availabilityDomain":"AD1","compartmentId":"ocid1.tenancy.oc1..aaaaaaaaifdc6yaaaa","compartmentName":"simovilmunen","definedTags":null,"eventGroupingId":"49730f57-c653-4ce8-b4c7-8575e070a9d5","eventName":"generateUnifiedAgentConfiguration","freeformTags":null,"identity":{"authType":"instance","callerId":null,"callerName":null,"consoleSessionId":null,"credentials":"ST$eyJraWQiOiJhc3dfeXl6XzE3MTMyMzE2NDYzODciLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJvY2lkMS5pbnN0YW5jZS5vYzEuY2EtdG9yb250by0xLmFuMmc2bGpyaHJ2c240eWM2YWhybXB1bzdsN2lxN29vaXNuampscnJ4a25xZ3JpaGNncjd3cW9jNzJwYSIsIm9wYy1jZXJ0dHlwZSI6Imluc3RhbmNlIiwiaXNzIjoiYXV0aFNlcnZpY2Uub3JhY2xlLmNvbSIsImZwcmludCI6IjI4OkFDOkVDOj

....

Summary

Kcat is useful tool to test connectivity to OCI streaming, maybe you want to validate settings that those are setup correctly or you want to see there are no firewalls blocking the connectivity.