629 questions
0 votes
1 answer
64 views
IIDR CDC Kafka - EntraID / OAuth2
Can someone tell me if the IBM InfoSphere DataReplication for Kafka (CDC 2 Kafka) is capable to communicate to a Kafka cluster/broker that needs authentication via Microsoft EntraID / OAuth2 with ...
-3 votes
1 answer
111 views
Can USB CDC receive data byte-by-byte in CDC_Receive_FS(), or is it always packet-based?
I’m working with an STM32 microcontroller using USB CDC (Virtual COM Port) and handling incoming data in CDC_Receive_FS(). I want to understand the behavior for UART-style communication: If the host ...
0 votes
2 answers
96 views
What are safe patterns for Service A to consume data from Service B without direct DB access? [closed]
I’m looking for practical ways to let Service A get the data it needs from Service B without querying Service B’s database directly. I’m not very experienced with distributed systems yet, so I want to ...
1 vote
1 answer
237 views
IIDR CDC Kafka Custom KCOP disable schema auto register
we are trying to replicate Db2 changes from z/OS to Confluent Kafka. Everything works out fine for the production of simple JSON records in Kafka. As our policies for Kafka Topics and Schemas are ...
0 votes
2 answers
136 views
How to capture delta records in Snowflake when a table is dropped and re-created every day?
I have a pipeline that drops and re-creates several Snowflake tables every day (effectively a full refresh using CREATE OR REPLACE TABLE). I want to capture daily deltas (inserts/updates/deletes) for ...
0 votes
0 answers
63 views
about EMQX Enterprise 6 Action Sink to Kafka cluster
I am a newbie in EMQX architecture. Recently, for test purposes, I deployed an EMQX enterprise 6 single node, with Kafka producer connector integration (Action Sink). The compatition matrix of the ...
0 votes
0 answers
80 views
Flink CDC + Hudi isn't working as expect, found log said state is cleared
I'm using Flink CDC + Apache Hudi in Flink to sync data from MySQl to AWS S3. My Flink job looks like: parallelism = 1 env = StreamExecutionEnvironment.get_execution_environment(config) ...
0 votes
1 answer
164 views
Why am I getting a null row when querying a CDC function in an ADF Lookup activity?
I am trying to build an ADF pipeline that incrementally updates a table using Change Data Capture (CDC). And I'm running into an issue when I try to use a CDC built in table function in an ADF Lookup ...
0 votes
0 answers
49 views
Debezium initial snapshot on demand
I have a debezium 3.0 in my python project. Postgres-debezium connector has the following structure: { "name": "dbz_name", "config": { "connector.class&...
2 votes
1 answer
184 views
Debezium + Flink Oracle CDC - "db history topic or its content is fully or partially missing" for some tables
I am using Flink with Debezium to consume CDC changes from Oracle DB tables via LogMiner. For some tables, everything works fine. For example, the following table works without issues: CREATE TABLE ...
0 votes
2 answers
131 views
how have CDC on redis?
I'm using CDC services like (Debezium) on my Mongo or Postgres but somehow I came up with situation that I need have to CDC on Redis . for example get streams of event occur in Redis like adding new ...
3 votes
1 answer
678 views
How to perform real-time Postgres to Redis sync?
I want to sync a Postgres table (2 million records) to Redis. Requirements: Full table should sync initially After that, realtime insert/update/delete should sync to Redis automatically (no polling) ...
0 votes
0 answers
54 views
MongoDB custom CDC handler using PySpark
I want to replicate a collection and sync in real time. The CDC events are streamed to Kafka and I’ll be listening to it and based on operationType (insert/delete/update) I’ll have to process the ...
0 votes
0 answers
74 views
how to add NEW tables to the mysql source kafka debezium connector
I am using AWS MSK cluster and I have created a MySQL debezium connector that runs on EC2 instance and reads from a specific table, and it is working fine, but it could be the use case where the user ...
2 votes
0 answers
31 views
In windows, I can't see usb cdc descriptions
I'm developing a new project by using stm32 MCU and USB is one of most important features in my project. But unfortunately I always see manufacturer as "Microsoft" and comport name as "...