article thumbnail

Building a Data Engineering Stack That Boosts Scalability

Chart Mogul

The aim was to ease data access across the company and allow data engineering functions to scale. We enabled logical replication AWS RDS instances, deployed Debezium and other Kafka connectors in our Kubernetes cluster which consume all real-time changes in Postgres tables, and stored them in relevant Kafka topics in our AWS MSK cluster.