3. Existing Solutions

Developers have several options when it comes to incorporating CDC pipelines into their infrastructure. A DIY solution is possible using multiple open-source tools. Enterprise solutions offer the benefit of managing the underlying infrastructure required for a full CDC pipeline, though they come with their own tradeoffs.

DIY Solutions

As many of the tools are open-source, a team that wants to avoid recurring costs or prefers to keep their data in-house may choose to build their own solution. However, many of these tools are complex, and often they are not something that most developers will be proficient in. Oftentimes, the significant time investment makes this choice less desirable.

Enterprise Solutions

There are a multitude of enterprise solutions for Change Data Capture. A prime example is Striim.

Striim provides log-based CDC across a wide array of transactional databases, with the tradeoff of requiring data to be sent to a managed data warehouse. As the majority of Striim’s native connectors are made for managed data warehouses like Snowflake or Amazon Redshift, it is not designed to route data to self-hosted databases. In addition, relying on an enterprise streaming platform means additional costs for data transport as well as costs associated with storing and processing within the data warehouse itself.

Another enterprise solution is ClickHouse Cloud in conjunction with ClickPipes. It allows a team to stream operational data into managed ClickHouse databases. However, as this solution treats ClickHouse as the ultimate destination for this data, it requires additional tools to allow this data to be consumed by other destinations.

Enterprise solutions come with several tradeoffs. Most notably, they are a substantial initial investment, as well as a source of recurring costs. They also come with the drawback of vendor lock-in, and make changing your strategy later much more difficult.