Treehouse Dataflow Toolkit (TDT) is a set of Lambda-based microservices that automate, scale, and manage the transfer of data from mainframe sources and Kafka pipelines into cloud-based analytics, machine learning, and AI platforms such as Amazon Redshift, Snowflake, Amazon Athena/S3, Amazon S3 Express One Zone, and Amazon Aurora PostgreSQL. TDT is designed to provide near-real-time synchronization, support for both historical and current data, and rapid deployment through AWS CloudFormation templates. It ensures compliance with best practices for security and scalability, and minimizes the time and resources needed for enterprises to modernize and leverage legacy data for advanced analytics.

Key Features

Automated Data Transfer Automates data movement from mainframe/Kafka to cloud analytics targets.

• Event-driven, Lambda-based microservices architecture
• Supports bulk-load and change data capture (CDC) processing

Cloud-Native and Scalable Built for cloud environments with high availability and scalability.

• Auto-scalable microservices
• AWS CloudFormation templates for rapid deployment

Comprehensive Data Synchronization Maintains both historical and current data for advanced analytics.

• Delta tables retain complete data history
• Self-materializing views for up-to-the-second snapshots

Security and Compliance Implements best practices for cloud security and resource management.

• Least-privilege IAM policies and roles
• Optional automated VPC and networking setup

Multi-Target Support Supports multiple analytics and AI/ML-friendly cloud data stores.

• Amazon Redshift, Snowflake, Amazon Athena/S3, Amazon S3 Express One Zone, Amazon Aurora PostgreSQL

Benefits

Accelerated Analytics Modernization Enables organizations to quickly leverage legacy data for analytics and AI.

• Reduces time-to-value by automating complex data integration tasks
• Eliminates months or years of manual development and configuration

Reliable, Near-Real-Time Data Availability Ensures data is always current and historically complete for analytics.

• Near-real-time updates for analytics and ML workloads
• Historical data retention supports trend and predictive analysis