Apache InLong

Apache InLong is a one-stop, full-scenario integration framework for massive data. It supports data ingestion, synchronization, and subscription, offering automatic, secure, and high-performance data transmission. Originally evolved from TubeMQ, InLong integrates the entire lifecycle of data collection, aggregation, storage, and processing, enabling real-time analytics and modeling across diverse industries such as advertising, finance, gaming, and AI 

Features

• SaaS-based service platform for easy data publishing and subscription
• High-performance processing for trillion-level data streams
• Real-time ETL and rule-based data sorting
• Integration with various message queue systems
• Unified system monitoring and alerting
• Fine-grained metrics for data visualization
• Pluggable architecture for modular expansion

Capabilities

• Supports batch and stream data processing
• Handles diverse data sources including files, databases, and MQs
• Offers modules for collection (Agent), transmission (DataProxy), queuing (TubeMQ), transformation (Sort), and management (Manager)
• Compatible with major data nodes like Kafka, MySQL, PostgreSQL, Hive, Hudi, Iceberg, ClickHouse, and more
• Real-time auditing and reconciliation across system components
• Lightweight and standard architecture options for flexible deployment

Benefits

• Simplifies complex data integration workflows
• Enhances reliability and scalability for enterprise-grade data pipelines
• Reduces operational costs with efficient data handling
• Enables real-time decision-making through streaming analytics
• Supports rapid development and deployment with modular design
• Backed by a strong open-source community and proven in production environments