Apache Pegasus

Apache Pegasus is a distributed key-value storage system designed to be horizontally scalable, strongly consistent, and high-performance. It fills the gap between in-memory systems like Redis and large-scale stores like HBase by offering persistent storage with low latency and robust consistency guarantees. Written entirely in C++, Pegasus uses RocksDB as its underlying engine and relies on the PacificA consensus protocol for replication and fault tolerance.

Features

• Strong consistency via PacificA consensus protocol
• Hash-based partitioning for horizontal scalability
• Three-way replication for data durability
• Automatic load balancing across ReplicaServers
• Cold backup and restore using local or distributed filesystems
• Eventually-consistent intra-datacenter replication (duplication)
• Online partition splitting for dynamic scaling
• Simple and well-defined APIs for client integration

Capabilities

• Supports high-throughput read/write operations with low latency
• Enables safe and fast data migration between data centers
• Provides fault-tolerant storage with multi-level data safety
• Facilitates real-time analytics through Pegasus-Spark integration
• Offers operational flexibility with rebalance and snapshot tools
• Compatible with distributed environments and cloud-native architectures

Benefits

• Bridges the performance-consistency gap between Redis and HBase
• Enhances system reliability with robust replication and backup
• Reduces operational complexity through built-in automation
• Improves scalability and availability for large-scale applications
• Supports both online and offline data processing scenarios
• Backed by the Apache Software Foundation for open governance