RubiX is a lightweight data caching framework that can be used by big data engines such as Hadoop. Rubix uses local disks to provide the best I/O bandwidth to the big data engines. Rubix is useful in shared storage architectures where the data execution engine is separate from storage. For example, on public clouds like AWS or Microsoft Azure, data is stored in cloud storage and the engine accesses the data over a network. Similarly, in data centers, Presto runs on a separate cluster from HDFS and accesses data over the network.
RubiX can be extended to support any engine that accesses data using the Hadoop FileSystem interface via plugins. There are plugins to access data on AWS S3, Microsoft Azure Blob Store, and HDFS. RubiX can be extended for use with any other storage system including other cloud stores.