Today’s cloud developers have to wrestle with a complicated cloud service landscape: which OLTP database engine(s) to use (e.g., PostgreSQL or Amazon Aurora), which data warehousing solution(s) to use (e.g., Amazon Redshift or Snowflake), and how to glue everything together. End-users of these complex data systems then need to figure out where and how to access the right data for their work. At the end of the day, developers and end-users could not care less about the underlying services and implementations as long as they get the best possible performance for a cost. So then, how can we simplify this complex web of cloud systems for cloud developers and end users?
In this project, we argue that the right approach is to provide a single unified implementation-agnostic application programming interface (API) for both transactional and analytical workloads and to automatically pick the right system architecture and implementation for the end-user. Furthermore, we believe that such an approach gives us the opportunity to automatically instance-optimize the data system for the end-user’s workload thereby providing the best possible performance-cost trade-off in the cloud.
To realize such a vision, we believe that there are two potential approaches, each at opposing ends of a spectrum. We call one such approach the “one interface” direction: optimally leveraging existing user-facing cloud services such as Amazon Aurora and Redshift or Google Spanner and BigQuery to serve both transactional and analytical workloads. Another approach is the “one system” approach: build a new customized database management system architecture that runs on low-level cloud infrastructure (e.g., on virtual machines, serverless functions, or on top of an elastic container service). Our lab is actively exploring both approaches.