Understanding NewSQL Databases

1. Overview

Databases are one of the ways we store collections of data and there are different types of databases available. Different DBMSs, such as SQL and NoSQL databases, became popular based on the requirements (performance, consistency, etc.) and the type of data (structured, schemaless, etc.) to be stored. We now have something called NewSQL Databases which combine the best of both SQL and NoSQL databases.

In this tutorial, we’ll take a look at SQL and NoSQL databases and then understand what NewSQL databases are all about.

2. SQL and NoSQL Databases

For decades, traditional SQL databases have served as the foundation of data storage and retrieval. SQL Databases provide robust ACID compliance, guaranteeing dependability, consistency, and data integrity. Some of the popular use cases include OLTP and OLAP applications and Data Warehousing applications.

On the other hand, database requirements changed along with the digital landscape. With the rise of the internet came an abundance of data, as multiple sources produced huge amounts of data instantly. Despite their consistency and dependability, traditional SQL databases found it difficult to handle the demands of these fast-moving data streams.

As a result, NoSQL databases emerged as viable alternatives. NoSQL databases put performance, scalability, and flexibility above ACID compliance. They use several data models (document, key-value, column-family, etc.) that enable them to perform well in particular use cases, including networked systems, real-time analytics, and unstructured data storage. Social media applications and document-oriented applications are some of the most common use cases.

While NoSQL databases offered a solution to the scalability problem, they came with trade-offs, most notably relaxed consistency models. In scenarios where strong data consistency and transactional guarantees were essential, NoSQL databases fell short. This led to the need for a new kind of database system which could have the best of both, SQL and NoSQL.

3. NewSQL Database

NewSQL databases try to address the limitations of the existing databases. They’re engineered as a relational database with a distributed and fault-tolerant architecture. They aim to provide solutions by providing a database with the following features:

• Scalability and ACID compliance: Designed to scale horizontally, the NewSQL databases handle large amounts of data by distributing them across nodes/clusters. Additionally, they maintain strict ACID compliance resulting in a system that is highly available and with strong transactional integrity.
• Performance optimization: Various techniques, such as in-memory processing, indexing, and caching, are implemented to provide low-latency data access and high performance.
• Distributed architecture: Multiple nodes are used to replicate data so that there is no single point of failure. Consequently, it ensures high availability and fault tolerance
• SQL compatibility: NewSQL systems are compatible with SQL query language thus avoiding re-learning and migration overheads

3.1. Use-Cases

NewSQL databases are most suited for applications requiring strong transactional consistency along with high performance and scalability. Let’s take a look at some of the use cases below:

• Financial systems: A large amount of data needs to be processed with low latency and high accuracy
• E-commerce platforms: Although the load is relatively stable and expected, periodically there may be a surge of data/load that needs to be supported
• Real-time feedback systems: Many systems that rely on real-time data analysis like airline pricing, fraud detection, etc. can benefit from a high-performant transactional system.
• Smart devices and cities: With increased automation across multiple sectors and use cases, the continuous data stream can be processed efficiently using a NewSQL database.

However, there are some use cases where the NewSQL database may not be suitable:

• Well-established data models: Applications with well-established data models with manageable load and/or scalability capabilities may not be suitable candidates for migration to NewSQL databases, e.g., legacy applications
• Predictable loads: Applications with predictable loads that may not need to leverage dynamic scalability
• Strict ACID compliance: SQL databases better serve applications with non-negotiable ACID properties

3.2. Popular NewSQL Databases

Let’s take a look at some of the popular NewSQL databases:

• CockroachDB: An open-source distributed database designed to survive different types of failures while still maintaining ACID compliance. It uses distributed architecture and provides strong failover capabilities along with automatic data replication.
• NuoDB: It uses a patented “elastically scalable databases” architecture to provide NoSQL benefits while retaining ACID compliance.
• VoltDB: An in-memory database that uses a shared-nothing architecture designed for high-velocity data ingestion.

3.3. Drawbacks

While NewSQL databases address the limitations of SQL and NoSQL databases, they come with their own set of drawbacks:

• Use case specific: Different NewSQL databases are suitable for different use cases and there is no single solution for all use cases
• Complex learning curve: Since NewSQL databases tend to be use case specific, each new implementation may require a non-overlapping learning curve
• Compatibility issues: NewSQL databases may not be always compatible with existing data models and schemas which may lead to considerable migration efforts

4. Conclusion

In this article, we explored the evolution of data storage and retrieval from traditional SQL to NoSQL databases and then finally to NewSQL databases. We also looked at the different use cases for NewSQL databases as well as some of the popular NewSQL databases.

NewSQL databases bridge the gap between SQL and NoSQL databases by combining transactional consistency with scalability and performance. We saw some of the use cases where it may be beneficial to use any of the enumerated NewSQL databases.