The need for digitalization started in the wake of a pandemic. This will continue and even grow in the future. The Statista Research Department predicts that the spending on technologies and services to enable digital transformation worldwide is expected to reach more than two trillion US dollars. As businesses and even economies become more digitalized, the foundation of any successful insight-led architecture will remain the database. Li Feifei, Vice President of Alibaba Group explained four database trends that can help businesses and economies to surge ahead in the wake of the pandemic. Let’s have a look at the details of these database trends.
Hybrid Transaction/Analytical Processing (HTAP)
Over the past ten years, databases and big data have distanced themselves in their functionality. One is responsible for online workloads, while the other covers batch processing vast quantities of data. However, from the perspective of applications, a single system that can solve the whole process of data generation, processing, storage, and consumption, can help create a better online experience for customers as they navigate retail websites and services.
From the retailer’s perspective, to improve performance, they need to synchronize several million transactions every day; each bringing with it several different data types as they improve recommendations, and consequently, the experience for online shoppers. However, by achieving online and batch workload integration or HTAP, these problems can be easily solved. And this trend will continue to grow over the next decade, which is why Gartner classifies OPDBMS (Operational Database Management Systems) and DMSA (Data Management Solutions for Analytics) into one Cloud DBMS market, and this is the core logic behind the integration of the two fields.
Cloud-native architectures and distributed processing techniques
It is a well-known fact that more and more organizations are adopting a cloud infrastructure. They are not only moving their generic business process applications onto the cloud but also their mission-critical applications and data.
According to recent industry research, 75 percent of all databases will be deployed or migrated to a cloud platform by 2022. This means that organizations need cloud services and solutions that will support their cloud-native architecture. Business organizations are already developing or sourcing cloud-native applications, as it allows the business processes to be more dynamic. Combined with distributed processing techniques, without heavy infrastructure set up, users can enjoy the elasticity and high availability brought by cloud-native databases with ease and efficiency.
Intelligent and self-driving databases
Cloud computing has changed everything because it has fueled the growth of data. But, we are still far from real AI. We use deep neural networks today and they need large-scale data to be useful. AI is a black box today, but AI tech used as heuristics has worked. It has made a mark in computer vision and speech recognition, for example. Now, it is making a mark in databases too. We will have self-driving databases in the future, and our roadmap is to fully automate a database.
The complexity in automating databases arises because usage changes from customer to customer, which makes it tough to automate the entire process. However, we can use AI for common scenarios. For example, we can help different workloads from e-commerce or traditional systems to tune system parameters to improve their latency and scalability and use ML algorithms to ensure that databases are secure and running without anomaly.
In a rapidly digitizing world, arriving at insights from heterogeneous data will continue to remain a challenge as businesses seek to derive the most out of newer advances in technology such as AI, IoT, and beyond. Databases will use cloud-native design to decouple storage and compute and be compatible with a wide variety of open-source standard interfaces. They will also support switching between open source systems and seamlessly connecting with multiple computing and analytical engines.
Besides, wide-column table and time series models will be supported, as well as the storage and analysis of structured, semi-structured, and unstructured data. This will deliver improved performance for IoT, especially for the multi-model data store, computing and analytics of device metadata, device operation data (time series data), and device logs.