In the wake of the pandemic, one trend that emerged is the need for digitalisation, and this will continue – and even grow – in the future. And, while it’s still not clear exactly what that actual economic impact of Covid-19 will be, IDC predicts that the digital economy will accelerate to counter this. The firm predicts that over 65% of APAC GDP is expected to be digitalised between 2020 and 2023 and spending will hit US$ 1.2 trillion.
As businesses – and even economies – become more digitalised, there’s no escaping that data and the ability to gain insights from it are at the heart of every successful organisation – and the foundation of any successful insight-led architecture will remain the database.
In this article, I will discuss five database trends that can help businesses and economies to surge ahead in the wake of the pandemic.
Over the past ten years, databases and big data have distanced themselves in their functionality. One is responsible for online workloads, while the other covers batch processing vast quantities of data. However, from the perspective of applications, with a single system that can solve the whole process of data generation, processing, storage, and consumption, it can help create a better online experience for customers as they navigate retail websites and services.
From the retailer’s perspective, to improve performance, they need to synchronise several million transactions every day; each bringing with it several different data types as they improve recommendations, and consequently, the experience for online shoppers. However, by achieving online and batch workload integration or HTAP (hybrid transaction/analytical processing), these problems can be easily solved. And this trend will continue to grow over the next decade, which is why Gartner classifies OPDBMS (Operational Database Management Systems) and DMSA (Data Management Solutions for Analytics) into one Cloud DBMS market, and this is the core logic behind the integration of the two fields.
It is a well-known fact that more and more organisations are adopting cloud infrastructure. They are not only moving their generic business process applications on to the cloud, but also their mission critical applications and data.
According to recent industry research, 75% of all databases will be deployed or migrated to a cloud platform by 2022. This means that organisations are in need of cloud services and solutions that will support their cloud native architecture. In fact, business organisations are already developing or sourcing cloud native applications, as it allows the business processes to be more dynamic.
Combined with distributed processing techniques, without heavy infrastructure set up, users can enjoy the elasticity and high availability brought by cloud native databases with ease and efficiency.
Cloud computing has changed everything because it has fueled the growth of data. But, we are still far from real AI. We use deep neural networks today and they need large-scale data to be really useful. AI is a black box today, but AI tech used as heuristics has worked. It has made a mark in computer vision and speech recognition, for example.
Now, it is making a mark in databases too. We will have self-driving databases in the future, and our roadmap is to fully automate a database. The complexity in automating databases arises because usage changes from customer to customer, which makes it tough to automate the entire process.
However, we can use AI for common scenarios. For example, we can help different workloads from e-commerce or traditional systems to tune system parameters to improve their latency and scalability and use ML algorithms to ensure that databases are secure and running without anomaly.
It’s hard to imagine that there was a time when there were other database models besides a relational one. However, there are more types, including document- oriented, graph database, time series, triple store, etc. In a rapidly digitising world, arriving at insights from heterogeneous data will continue to remain a challenge as businesses seek to derive the most out of newer advances in technology such as AI, IoT, and beyond.
Databases will use cloud native design to decouple storage and compute, and be compatible with a wide variety of open-source standard interfaces. They will also support switching between open source systems and seamlessly connecting with multiple computing and analytical engines.
In addition, wide-column table and time series models will be supported, as well as the storage and analysis of structured, semi-structured and unstructured data. This will deliver improved performance for IoT, especially for the multi-model data store, computing and analytics of device metadata, device operation data (time series data) and device logs.
Once a buzzword and now a fundamental requirement of any application, interface, product and business is security. In the cloud era, maintaining the highest levels of trust and transparency, while securing data in all its forms, will remain a focus for some time to come.
Techniques like encryption in motion with TDE (Transparent Data Encryption), flexible key management, encryption at rest including SSL, traceability and auditing are mandatory and evolving with better efficiency.
In time, what will be significant will be how to combine blockchain technology to provide immutable modification in the database system, and how to leverage new hardware to encrypt data and process encrypted data without decryption inside databases.
