Making a Modern Data Centric Organization — Part 4

Define a good Modern Data Technology Infrastructure

Last but not least,

To create a “MODERN DATA ORGANIZATION”

it is imperative

to choose the right data infrastructure.

1 The first trend is

Foundational

Data as a mainstream where the data related technology/processes has reached the plateau of productivity as per Gartner’s terms. Once thought dead is back into business due to cloud and modern data platforms. As part of this trend, Data Warehouse is back with modern cloud databases, Big Data is revived with Spark, increasing use of Artificial Intelligence with multiple PAAS and SAAS providers.

Gaining maturity: In Finance and health care, Data protection, security and privacy is not a new concept. While for the internet organizations the data protection and security is evolving.

2 The second trend is

Trusted

With Internet organization’s focusing on Data protection/security/protection, I would like to highlight “Trusted” as the initial trend — Ringfencing Data from generation till archival and. beyond.

In my opinion Trusted trend comprises of

1. Differential privacy — To cut the story short, differential privacy is about adding “noise” to the data, yet make it meaningful.

2. Authenticated / Data Provenance (aka lineage)

One of the upcoming trends is the “Authenticated Provenance” and how do you know the data is real and valid when it is created. Authenticated provenance is part of “Algorithmic Trust”, Blockchain helps to track the origin. This is especially useful for niche and costly and unique products in place or extremely sensitive/critical information that flows through. Garbage-in is Garbage-Out — Data Provenance is the key to understand the source and authenticity of data.

3 The third trend is

Strategic

The core of the Strategic trend is “Simplification of Data”. As with the modern data platforms, more and more organizations want to store and retrieve information from a single area and be able to have a simple yet efficient data architecture and platforms.

Data warehouse a big come back on Data Lake: eg., Redshift & Data Lake (Glue and S3) in…