Case Study: Let us take a simple example, Company X’s Indonesia Branch, Source, and Ingest Data from 3 regions (of the many). The company has a central Data Warehouse and the BI and ML consumption layer is across multiple regions. They run several businesses and is an e-commerce firm — Internet Organization. The volume of the business, number of users, types of applications, sharing and use of data are discussed in the below points.
Massive data input: Data sourcing and ingestion from 100’s of places out of which the below listed 3 regions load petabytes of data as these are…
This article is a part of a multi-part series Data Technology Trends (parent article). Previous part — Data Technology Trend #6: Actionable outcomes and Next part — Data Technology Trend #8: Data Next — part 1.
Data can be best monetized if the organization can build Data & AI marketplace and exchange platforms are it for internal or external use.
1. Strategy: Choosing a plan for the organization to create a Data value strategy
2. Improving data monetization capabilities — Building strong data pipelines, data platforms.
3. Designing the extendible and live information solutions
4. Continuously improving and generating value…
In this multi-part AWS series, I intend to cover the general aspects of AWS in simple terms, the business case for cloud, some deep dives where required, migration strategy, AllOps, security by design framework, reference architectures, and/or demo, and more. I am putting up a Lego bricks approach with multiple layers (in conjunction with the OSI / TCP/IP Layer) and will be adding several Reference architectures (for Web, Batch, Mobile, Data Lake, Big Data, Machine Learning, etc) after assorting and categorizing these Lego pieces. …
We are in the information age where we have abundant data. Every organization is generating massive amount of data and wants to easily access data on-demand preferably from a single place. Getting more value from the Data quickly with the highest quality is increasingly becoming a challenge for many organizations whatever the size the organization is.
With tremendous data growth in the organizations,
I am taking the approach to talk about what are the bare minimum steps required to get Microservices working and moving ahead as you scale. The previous part of this series “Microservices: Know why & Know how”.
The very first question the Technology team should ask themselves before jumping into Microservices is, how this idea of rolling out Microservices came about? Is it a solution for which we are looking for a problem? or genuinely over years, we have seen how the business struggled because of the monolithic, giant, and complex legacy architecture that kept delaying the release cycles and…
Get phase — Discovery, the As-Is — current technology, people, and process.
This article is a part of a multi-part series Modern Cloud Data Platform War (parent article). Previous part — Modern Cloud Data Platform War — DataBricks (Part 4) — Machine Learning and Analytics.
Let us assume Company X is using on-premise Hadoop several variants such as Cloudera Hadoop, Apache Hadoop, etc. leverages commodity hardware for large-scale distributed systems involving several 100’s of nodes. For the processing layer, they have been using Map Reduce for the past 8+ years and the firm is spending several millions of dollars every year.
Unlike the previous DataBricks article, I am not just…
This article is a part of a multi-part series Modern Cloud Data Platform War (parent article). Previous part — Modern Cloud Data Platform War — DataBricks (Part 3) — Data sharing.
Different types of Machine Learning algorithms run on these massive data sets from Recommendation engines to fraud detection etc.
DataBricks provides a very good unified stack that enables your organization to have a very efficient Lakehouse architecture. …
This article is a part of a multi-part series Modern Cloud Data Platform War (parent article). Previous part — Modern Cloud Data Platform War — DataBricks (Part 2) — Data Fluctuations.
Massive loads of Data Sharing: Another case for the same firm is that it has to share loads of data with other organizations — say every month-end they transfer 100 PB of data over.