AWS Series: Multi-part series on AWS Cloud and related guidelines
In this multi-part AWS series, I intend to cover the general aspects of AWS in simple terms, the business case for cloud, some deep dives where required, migration strategy, AllOps, security by design framework, reference architectures, and/or demo, and more. I am putting up a Lego bricks approach with multiple layers (in conjunction with the OSI / TCP/IP Layer) and will be adding several Reference architectures (for Web, Batch, Mobile, Data Lake, Big Data, Machine Learning, etc) after assorting and categorizing these Lego pieces. Along the way, I will also discuss the adoption of cloud for different sizes of organizations and building a cloud for scale and how best can make it built to last and at the same time extend it to handshake with other cloud providers to enable Poly Cloud / Multi-Cloud based adoption for the organization.
Layers:
I take the layering approach for designing AWS cloud. These layers are essentially divided into 4 groups (1) The Must have stack (2) The Input Stack (3) The Processing Stack and (4) The Output Stack.
AWS Cloud series includes the below which is loosely mapped to the OSI layer.
Part 1: Cloud Models
1. Understanding and Deep Dive into Cloud Models
Models: IAAS / PAAS / SAAS / Serverless
Deployment Models: Public cloud, Poly Cloud, Multi-Cloud, Private Cloud, Hybrid cloud, Third-party dedicated Data Centres, Shared cloud services.
2. Key differences of various cloud models
3. Migration, SLA, and AllOps of different cloud models
4. Putting a business case for cloud and why it is important?
Links to Part 1 of the Article:
Cloud Deployment Models (Part1 & Part2)
AWS Cloud Migration (Pre-cursor & Migration)
Part 2: Security Layer
This section discusses the following
1. Understanding and Deep Dive of Security Architecture
Login: IAM, Cognito, Secrets Manager, Single Sign-on, MFA, ACM
Web Apps: Guard Duty, WAF, Shield, Certificate Manager
Data: Encryption at Rest (for EBS/EFS volume, Encryption, S3 SSE), Encryption at Transit, TDE, KMS (CMK)
Firewall / ACLs: Security Groups, NACL.
Data Loss Prevention (DLP): Amazon Macie
Governance & Risk: AWS Security Hub, AWS Compliance centre
2. How to build a Security by Design framework and Security-First approach for your organization.
3. How much of a Security Layer is required for the organization
Links to Part 2 of the article:
AWS Series #2: Cloud Security Roadmap
AWS Series #2: AWS Security Layer — Login
AWS Series #2: AWS Security Layer — Network & Web Apps
Part 3: Infrastructure and Network Layer
1. Understanding and Deep Dive of Infrastructure and Network layer
Infra & Network: Region, VPC, Availability Zone
Request / Response: CloudFront, Route 53, API Gateway
Links: Direct Connect | Private Link | VPC Private Gateway
Gateways: Internet Gateway, Storage Gateway, Transit Gateway, VPC Customer Gateway
Load Balancer: ALB / ELB / CLB
2. How to design load balancing across different region and your customers for your organization
Part 4: Sourcing & Streaming Layer
1. Understanding and Deep Dive of sourcing and streaming layer
Sourcing: SFTP, DMS & SCT
Streaming: Kinesis Data Stream, Kinesis Firehose, Kinesis Data Analytics, Kinesis Video, MSK (Managed service Kafka), SQS
2. Distributed system architecture using Streaming
3. How you should choose a Streaming service for your organization
Part 5: Ingestion Layer
1. Understanding and Deep Dive of Ingestion layer
ETL: Glue ETL, Data Pipeline, ETL using a Step function
2. How you should choose a Ingestion layer for your organization
Part 6: Data Stack Layer
1. Understanding and Deep Dive of Database Layer
Part 6a: Database Layer
RDBMS: RDS, Redshift, Snowflake (hosted)
Graph Database: Neptune
NoSQL: DynamoDB, Document DB, Elastic Cache
Query / Analysis: Redshift Spectrum, Athena, Elastic Search
Part 6b: Data Lake Layer
1. Understanding and Deep Dive of Database Layer
Sourcing & Streaming Layer, Data Lake, Querying, Visualization
Part 6c: Big Data Layer
Collection: Kinesis, Kafka | MSK, Greengrass (IoT Core), AWS Snowball, SQS, DMS, Direct Connect
Storage: EMR FS, S3, S3 Glacier, Dynamo DB, Elasticache
Processing: EMR HDFS, Lambda, AWS ML / Sage Maker, Glue, EMR, Data Pipeline.
Analysis: Athena, Elastisearch, Redshift
Visualization: QuickSight
Security: IAM, KMS
Part 6d: Data Science Layer
Amazon Sage Maker, Amazon ML, Rekognition, Polly, Texteract, Log analytics
2. Which Data Stack @ AWS should be chosen for your organization?
Data is in the center and all the applications are built around data in order to facilitate the business with performance and decision making.
Part 7: Configuration, Path, and IAC Layer
1. Understanding and Deep Dive of Configuration and IAC layer
AWS Config, Cloud Formation, Terraform (3rd party), Ansible, AWS System manager
2. Multi-Cloud and Infrastructure as code
Part 8: Application Layer
1. Understanding and Deep Dive of Application Layer
Server based: EC2 | EC2 Image Builder
Serverless: Lambda | Step Functions, Elastic Beanstalk
DevOps / CI/CD Layer: CloudGuru, Cloud9, Code Build, Code Commit, Code Deploy, Code Pipeline.
DataOps, MLOps / AIOps: on AWS using third-party solutions.
Architecture Patterns: Event-driven (Microservices), Time based, etc.,
Containers: ECS, EKS| ECR, ELK, Fargate
Part 9: Consumption Layer
1. Understanding and Deep Dive of Consumption Layer
Web App Delivery: CloudFront
Mobile Applications: Amplify
Database / Querying: Database Layer, Quicksight
Files: Data Lake (S3), Delta Share (Databricks integrated)
Part 10: Backup & Recovery Layer
1. Understanding and Deep Dive of backup and recovery Layer
AWS Backup, Amazon S3 Glacier
2. Backup and recovery for the application server and database server for your organization — best approach.
3. Multi-Cloud as backup and recovery option.
Part 11: Messaging Layer
1. Understanding and Deep Dive of messaging Layer
SNS, SES, PinPoint, Amazon Connect
2. Which is your best messaging layer for your organization
3. How to enable messaging across distributed architecture and multi-cloud / hybrid cloud architecture.
Part 12: Monitoring Layer
1. Understanding and Deep Dive of monitoring Layer
Cloud Watch, Cloud Trail, Trusted Advisor, Compute Optimize, AWS Inspector
2. How to enable distributed tracing and monitoring for microservices design pattern | Containers architecture.
3. How to enable monitoring across multi-cloud / hybrid cloud.
Summary
Link to Azure Series Multi part parent article.
For other articles please check & subscribe to luxananda.medium.com
