Livoa LogoLivoa
DETAILED SERVERLESS DATA LAKE ARCHITECTURE
DATA SOURCES
📡 IOT DEVICES (AWS IOT CORE)
☁️ AWS (/Mobile Apps)
📱 Web/Mobile Apps
📄 Files
🗄️ Relational Databases (RDS)
INGESTION & PROCESSING LAYER
BATCH / COLD PATH
REAL-TIME (HOT PATH)
📡 Amazon Kinesis Data Streams
🖥️ Amazon Managed Service Apache Flink
🔍 AWS Glue Data Catalog
👤 AWS Glue ETL (Cleaned (SERVERLESS SPARK) Lake)
🗄️ Amazon S3 (Raw/Landing Lake)
⚡ Amazon (SERVERLESS SPARK)
📊 Amazon CloudWatch (Monitoring & Logs)
Retils Data
Batch Data
INGESTION & PROCESSING LAYER
CONSUMPTION LAYER
📊 Amazon Athena (SQL Analytics)
🗃️ Amazon Redshift (Data Warehaase)
📄 Train/Predict
(ML Platform)
SQL Query
SQL Data
SQL Query
OPERATIONAL / SECURITY / GOVERNANCE
🔐 AWS IAM (Authentication/Authorization)
🏞️ Amazon Lake (Data Governance)
🏞️ Amazon Lake (Data Governance)
📜 AWS CloudTrail Audit Logs
⚙️ S3 Intelligent-Tiering (Cost Optimization)
Policies/Roles
Access Control (Data Governance)
Orcheates ETL / Audit Logs
Lifecycle Management

archi

by akash

0
0 uses