Livoa LogoLivoa
Prod
DEV
50GB
DEV
100-150GB
STAG
API
API
extract Face Feature
extract Face Feature
TODO:


1. DDL (mydid-db-photo-cdc)

2. daily CDC (15,000 rec)

3. timeline

4. compress on INSERT (photo)

5. archive TABLE (old & DELETE)

6. cleanup procedure (cron)

7. monitoring (queues & count)

TODO:


1. DDL

2. masa untuk extract Face Feature

3. timeline

4. extract by tier

5. benchmarks

TRIGGER


insert

update

delete

CDC

snapshot / initial load
25 mil

1mil / 1 thread & 1 GPU
18:32 31st Oct (C.70)
8:00 1st Nov
C.70 GPU 1 (20%)
C.71 GPU 2 (80%)
Development env (Spark Worker)
disk space left (GB):
1. prod-postgres-image-1: 225
2. prod-postgres-image-2: 135
3. prod-postgres-image-3: 185
Showstoppers: 1. disk space set to 0 2. isu CDC (tiada gambar / gambar tak aktif) 3. high availability & connection pooler (3 DB)
TODO: 1mil, 32 threads & 2 GPU (with JK)

Face Feature Vector JPN

by Rizal

0
0 uses