article thumbnail

Building a Data Engineering Stack That Boosts Scalability

Chart Mogul

Nowadays, these are packaged and deployed in docker images, and executed via AWS lambda or EKS pods each day. Initially, these were implemented in Jupyter notebooks and manually executed by our analysts whenever they needed to deliver a report. We used our daily AWS RDS snapshots and loaded them daily to Snowflake tables.