Walt Disney Big Data Architecture Slides.pdf
1.8 MB
Traditional #Lambda architectures ingest live events, transform and cache these #transformations, and then store them in a data warehouse for #batch-based ML training or BI reporting.
Lambda requires two codebases:
1️⃣ Streaming code base is used to optimize for latency
2️⃣ Batch codebase is used for correctness.
However, #Kappa in general, it is easier to maintain a single code base than to keep up with changing business logic for batch and streaming. Many companies embraced Kappa #architecture such as Uber, Linkedin, Disney and Twitter with different adoption levels
Here is The Walt Disney Company journey to Kappa and the lessons learned. it is a good use that identifies the limitations of Lambda and the journey towards Kappa with its challenges.
Lambda requires two codebases:
1️⃣ Streaming code base is used to optimize for latency
2️⃣ Batch codebase is used for correctness.
However, #Kappa in general, it is easier to maintain a single code base than to keep up with changing business logic for batch and streaming. Many companies embraced Kappa #architecture such as Uber, Linkedin, Disney and Twitter with different adoption levels
Here is The Walt Disney Company journey to Kappa and the lessons learned. it is a good use that identifies the limitations of Lambda and the journey towards Kappa with its challenges.
👍10