Drastically Reducing Out-of-Memory Errors in Apache Spark at Pinterest
Felix Loesing | Software EngineerIn 2025, we set out to drastically reduce out-of-memory errors (OOMs) and cut resource usage in our Spark applications by automatically identifying tasks with higher memory demands and retrying them on larger executors with a feature we call Auto Memory Retries.Spark...
AV1 — Now Powering 30% of Netflix StreamingLiwei Guo, Zhi Li, Sheldon Radford, Jeff WattsStreaming video has become an integral part of our daily lives. At Netflix, our top priority is delivering the best possible entertainment experience to our members, regardless of their devices or network condit...
Dmitry Kislyuk | Director, Machine Learning; Ryan Galgon | Director, Product Management; Chuck Rosenberg | Vice President, Engineering; Matt Madrigal | Chief Technology OfficerForeword from Bill Ready, CEOThe AI landscape is undergoing a fundamental shift, and it’s not the one you think. The competi...
Post-Training Generative Recommenders with Advantage-Weighted Supervised Finetuning
Author: Keertana Chidambaram, Qiuling Xu, Ko-Jen Hsiao, Moumita Bhattacharya(*The work was done when Keertana interned at Netflix.)IntroductionThis blog focuses on post-training generative recommender systems. Generative recommenders (GRs) represent a new paradigm in the field of recommendation syst...