Zillow Just Gave Us A Look At Machine Learning's Future

From Vin Vashishta discussing the implosion over at Zillow:

For Data Scientists, there are several lessons. You’d better understand the science before taking on significant projects. When revenue starts getting booked against Machine Learning projects, model reliability is critical. Research methodology is essential. When revenue growth starts being built around Machine Learning capabilities, those capabilities need to be comprehensive. The body of evidence supporting core models needs to be substantial. Even when the burden of accountability rests firmly on senior leadership, it’s the Data Science team’s jobs that are on the line.

The other good lesson is that no matter how cutting-edge your team is, you can always be made to be a scapegoat for bad management. In reality, you probably don’t have much say in that scenario, except for when you are picking your employer.