First, humans are bad at discerning the value of their data. Personal data just appears out of nowhere, exhaust out of life’s tailpipe, so why not trade it for something small? I’m personally willing to hand over my own GPS location full time just so I have access to Angry Birds and a smartphone flashlight app.
As machine learning on top of personal data is used to dissect us more, the odds of someone breaking out of marginalized society decrease. Models say they’re a credit risk, so they’re fed terrible loans. Poverty is reinforced. Data says they’ll like Skittles, so they’re advertised Skittles. Obesity is reinforced. And rainbows.
It's hard for me to imagine that anyone not working in a field that relies on data aggregation and analysis understands how accurate a data-driven hypothesis can be. There are multiple fire-hoses of data available and it's not just marketing companies looking for that data. The consequences may not just be the design of ads we see, but the kinds of external forces are applied to all aspects of our life.
I think we have passed the point of hypothetical dystopian futures.