Mark Buchanan cites tech engineer Jaron Lanier's caution about paranoia-optimized advertising algorithms.
… the internet and social media … technology may be acting as a vast optimized engine of social degradation. That’s the argument of a recent book by former tech engineer Jaron Lanier, …
Facebook, for example, makes money by helping advertisers target messages — including lies and conspiracies — to the people most likely to be persuaded. The algorithms looking for the best ways to engage users have no conscience, and will simply exploit anything that works. Lanier believes that the algos have learned that we’re more energized if we’re made to feel negative emotions, such as hatred, suspicion or rage.
“Social media is biased not to the left or the right,” as he puts it, “but downward,” toward an explosive amplification of negativity in human affairs. In learning how to best to manipulate people, tech algorithms may inadvertently be causing mass violence and progressive social degradation.
… studies looking at how different kinds of emotions affect the engagement of online viewers find that messages designed to stir negative emotions including fear or anger tend to work better.
… paranoid messaging taps into deep human emotions and instincts, and therefore tends to get the most attention. Get rid of the advertising model, Lanier notes, and anyone will still be completely free to pay to see poisonous propaganda. It’s just that no one will be able to pay in secret to have poison directed at someone else. That would make a big difference. [emphasis mine]
In his blog Media Democratization and the Rise of Trump, Nicholas Carr discusses Silicon Valley's illusions in this regard.
He wrote an excellent review of Lanier's Ten Arguments for Deleting Your Social Media Accounts Right Now:
. . . Although given to windiness, Lanier is an astute critic, able to see things others miss. But his analysis is distorted by a flawed assumption. He views the problems of social media as “blessedly specific,” resulting from Facebook’s and Google’s reliance on personalized advertising to make money. By closing our social media accounts, he contends, we’ll give Silicon Valley an opportunity “to improve itself” — to retool its business in a socially responsible way. That’s a cheery notion, but it’s naive to think that, if we just hit the reset button, Silicon Valley will reform itself and right its wrongs.