John Harris calls attention to "how the internet and the tiny handful of companies that dominate it are affecting both individual minds and the present and future of the planet."
… Tristan Harris, a former high-up at Google …
… if we’re not careful, we will soon be at risk of being locked into mindless behavioural loops, …
… listen to Tristan Harris and his campaign. “Religions and governments don’t have that much influence over people’s daily thoughts,” …“But we have three technology companies” – he meant Facebook, Google and Apple – “who have this system that frankly they don’t even have control over … Right now, 2 billion people’s minds are already jacked in to this automated system, and it’s steering people’s thoughts toward either personalised paid advertising or misinformation or conspiracy theories. And it’s all automated; the owners of the system can’t possibly monitor everything that’s going on, and they can’t control it.” [emphasis mine]
Frank Pasquale warns that major digital firms are displacing government roles, exerting regulatory control in their fields. Gradually companies like Amazon and Google are emerging as functional sovereigns as territory-based nation states weaken.
In functional arenas from room-letting to transportation to commerce, persons will be increasingly subject to corporate, rather than democratic, control.
..., they aspire to displace more government roles over time, replacing the logic of territorial sovereignty with functional sovereignty. [order changed]
Charlie Stross fills in control-you details:
Apple have put a lot of effort into making realtime face recognition work with the iPhone X. ... it does depth mapping to ensure your eyes are in the right place (and can tell whether they're open or closed) and recognize your face from underlying bone structure through makeup and bruises. It's running continuously, checking pretty much as often as every time you'd hit the home button on a more traditional smartphone UI, and it can see where your eyeballs are pointing. ... it means your phone can monitor your facial expressions and correlate it against app usage. Your phone will be aware of precisely what you like to look at on its screen. With addiction-seeking deep learning and neural-network generated images, it is in principle possible to feed you an endlessly escallating payload of arousal-maximizing inputs. It might be Facebook or Twitter messages optimized to produce outrage,... ..., the app now owns your central nervous system—and you will be monetized.
In sum, already we have over two billion people hooked up into a mostly-automated interconnected corporate system, more tightly leashed than by allegiance to countries or religions. This ascending power base has addiction-level control based on deep learning and neural-network technologies. A handful of companies have taken substantial control of our present and future, without anyone paying attention.
Marshall McLuhan said that new technologies change our perception and thought without us noticing.