POD! On YIMBY Pod this week, Martin and I do our best impression of Tim Harford on More or Less, debunking some questionable Guardian reporting on data centres. Then we speak to James Howat from ThinkLabour, who argues that we can build more infrastructure if we bring back municipal bonds. Listen here!
GAME! How many MPs can you remember? Try my stupid little MP memory game to find out!
My partner asked me what I want for Christmas the other day.
This is a surprisingly difficult question. We’re middle-class and childless, and though we’re not particularly wealthy in any meaningful way, this does mean that if there’s something reasonably priced that we want, like a book or a video game… we just buy it. We don’t wait for a special occasion, because why bother?
So I sat there thinking for more than a few moments, my mind blank, when all of a sudden something depressing hit me. Could she buy me some more AI tokens?1 Does ChatGPT sell gift cards?
As bizarre as it may seem to most people, a lack of AI compute is, with only a little exaggeration, one of the biggest problems in my life. ChatGPT has proven so useful to me as a knowledge engine, software development platform and work assistant, that I’ve recently found my supply of AI tokens running dry, to the point where, even though I pay for the $20/month ‘Plus’ tier, it eventually refuses to respond to my queries for several hours – or even days.2
From a business perspective, this makes total sense for OpenAI. Though an app on a computer screen may feel weightless to the user, the computation to make the magic happen is extremely resource-intensive. If you prompt ChatGPT, in order to compute the answer, something roughly equivalent to an entire rack of computers in a data centre briefly dedicates itself to your query. That’s why AI energy usage is a significant environmental concern, as well as a bottleneck for development – it’s a non-trivial expense that the AI companies need to pay for.
But from my perspective, knowing this does not make it less annoying. Though there is a way around this – I could simply pay more. ChatGPT would love me to upgrade to the $100/month tier for a 5x increase in the number of AI tokens I can use.
However, here I hit a different roadblock. I can’t really afford to spend $100/month on AI compute. I mean, I suppose I could – but at that level it becomes a middle-class spending trade-off, where I’m forced to choose between my precious tokens and, say, subscriptions to Netflix, Amazon Prime, and a myriad of other subscriptions. And I should probably continue to pay my half of the mortgage, for the sake of my relationship and having a house to live in.
Anyway, it was when I was experiencing this great pain that I realised something important. Though my dilemma was simply annoying, I think in the not too distant future it could become an important challenge for society.
Because what happens when this problem hits people who are not as fortunate as I am? If the most optimistic predictions about AI are true, and it really is world-changing in its utility, and if compute remains a constrained and expensive resource – what will limitations on AI usage mean for the poorest people in society? Won’t it be unfair if the rich can buy all of the compute they need, and the poor are left without?
In other words… Am I crazy to think that AI could soon become a social justice issue?
Power to the processors
I wrote recently about how AI has broken the link between human brains and cognition.
Though we can debate at exhaustive length about exactly which tasks AI can perform, or the limits of its ‘intelligence’, it’s increasingly true that human-like cognition can be spun up on demand with a credit card.
This is a weird analogy, but it’s a bit like the transition to cloud computing in the mid-2000s. In the before times, companies and individuals needed to make huge capital outlays on computing equipment – essentially placing bets on their future needs.
But then cloud arrived and smoothed the correlation between demand and costs. When more people want to use your app, the cloud platforms automatically spin up more computers to meet the demand, and when fewer people use it, it scales down again.
And we’re arguably already witnessing the same happen with ‘intelligence’. The amount of smarts you will have to dedicate to any given task will be the sum total of both your brain and however much AI compute you have access to.
In other words, there will be a similarly smooth correlation between intelligence and how much you can afford to pay.
So wind the clock forwards and the implications of this are scary. The rise of AI could end up having significant distributional impacts, and could become a source of significant inequality.
Now, I know what you’re thinking. “Finally O’Malley gets it! That naive AI booster is conceding that the hectoring critics have been right all along!”
So let’s be clear. I’m not talking here about the potential AI impact on jobs, intellectual property or any of the other well-worn criticisms of the technology. Those are different debates for other pieces.
What I’m talking about specifically is access to AI intelligence – especially if the most optimistic Silicon Valley predictions about AI turn out to be true.
This is because if AI is as transformative as the Tech Bros hope, the technology will inevitably be woven into all aspects of our lives. If intelligence is available on tap, it’s plausible that we’ll all essentially have AI agents running in the background of our lives and this will be completely normal – a bit like how today it is basically impossible to function in society without a smartphone.
What will these agents be doing? It would be silly to speculate on what specific products might emerge. Though, for the sake of giving you something to picture, imagine a virtual personal assistant that is like Siri on steroids. Something that can proactively work on your behalf, without needing prompting. It could manage your money, organise your calendar, negotiate with the government over your benefits, or perform ten billion other tasks basically autonomously to reduce the friction in your life.3
Whatever conveniences end up being invented will require compute resources – and those resources will be subject to the same dynamic: Those with more money will have access to more compute, and therefore will be able to do more stuff, further entrenching a society of haves and have-nots. And unless we plan for it, it could turn out that not having access to compute is just as debilitating as living without a phone is today.
Uncertain assumptions
Obviously, my argument here is highly speculative. It rests on several assumptions about the future coming true. Perhaps the AI sceptics will be proven right.
But even if the technology does prove maximally useful, there are other examples where analogous to this where any problems have simply disappeared.
For example, when my parents first got a dial-up internet connection in the 90s, access to the internet cost a penny per minute, which meant that time online was severely rationed. I couldn’t spend an evening wasting my life, scrolling listlessly because there was a permanent ticking clock running, and it could add up to serious expense.4
But eventually, this problem mostly disappeared as the market shook out so that instead today everyone pays a flat fee, in exchange for internet usage that is effectively unmetered, no matter how long we spend on Geocities, or whatever it was we did in the 90s.
The same happened to mobile internet usage too. At the beginning of the iPhone era, phone contracts severely limited everyone’s data allowance, but eventually mobile infrastructure caught up and the cost of data collapsed, which is why today most people can afford a package that is either unmetered, or large enough for data limits not to really matter.5
So there is a strong argument that AI will likely go the same way. The reason you can easily hit a hard limit on ChatGPT and Claude today is that the availability of compute is severely limited. As soon as the world builds more data centres (as is happening), the theory is the price of compute will fall and AI use will be effectively unmetered.
And then there’s the related argument that in the future, most AI compute will happen locally – your AI data crunching won’t happen in a distant data centre, but on the chip inside your phone or computer. That would mean no need to pay a subscription, and you can use AI as much as you like for free.6
I readily concede that, maybe, this is a possible future too. However, I must admit, I’m a little sceptical in the medium term, because of the huge gulf in capability between local models, designed to run on home devices, and the latest frontier models that run in the cloud. Maybe I just lack the imagination, but given the hardware and energy requirements, I’m not sure how all of that can be shrunk down into a smaller model that can work just as effectively on significantly less capable hardware.7
But anyway, my point is that though these are two possible futures – there is a third. And this is where the danger lies. What if the cost of compute doesn’t fall, or if the price equilibrium lands somewhere inconvenient and out of reach of many people?
We got lucky with broadband and mobile – it’s so cheap that most people can afford a home internet connection and a SIM card with a reasonable data allowance.
But what if AI is so unbelievably useful to the rich and to corporate users that the cost of compute for people on low incomes, or small businesses just starting out, is just too out of reach? So that when rich and poor compete for access to this new, scarce resource, there’s only one winner?
I don’t think this is as hard to imagine as it might seem, as we’ve already seen a microcosm of what this might look like, and the victims are some of the least sympathetic people on Earth.
I’m talking, of course, about gamers.
A significant consequence of the AI boom is that because data centres require similar chips, RAM and other components to the computers in our homes, manufacturers have dedicated their capacity to providing for data centres – sending the cost of home computing spiralling.
For example, earlier this year Apple raised the price of the entry-level MacBook Air from £1,099 to £1,299, and amazingly the cost of a new Playstation 5 today is £569.99 – 27% higher than when the console was first released six years ago.
This is completely unprecedented. Typically what you’d expect to happen is for consoles, phones and computers to get cheaper over time as devices age and components become cheaper to make. But cold business logic dictates that spending on AI is just much more lucrative, which has left everyone else paying over the odds for hardware.
So my point is – what if we end up in a similar position in terms of access to compute? What if the marginal value of a little bit of extra compute is much higher to a large corporation to the point where it makes compute unaffordable for regular people? And what if this isn’t a problem that goes away quickly like we might expect as supply catches up to demand?
Similarly, I don’t think this is as unlikely as it might sound.
Though the current AI build-out is often described as a bubble, the record investment could simply be a reflection that this really is a world-historic, valuable technology. And perhaps we are starting to see the first signs of this. Anthropic is now forecasting that it expects to bring in upwards of $100bn in the next year. That’s not hype – that’s cash from customers who want to buy the AI capabilities that the company is selling.8
And I think demand for compute could still conceivably go even further too. As I wrote previously, this is a new general-purpose technology – there are countless tasks to which this new resource of cognition-on-demand can be applied. And as Ben Thompson has argued, what will drive this demand for compute – or ‘inference’9 – isn’t just humans typing queries, but AI agents working autonomously – compute begetting more compute without human intervention:
Today’s agents are fancy answer inference; in the future true agentic inference will be work done by computers according to dictates given by other computers, and the market size scales not with humans but with compute.
Given that agentic computing is already exploding, I think it is at least plausible that we will reach a point where it is far more valuable for corporations to pay a premium for marginal compute, creating the market failure I describe above. And it’s not a given that just because demand will be there from individuals, supply will catch up – because building isn’t without friction, as data centres are hugely controversial and subject not just to market forces, but the arbitrary whims of planners and other political interests.
And if this sounds familiar, it’s because though I’m describing a hypothetical problem for AI, there’s another sector where the corporations and individuals are already locked into a competition for the same scarce resource that is subject to the same dynamic: energy.
As things stand, if electricity was a pure free market, we’d build vastly more energy infrastructure to reduce price and service demand, but energy infrastructure is subject to other constraints. And likewise, if domestic energy prices were set by the market, they would be significantly higher, and the prices would disproportionately punish the poor.
That’s why, in the case of energy, the government created the energy price cap. Though it doesn’t completely delink household energy costs from the market, it does attempt to mitigate it to reduce the burden.
So is it that crazy to imagine something similar being required to ensure AI compute is shared fairly too?
Left behind?
So that’s my wild speculation on how access to compute could conceivably, in the not-too-distant future, become a social justice issue. Like I say, perhaps it won’t happen – perhaps one of the other futures will emerge, and availability of compute will be so commoditised this simply doesn’t become an issue.10
But if this is what the future looks like, and AI does live up to the hype, if you care about social justice and poor people, then this is something everyone – particularly those on the left of politics – is going to have to think about.
And so far, I don’t see much thinking along these lines. Much of the left-of-centre response to AI so far has been either outright denial of its capabilities11 or has been stuck in the intellectual dead end of “do we even want this?”, as though the LLM genie can somehow be stuffed back into the bottle. Then more credibly, there are more serious leftist critiques that are worth taking seriously, such as the impact AI will have on jobs, or how it will shift power from labour to capital.12
But what has been missing from all of this is any discussion of how to manage AI if the most optimistic expectations are right. We’ve invented this incredible new technology, capable of so many potentially positive things – but how can we ensure that the fruits of artificial intelligence are fairly shared?
I suspect the reason for this blind spot is mostly because, for people on the left, it is faintly embarrassing to talk about such a sci-fi idea, and not to mention politically awkward to contemplate given other ideological commitments. To entertain the idea that compute is a social justice issue is to implicitly concede that the hated Tech Bros will have created something good for the world.13
But I do think issues like this have a habit of seeming like a joke until suddenly they are very much not.
It wasn’t so long ago that you would have sounded mad for worrying about the moderation policies of a social platform for university students – but now those same rules help decide elections. Likewise, the micro-blogging platform where you used to post jokes and memes has helped smash the cordon sanitaire between the mainstream and fascist right. And that funny video app where people post dancing videos and comedy skits? It’s handed the Chinese Communist Party the tools to shape the minds of younger generations across the West.
So if AI does turn out to be as transformative as many people expect it to be, we should start thinking now about how to ensure that access to compute does not further entrench inequality in society, and so that everyone can share the benefits of this new technology.
I’d ask ChatGPT how to get started with this, but I’m all out of tokens.
This is the unit by which AI usage is typically measured. A ‘token’ isn’t an individual character, but, well… it’s complicated.
Technically it doesn’t outright refuse to work. When you run out of usage, it bumps you down to a more basic model, which crucially cannot use more advanced functionality like using plugins or calling tools.
What I’m describing here has almost, sort-of, if you squint, been invented. Meta recently launched Muse, a personal assistant that does something like I describe, in the US. On the surface, it’s a cuddly avatar and a bizarre Tamagotchi-style device, but under the hood it’s basically a virtual machine running an LLM that you can hook up to your email and other services, that can act on your behalf.
Also I guess at a certain point my mum might have wanted to use the phone.
Weirdly before the iPhone era, I actually briefly had an unlimited data contract on my Nokia N95, presumably because it was inconceivable how anyone could possibly use so much data before video streaming and so on.
Apart from the extra energy draw adding to your electricity bill.
Compare Siri AI, which is a distilled version of Google’s Gemini model, which works locally on iPhone. It’s definitely better than the old Siri – but it’s utterly woeful compared to the latest frontier models from OpenAI and Anthropic, which only work in the cloud.
The only reason Anthropic and OpenAI aren’t posting enormous profits every quarter now is because they’re reinvesting most of the cash they generate in training future models, in order to stay ahead.
Inference is the computation that is done when your prompt is fed to a model to elicit a response.
It’s also worth noting that it’s pretty unlikely that ‘compute’ will be a cost that is passed on to households directly, like a gas or electricity bill. But it will definitely become something that plays into the pricing of different products that will determine our standard of living and what we can do with our lives. (A bit like how iCloud has differently priced tiers of cloud storage that determine how many photos you can store in the cloud).
Amazingly the ‘stochastic parrots’ people are still going, despite the evidence in front of our eyes.
If workers derive their power from being able to withdraw their labour, that won’t work as well when cognition is available on tap, no humans required.
I think this is similar to how there’s a persistent strain of leftist thinking that is hostile to ‘Big Pharma’ – it’s difficult to concede that capitalism has developed useful new drugs when your ideology says that capitalism is bad.
Very interesting and thought provoking article. Evidently, like a great many technologies, AI is a mixture of good and bad. In the hands of good actors, then yes, AI can have unbelievable benefits. But what happens when it gets into the hands of bad actors, though admittedly, it all depends on perspective, as what's good to one set of people may not be so good to another.
The article raises the concerns of social inequity and energy usage, both if which are likely to increase as AI becomes more and more embedded in our lives. These are definitely my two greatest, though by no means the only, concerns. How are we possibly going to wean ourselves off fossil fuels, at least to net zero, if we continue to build energy guzzling infrastructure such as massive data centres?
There are doubtless many different centricities in our World, for example those who think purely of the benefits such as the money AI will make them, but disregard the issues, no doubt thinking either that they don't apply at all, or only apply to others, not themselves. There will need to be some serious debates around the World to discuss all of this, and somehow try and strike a balance which benefits as many people as possible. But despite all this, it is inevitable that some, perhaps a great many, people will lose our. Perhaps our whole planet will eventually lose out if we ignore the effect that all this energy guzzling will have on climate change. Just a thought.