What Are AI Data Centers? How They Power the AI Boom

By Muhammad Ramzan Aatish

Here is a number that reframes the AI boom: the racks inside a modern AI data center can draw over 135 kilowatts each. A generation ago, a typical rack sipped about 20. That single fact, straight from NVIDIA’s own engineers, explains the trillion-dollar construction spree, the water fights in small towns, and the moratorium bills piling up in legislatures. So what are AI data centers, exactly? Not bigger versions of the buildings that host websites, factories. Thousands of graphics processors wired to behave like one machine, engineered for the most power-hungry manufacturing process of this decade: training and running artificial intelligence. Every prompt you send to ChatGPT, Claude, or Gemini passes through one of these buildings. Here’s what they do, why they drink so much power and water, who is building them, and why your neighbors might already be fighting about one.

What Are AI Data Centers, Really? Factories for Intelligence

An ordinary data center is a big, secure building of servers running websites and cloud apps. An AI data center rebuilds that idea around one punishing job: thousands of GPUs, processors first built for video games, working on the same calculation in tight synchronization, for weeks or months on end.

NVIDIA calls these facilities “AI factories”, more honest than it sounds. A regular data center juggles millions of small tasks; an AI data center runs a few enormous ones: teaching models by repeating the same math billions of times, then answering millions of prompts in seconds. As Crusoe’s CEO Chase Lochmiller puts it, the foundation of AI starts with energy and data centers. The software looks weightless. The buildings are anything but.

The Two Jobs Every Rack Is Built Around

Everything inside serves two workloads. Training is how a model learns: it guesses over massive datasets, measures the error, and adjusts its internal settings through backpropagation, repeated until it gets good. Inference is what happens when you use the model: one forward pass, no learning, tokens back in seconds.

The training vs inference distinction decides where these buildings go. Training rewards concentration, thousands of accelerators acting as one computer, so it clusters in enormous campuses where power is cheap. Inference rewards proximity, it sits closer to users to cut delay. McKinsey expects inference to become the dominant AI workload before the decade is out, so the geography of training vs inference is about to spread out, not concentrate. Want the GPUs without the gigawatt campus? See the comparison of the best cloud GPU providers for AI.

Not Just Bigger: A Different Species of Building

Honestly, calling both of these things “data centers” is like calling a bicycle and a freight train “vehicles”, technically true and useless. The comparison that matters:

The load-behavior row is the one power engineers lose sleep over: AI training ramps tens of thousands of processors simultaneously, then drops just as fast, multiple times a minute. Equipment built for gradual changes now absorbs shocks that damage machinery and shorten asset life, so batteries have become the shock absorber between the volatile load and slow-to-respond generation. The IEA calls onsite battery storage “a critical technology for the next generation of AI data centres.” From the grid’s point of view, that is what are AI data centers: the most badly behaved large customer the power system has ever met.

Cooling Is the Whole Ballgame

At 120 kilowatts per rack, air cooling hits a wall: those racks would need below-freezing air or near-gale winds, per NVIDIA. So in liquid cooling data centers, coolant is piped to within centimeters of the chips, or servers are dunked in engineered fluid. NVIDIA claims 300 times the water efficiency of air-cooled designs, with 45°C coolant so simple fans often replace power-hungry chillers. Vendor figures, but the direction is not in dispute.

The water story is where marketing thins out. Rystad Energy estimates data centers consumed 222 billion liters for cooling worldwide in a single recent year, potentially nearly tripling by decade’s end without adaptive measures. Cutting water usually means spending more electricity on dry cooling, researcher Andy Masley notes. The big operators are improving: Microsoft and AWS report water-use efficiency gains of 25% and 37% over three years, both claiming closed-loop systems with no net water loss. But the hidden water behind a facility’s electricity can be twice its own direct consumption in the US. A campus that evaporates nothing can still be thirsty through the power plant that feeds it.

Power, Not Chips, Is the Bottleneck Now

A couple of years ago the constraint was GPUs. Now it is the grid. The IEA’s latest analysis found data center electricity demand soared 17% in a single year against 3% global growth, and expects data center power use to roughly double by 2030, AI-focused facilities tripling. In the US, Lawrence Berkeley National Laboratory puts data centers at about 4.4% of national electricity, rising to 6.7 to 12% within a few years. AI data center power consumption is no longer a side statistic in energy planning. It is the plan.

The money agrees: five large tech companies pushed capital spending past $400 billion in one year, set to rise 75% the year after. McKinsey puts the world on pace for $6.7 trillion in data centers by decade’s end, $5.2 trillion AI-specific, about 60% chips, while warning the figure only materializes if AI finds real business value. Treat the trillions as a scenario, not a prophecy.

Constrained by slow grid connections, developers improvise: tech signed roughly 40% of corporate renewable power deals in a recent year, small-modular-reactor deals nearly doubled from 25 to 45 gigawatts in eighteen months, and many US projects are advancing with onsite natural-gas generation. As IEA chief Fatih Birol puts it: there is no AI without energy.

Who Is Pouring the Concrete

The flagship is Stargate, the private AI-infrastructure program backed by SoftBank, OpenAI, and Oracle, envisioned at up to $500 billion, with $100 billion committed immediately. Its first Stargate AI data center campus, rising in Abilene, Texas on Lancium’s site and built by Crusoe, shows what “AI factory” means: eight buildings, 1.2 gigawatts, each able to run up to 50,000 NVIDIA GB200 GPUs on a single fabric, cooled by direct-to-chip liquid cooling with zero water evaporation. An $11.6 billion financing package funds the expansion. Crusoe separately closed a $3.9 billion round at a $30.9 billion valuation, read what that round signals about the AI data center boom.

Following the backlash coverage, the detail that stuck was a surrender: after local opposition killed its proposed Wisconsin campus, Microsoft published a “Community-First AI Plan”, authored by vice chair Brad Smith, pledging fair electricity pricing, grid upgrades, full tax payments, water stewardship, and community investment wherever it builds. IDC’s Mary Johnston Turner called the moratorium wave a confluence of affordability, quality-of-life, sustainability, and economic concerns. The Stargate AI data center program keeps expanding, into a country that has started negotiating the terms.

The Backlash Is Local

This is what the fastest-growing searches are really asking about: the environmental impact of AI data centers, and who pays it. Costs land in specific places while benefits diffuse everywhere, and specific places are pushing back. Sanders and Alexandria Ocasio-Cortez proposed a federal moratorium until national safeguards exist; 36 US projects were delayed or blocked in one twelve-month stretch, disrupting an estimated $162 billion in investment; Maine advanced a temporary ban. One industry CEO puts public sentiment at six-to-one negative in some regions.

In the Wall Street Journal’s Hazle Township, Pennsylvania reporting, the number that lands hardest is $10,000, what a developer offered every household if its data center gets built, plus $120 million in community payments over fifteen years. It did not work. A retired foundry worker: “we don’t need that around here.” A homeowner organizing the opposition: “are they trying to bribe us here?” The township rejected the project on zoning, the developer sued, and a temporary moratorium went into effect. The industry keeps selling abundance; the zoning hearings keep telling a different story.

The counterpoint deserves airtime. The IEA found that rising electricity demand does not have to raise prices, but that data centers create special affordability challenges, since their loads are large, concentrated, and scale fast enough to trigger new generation and grid investment. Whether your bill moves depends less on national statistics than on how your state’s regulators assign those costs. Less satisfying than the headlines. Also the honest answer.

What to Watch, and What You Can Actually Do

Most readers will never tour one of these buildings. Their effects are reaching towns, utility bills, and water tables anyway. What is genuinely useful:

• If a campus is proposed near you, show up at the zoning hearing. Demand three things: a closed-loop cooling commitment, full tax payments instead of abatements, and written protection for residential electricity rates.
• Watch grid-connection news, not chip announcements. Interconnection queues and permitting fights decide which campuses get built, the IEA names them explicitly.
• Don’t carry personal guilt over individual prompts. Per-task power is falling fast; the honest lever is the providers’ efficiency race, not your chat history.
• Read vendor sustainability claims as claims. “No net water loss” at the facility says nothing about the hidden water behind its electricity. Ask where the power comes from first.

So what are AI data centers, in the end? Factories, running the most power-hungry manufacturing process of the decade, built at a speed the grid was never designed for, in towns only now learning they get a vote. The AI boom looks like software from the outside. Underneath, it is construction, copper, water, and megawatts.

FAQ