Solving the Mystery of One, then Two, Dead Magnets at Fermilab 

During a routine test, two HL-LHC magnets unexpectedly flatlined. Was it just a coincidence, or did they have a common foe? 

Not exactly murder is involved but there is plenty of sleuthing and baffling dead-ends in “The Magnet Detectives”, a true tale of one, then two, dead magnets amidst some intricate physics at the Fermi National Accelerator Laboratory in Batavia, Illinois. Run by the U.S. Department of Energy, Fermilab is America’s particle physics and accelerator laboratory, tasked with bringing the world together to solve the mysteries of matter, energy, space and time. 

The fascinating tale begins when the reading on the current-meter was gradually climbing. Until it wasn’t. In fact, it suddenly dropped. “Uh-oh,” Giorgio Ambrosio remembers thinking. “Houston, we have a problem.” 

Ambrosio is a scientist at the US Department of Energy’s Fermi National Accelerator Laboratory who oversees the assembly of the US-built magnets for the high-luminosity upgrade to the Large Hadron Collider. By doubling the number of protons inside the LHC and improving the beam dynamics, the upgrade will increase experimental datasets by a factor of 10. This will enable physicists to study rare physical phenomena and learn more about the origin of mass and matter in the universe. 

The HL-LHC upgrade needs 150 new accelerator magnets. In the US, the LHC Accelerator Upgrade Project is building new focusing magnets, which will squeeze the beam just before collisions and will be twice as powerful as their predecessors. But the new focusing magnet that Ambrosio and his US colleagues were testing had failed; the magnet was dead. 

The 5-meter-long electromagnets need to carry 16,530 amperes of current—roughly as much as a bolt of lightning. Because they become superconductors when cooled to very low temperatures, the magnets can carry this huge current without generating any heat. Scientists like Ambrosio introduce and gradually increase the current to test and “train” the magnets before use. 

Ambrosio and colleagues like CERN fellow Alice Moros know that “things not working” is part of the process of science—especially when working with complex, one-of-a-kind, not commercially available technology. “A magnet, like everything else in the world, cannot be perfect,” Moros says. So, the US team put the magnet to the side and started testing another one. Again, the current ramped, and again, it suddenly dropped. They tried to revive it, but the current refused to climb: A second magnet was dead. 

“The worst part is that this happened twice—on two magnets—back-to-back,” Ambrosio says. “It felt like being on the Titanic after it encountered the iceberg.” 

Was it just two unlucky coincidences? Or was a weakness in the magnets’ design about to sink the entire LHC upgrade project? Finding the culprit was vital: The future of the LHC research program depended on it. In this article, we’re going to give you a few clues but to learn the whole story and to find out the ending, you will need to visit Symmetry Magazine, an online publication of Fermilab and SLAC National Laboratory about particle physics, where the story is told. 

Fermilab, focused on solving the mysteries of physics, encountered a mystery of two dead magnets. 

Suspect Number 1 was the cable winding. Each magnetic coil consists of a single cable, which is wrapped 50 times lengthwise around a 4.2-meter-long titanium pole. The cable is delicate because it consists of 40 twisted wires filled with brittle niobium-tin filaments. If one of these wires fell out of place, it could create a pinch point that would compromise the entire coil. In the broken coil, “we had to fix a wire two times during the winding,” Ambrosio says. “So we had the question, did the wire move again?” 

Suspect Number 2 was an unforeseen delay. After winding, technicians transform the raw coil into a functional and sturdy superconducting magnet through heat treatments, applying binding agents, and soldering on components. The winding and curing usually take a couple of weeks, but their victim happened to be one they were working on when COVID first struck. “We started the coil fabrication and then had to stop for 12 weeks,” Ambrosio says. “This is pretty unusual… But did it have an impact?” 

Suspect Number 3 was a broken symmetry. The focusing magnets were quadrupoles, meaning they were each made from four separate coils fitted together to form a long and hollow cylinder. When the magnet is switched on at full current, electromagnetic forces pull the ends of the four coils with a 100-ton force. This is usually fine: If the magnet is perfectly symmetrical, then the forces are equally balanced, and the strong support structure prevents damage to the delicate conductor. But what if they hadn’t been? 

The support structures have small alignment gaps to compensate for the expansion and compression of the material when the magnets are cooled and warmed. Ambrosio and his team saw that the alignment gaps had closed in the broken coil’s support structure but had stayed open in the other three. Could that have condemned the coil? 

A loose wire. An unexpected delay. A broken symmetry. Ambrosio and his team had gathered circumstantial evidence against each suspect. But that wasn’t enough. They needed an autopsy. 

They quickly realized that the only way to get accurate results was to do the analysis the old-fashioned way: inspecting every micron of the samples visually. Crouvizier and Moros spent several days hunched over a microscope. “It’s not so much the neck that hurts—it’s your back,” Moros says. 

After weeks without unearthing any conclusive evidence, Ambrosio had an idea. “We had been cutting [the coil] like salami—perpendicular to the wires—and finding nothing,” Ambrosio says. “But what if we cut it longitudinally—along the wires?” 

A scanning electron microscope revealed broken filaments of Nb35n. Photo courtesy of CERN. 

“Salami” cuts are optimized to look for cracks that travel along the length of the wires. But more sinister damage—a tear that rips a delicate filament in half—is usually invisible to a salami cut. Because longitudinal cuts traverse the length of the wire, they can reveal very small tears. But longitudinal cuts are very tricky to do; the investigating team would have to cut very precisely along sub-millimeter wires without destroying the evidence they were looking for. 

Another clue, but we’re not going to give away the ending. Photo courtesy of CERN. 

Suffice to say, the mystery was solved with collaboration from colleagues at other laboratories around the world. Today, subsequent magnets have a design change and a procedure change. See the full story at www.symmetrymagazine.org