Thanks for reading my newsletter! All opinions expressed here are strictly my own. Find me on TikTok, Instagram and Bluesky. Ultracold publishes every Monday and my debut book Entangled States: A Life According to Quantum Physics is now available everywhere books are sold.
WORLD’S BEST MATHEMATICIANS DISCUSS AI WITH ME
When I first started working as a science journalist five or so years ago, if I came across a scientific study that boasted the use of AI that usually meant that researchers had used a computational tool that was mathematically novel but not necessarily a cause for a big philosophical paradigm shift. These were typically neural networks and machine learning algorithms that could automate some part of the work, for example sifting through lots of data on candidate materials for designing new batteries, like I wrote about in 2024, or identifying synthetic proteins that could have some antibiotic properties, like I described in 2023. When I reported these stories, some new AI programs, such as ChatGPT, had already been launched, but researchers weren’t using the word “agentic” all that much in interviews nor was the siren call of AI agents particularly mellifluous. These programs, which stand out in that you can prompt them to complete a task by using only words, were simply not yet good enough to make scientists want to relinquish almost any part of their work. Very few people that I spoke to were chatting with an algorithm, even fewer were in danger of mistaking one for a co-worker or a friend. Now, things are radically different.
As I wrote when I attended the biggest physics conference in the world this past spring, physicists of all stripes are both captivated by and worried about the rise of AI use in their science. Concurrently, the US Department of Energy has launched an initiative to make AI more accessible and more integrated into American science, the Genesis Mission, and they received more grant proposals than ever before in the Department’s history. This may partly be a consequence of other sources of funding for American science shrinking, but it is still a sign of where scientific research is in the US now, and where it is being nudged to go to. The field that has been undergoing the biggest transformation, however, is a somewhat unlikely one. Despite its status as the most abstract, most challenging, and most in need of human creativity and genius, mathematics has also drawn the most interest from firms that make and sell generative AI models such as ChatGPT and Claude. Consequently, mathematicians have been the loudest about the pace at which this attention is changing science.
Being a physics reporter, I share some interests with my colleagues who report on technology and mathematics both, but I don’t think I could have ignored the absolute torrent of news about AI models resolving dozens of long-standing maths problems even if that hadn’t been the case. Seemingly every week problems that have kept human mathematicians up at night for years are succumbing to someone who is directing an appropriately clever prompt – and sometimes not even that – at one AI agent or another. What does this mean for mathematics and what does it say about the utility or power of AI? The more you look into it, even armed with the skillset of a journalist and surrounded by knowledgeable colleagues, the harder it seems to get all the nuances right. Certainly, I have a lot of clarity on just how negative the environmental impact of AI is and how detrimental it can be to human relationships and learning, but getting to the bottom of how good it actually is at something like mathematics strikes me as less black-and-white.
To do better, I recently took the train from New York City to Philadelphia and attended the International Congress of Mathematicians where AI was yet again the hottest topic among thousands of mathematicians that flocked to the city of cheesesteaks and the beer-and-shot special. I was lucky enough to sit down to discuss AI with two of them: Terrence Tao, who is arguably the world’s greatest living mathematician, and Jacob Tsimerman, who was awarded the world’s most prestigious mathematics prize on the first day of the Congress. They told me two very different stories, both of which speak poignantly about the current state of mathematics, AI and society. I wrote about both for New Scientist.1
For his part, Tsimerman is leaving a career in mathematics, which has been incredibly successful and has now been anointed as one that could transform the field, to work on AI safety at OpenAI, the makers of ChatGPT. We spoke for almost an hour and I was struck by his candor and what seemed to be genuine altruism. He told me that he has not found the same joy in researching AI that he finds in mathematics yet, but that he feels compelled to make the pivot because of how great the risks are. Emphasizing the dangers of AI benefits the companies that make and sell these programs as it underscores their power, but hearing such concern coming from one of the best mathematicians of his generation rather than a corporate talking head did rattle me a little.
When I pushed back on some of the “doomy” language that AI researchers and evangelists sometimes use, Tsimerman challenged me on whether I’d have similar criticisms of someone who was warning about the dangers of nuclear weapons. The analogy is not perfect, and I am still not sure whether I believe that AI could lead to “omnicide,” as Tsimerman has explored in past work, but I came out of the conversation with a reaffirmed belief that this technology really does need to be a lot more regulated. Even if it does not destroy the world, its many other detrimental effects call for it, and Tsimerman argued that there ought to be stringent testing of each new AI model. This should be done through international collaboration, not unlike some nuclear weapons treaties of the past, he asserted. I can only hope that he’ll be as effective in bringing this about as he has been in pushing mathematics forward.
My conversation with Tao was a different flavour of existential but no less important. A veritable child prodigy who started attending college-level mathematics classes before he was a teenager and has improved numerous fields of mathematics since, Tao is uniquely positioned to take stock of how mathematics works and what is happening to it. He has always been versatile and experimented with new tools and ways of working and is in no way a maths purist or a conservative, despite his remarkable talent and renown. With all of that under his belt, Tao thinks that AI has plunged mathematics into a deep crisis of practices and values.
He candidly spoke to me about how technology companies are trying to co-opt and reshape the narrative of what mathematics is and what the role of mathematicians is, how mathematicians should be talking to philosophers of science more, how the culture of chasing prestige has made mathematics ripe for a crisis, and how there must be new emphasis on mathematics being about creation of knowledge rather than just knocking out theorems like pins in a bowling alley. Though Tao is remarkably analytic and pragmatic and could present his arguments with the exactness and precision you’d expect from a mathematician, while talking to him I heard echoes of conversations I’ve been having with other writers and creatives. Taos’ worries about mathematics becoming a field that is fully goal-oriented and all about competition and productivity reminded me of arguments that favour using AI for writing instead of first struggling through draft upon draft of bad writing, which is crucial for building skill. This made me worried. It also greatly inspired me to hear the best mathematician in the world say that it’s important to organize, get political, and fight for the heart of the profession that you love.
Where did I come out at the end? I am far from having all the answers but I continue to believe that generative AI is a technology that should be used only selectively, with extreme discernment, and only after extensive training in traditional methods for research and art. I don’t doubt that there are use cases for AI in science that are valid and justifiable, but most are currently not at all clear cut and a lot more rigorous and replicable research is needed to clarify them. I also believe that AI should be regulated much more tightly, from more scientific and transparent testing to exponentially more emphasis on harms it does to the planet and people. And I still believe in the intrinsic value of human ingenuity, idiosyncrasy and messiness. As both Tsimerman and Tao told me, much of the work of a mathematician, or any scientist really, is in failing over and over again, until something clicks, until some moment of creativity and resolution happens. In my view, when we shortcut that process by using a machine, we deprive ourselves of something extremely human, we become less rich in both cognition and spirit.
This may sound biased, and it is as it is just my own human opinion. In the two stories I wrote, you can read about two exceptional humans’ opinions and what they base them on as scientists at the forefront of their field. I hope that these conversations inspire some new thinking about what the process of meaning-making and knowledge production is currently like and what it ought to be going forward, not just in mathematics but for all of us.
Best,
Karmela
If you do not have access to New Scientist but want to read these stories, please reach out