Does Generative AI “Work”? That’s a Misleading Question.

Source: The New Republic · Bias: Left

Summary

For a decade and a half now, my work has fallen into two categories: collecting evidence on the threat posed by fossil fuels, and deploying written and spoken words to urge action against it. Recently, generative AI systems have entered both of these spheres at a pace I struggle to process.In jobs that depend on analytical rigour, as well as a desire to craft sincere, authentic and honest human communication, the advent of a ubiquitously available plagiaristic machine that convincingly fabricates facts and feelings seems bad, just on its face. But I don’t think the ethical whiplash is the only reason this moment feels so rotten. There are, in fact, some troubling parallels between how fossil fuels operate and how generative AI operates. In the ugly process of sense-making around what is a significant change in how we enact analysis and write words, there has been an exhausting debate around whether generative machine learning “works” or “doesn’t work.” You can find a nice example of this in a December 2024 newsletter by tech writer Casey Newton, in which he slots this fight into two camps: “The first camp, which I associate with the external critics, holds that AI is fake and sucks. The second camp, which I associate more with the internal critics, believes that AI is real and dangerous.”Many reasonable responses to Newton’s piece highlighted the false dichotomy. Plenty of critiques of AI deployment highlight the fact that it tends not to “work” well at the functions it’s marketed for. And that could be perceived as a good thing: As researcher Eryk Salvaggio observed, “systems that don’t work would pose no threat to labor; systems nobody uses would pose no threat to the environment, and systems propped up by a failing industry will collapse—all we have to do is wait.”But here’s the problem: Something can feel like it’s “working” when really the work is subtly worse, and paired with a shocking but invisible array of secondary harms. Fossil fuels have themselves been persistently marketed by lobby groups for decades as not only being effective carriers of energy, but valuable humanitarian pathways for the alleviation of poverty. In fact, fossil fuels “work,” but they also murder their end-users both through air pollution that poisons people, and by stimulating the rapid overheating of Earth’s life support systems. They “work” right up until the moment they don’t, such as the deadly failure of fossil gas during Texas’s 2021 winter freeze, or the crippling global impacts of closing a single 100-mile wide channel in the Middle East. Fossil fuels “work” only with the severest, most narrow definition of “work,” and it’s the same for generative AI.User interactions with chatbots as part of day-to-day professional work can indeed produce what some might consider satisfying answers to prompts, thanks to the fundamental nature of what these systems do. Based on their training data and pattern matching capabilities, generative AI tools can produce responses that are convincingly answer-shaped.If you already have the answer, or deep expertise, you might spot how pattern-matched text output misses nuance, is incorrect, or is misleading in critical ways. But if you have those things, you wouldn’t be asking the chatbot in the first place. The failure mode of generative AI outputs is subtle and insidious, rather than immediately obvious.For some time, I’ve been using different generative AI systems to duplicate—shadow, if you will—work I’m already doing in the course of my normal workday, often in response to a function I’ve seen someone demonstrate online. Sometimes, it delivers both accurate information and reasonable references; similar to a successful Google search. But go beyond narrow, simple factoids, and things go haywire very quickly. When I prodded Anthropic’s Claude to provide some quotes from me, it initially refused on “copyright” grounds—principled, if a bit rich. Then it suggested readers go visit my Substack: a newsletter platform I do not use, and which I have frequently criticized for courting and monetizing viciously racist neo-Nazi groups.I recently heard from a friend in energy analysis that you can extract tabulated data from a chart using Gemini, or Claude. I’ve been doing this using manually-assisted tools for years, but drag and drop makes this a game changer. To test this, I created a chart of year-to-year U.S. power sector emissions from data I already had, and then asked AI tools to do the reverse: generate the data based off the chart. Then I compared the resulting table to the original. The data were close-ish, but with slight differences. I created a chart from the new, reverse-engineered data and ran it again, and repeated that process four times. After four runs, the emissions totals for some years saw a variance of 8 percent, with an average shift of about 2 percent for all the data points.

Related Coverage

More Headlines From March 16, 2026

arallaxNews.io
Does Generative AI “Work”? That’s a Misleading Question.
The New Republic

Does Generative AI “Work”? That’s a Misleading Question.

Left

For a decade and a half now, my work has fallen into two categories: collecting evidence on the threat posed by fossil fuels, and deploying written and spoken words to urge action against it. Recently, generative AI systems have entered both of these spheres at a pace I struggle to process.In jobs that depend on analytical rigour, as well as a desire to craft sincere, authentic and honest human communication, the advent of a ubiquitously available plagiaristic machine that convincingly fabricates facts and feelings seems bad, just on its face. But I don’t think the ethical whiplash is the only reason this moment feels so rotten. There are, in fact, some troubling parallels between how fossil fuels operate and how generative AI operates. In the ugly process of sense-making around what is a significant change in how we enact analysis and write words, there has been an exhausting debate around whether generative machine learning “works” or “doesn’t work.” You can find a nice example of this in a December 2024 newsletter by tech writer Casey Newton, in which he slots this fight into two camps: “The first camp, which I associate with the external critics, holds that AI is fake and sucks. The second camp, which I associate more with the internal critics, believes that AI is real and dangerous.”Many reasonable responses to Newton’s piece highlighted the false dichotomy. Plenty of critiques of AI deployment highlight the fact that it tends not to “work” well at the functions it’s marketed for. And that could be perceived as a good thing: As researcher Eryk Salvaggio observed, “systems that don’t work would pose no threat to labor; systems nobody uses would pose no threat to the environment, and systems propped up by a failing industry will collapse—all we have to do is wait.”But here’s the problem: Something can feel like it’s “working” when really the work is subtly worse, and paired with a shocking but invisible array of secondary harms. Fossil fuels have themselves been persistently marketed by lobby groups for decades as not only being effective carriers of energy, but valuable humanitarian pathways for the alleviation of poverty. In fact, fossil fuels “work,” but they also murder their end-users both through air pollution that poisons people, and by stimulating the rapid overheating of Earth’s life support systems. They “work” right up until the moment they don’t, such as the deadly failure of fossil gas during Texas’s 2021 winter freeze, or the crippling global impacts of closing a single 100-mile wide channel in the Middle East. Fossil fuels “work” only with the severest, most narrow definition of “work,” and it’s the same for generative AI.User interactions with chatbots as part of day-to-day professional work can indeed produce what some might consider satisfying answers to prompts, thanks to the fundamental nature of what these systems do. Based on their training data and pattern matching capabilities, generative AI tools can produce responses that are convincingly answer-shaped.If you already have the answer, or deep expertise, you might spot how pattern-matched text output misses nuance, is incorrect, or is misleading in critical ways. But if you have those things, you wouldn’t be asking the chatbot in the first place. The failure mode of generative AI outputs is subtle and insidious, rather than immediately obvious.For some time, I’ve been using different generative AI systems to duplicate—shadow, if you will—work I’m already doing in the course of my normal workday, often in response to a function I’ve seen someone demonstrate online. Sometimes, it delivers both accurate information and reasonable references; similar to a successful Google search. But go beyond narrow, simple factoids, and things go haywire very quickly. When I prodded Anthropic’s Claude to provide some quotes from me, it initially refused on “copyright” grounds—principled, if a bit rich. Then it suggested readers go visit my Substack: a newsletter platform I do not use, and which I have frequently criticized for courting and monetizing viciously racist neo-Nazi groups.I recently heard from a friend in energy analysis that you can extract tabulated data from a chart using Gemini, or Claude. I’ve been doing this using manually-assisted tools for years, but drag and drop makes this a game changer. To test this, I created a chart of year-to-year U.S. power sector emissions from data I already had, and then asked AI tools to do the reverse: generate the data based off the chart. Then I compared the resulting table to the original. The data were close-ish, but with slight differences. I created a chart from the new, reverse-engineered data and ran it again, and repeated that process four times. After four runs, the emissions totals for some years saw a variance of 8 percent, with an average shift of about 2 percent for all the data points.

Read Full Story at Source