The Distinction Between Imaginary Science and Magic
Ted Chiang received the Humanist Inquiry & Innovation Award at the American Humanist Association’s 83rd Annual Conference, held virtually in September 2024. This award honors those who have advanced human understanding and innovation in ways that uphold humanist values, work that exemplifies the power of inquiry and innovation to promote human dignity, freedom and progress.
Chiang is a renowned science fiction writer. His fiction has won four Hugo, four Nebula, and six Locus Awards, and has been reprinted in Best American Short Stories. His first collection Stories of Your Life and Others has been translated into twenty-one languages, and the title story was the basis for the Oscar-nominated film Arrival. His second collection Exhalation was chosen by The New York Times as one of the 10 Best Books of 2019.
This text is excerpted form Mr. Chiang’s acceptance speech at the Conference.
THANKS VERY MUCH to the American Humanist Association for this award.
Rather than give an acceptance speech, I’m going to give a talk, which I’ve given a couple times in the past on the topic of magic, and imaginary science in fiction.
Arthur C Clarke famously said that any sufficiently advanced technology is indistinguishable from magic, and that statement has been quoted so often that a lot of people will treat it as a fact. I agree with Clark’s statement, in one sense: I think that technology will one day enable us to do things that currently seem out of reach. However, sometimes people interpret Clark’s statement as saying that there’s no fundamental difference between advanced technology and magic, and this is something I fundamentally disagree with.
Some people will say that if you don’t understand how something works, it might as well be magic to you. For example, a lot of people don’t know how their smartphones work, and so for them, smartphones are basically magical devices. I don’t think that this is a useful definition of magic. I don’t want to say that radios are magic to some people, but not magic to others. I don’t think the criterion for whether something is magic should be whether I can find someone who is baffled by it in casual conversation.
We might use the word magic as a synonym for amazing, but I’d like to see if it’s possible to come up with some more objective criteria. I most often hear the claim that there’s no difference between advanced technology and magic when someone is suggesting that the difference between science fiction and fantasy is primarily cosmetic.
According to this position, if a story is set in the future and the characters carry devices adorned with buttons and blinking LEDs, it’s science fiction, and those devices run on imaginary science. But if the same story is set in the past, and the people will carry devices that are adorned with jewels, then it’s a fantasy story, and those devices run on magic. I’m going to return to discussing science fiction versus fantasy a little later on, but for now, I want to address the question of these cosmetic differences.
Let me describe two hypothetical stories for you. In the first story, a technique has been discovered for transmuting lead into gold, and the process is now being performed on an industrial scale. Gold is so cheap that it has ceased to become a symbol of wealth. Gold is used everywhere you need a metal with high conductivity, but it’s also used everywhere people previously used lead because gold is non-toxic. So now, when people go fishing, they use gold sinkers to hold down their fish hooks, and when you get your teeth x-rayed at the dentist, you wear an apron lined with gold for protection.
In the second story, it is also possible to transmute lead into gold, but there are just a handful of individuals who are able to do it. Only people who were born with a certain predisposition are candidates, and even they have to practice for years in order to be able to perform the procedure properly. They can only transmute a little at a time, because it is so exhausting. Maybe they can create enough gold to make themselves wealthy, but they cannot make enough to flood the market and bring down the price of gold.
To me, the first story feels like a science fiction story, while the second feels like a fantasy story, and it has nothing to do with cosmetic differences. Note that I didn’t say that one story was set in the future, while the other was set in the past. You could set both of these stories in roughly the present, and I would still say that there’s a difference between the two. Similarly, I did not describe any of the equipment used in the process, so the difference has nothing to do with whether a device has blinking LEDs or jewels on it. And, of course, the difference is not because I think one story is possible and the other one is not. In both stories, lead is being turned into gold, and while that is theoretically possible to do with a particle accelerator, it would cost at least a quadrillion dollars per gram. So, for the purposes of this discussion, we’ll just say that both of these stories are describing the impossible.
The reason these stories feel different to me has to do with the way these stories treat this impossibility. One feels like it’s a story about an imaginary scientific discovery, while the other one feels like it’s a story about magic. I think that the difference in the way these stories feel points to something significant. If I’m told that a phenomenon is dependent on the practitioner, that makes me think it’s magic, because think of the ways that magic is commonly depicted.
Sometimes magic works only for people born with an innate gift. Sometimes magic only works for people who have purified their soul through years of study. Sometimes magic only works for people who have good intentions, or it works differently for different people, depending on whether their intentions are good or bad. Sometimes magic requires intense concentration to be effective, or it requires that you make a sacrifice.
None of these things are true of scientific phenomena. When you pass a magnet through a coil of wire, electric current flows no matter who your parents are or whether your intentions are good or bad. You don’t have to concentrate hard or offer a sacrifice in order for a light bulb to turn on. Electricity does not care.
One of the central criteria for a scientific result is that it be reproducible, that it be it can be recreated anywhere by anyone. It does not depend on a specific person’s presence or participation. If an experiment only works when one particular person conducts it, then we discard that data as spurious. When radio waves were first discovered, they might have seemed magical to the casual observer and the scientists who first transmitted messages via radio might have seemed like wizards because they were able to communicate over long distances using an invisible medium, but because radio waves are reproducible, because they don’t rely on any particular person’s participation, it eventually became possible to build radio transmitters and receivers by the thousands and then the millions, and now, literally anyone can use a radio. It is no longer restricted to a handful of scientists.
In that first hypothetical story that I described, the clear implication is that the procedure for transmuting lead into gold is similarly reproducible. It does not rely on any particular person. So it’s possible to have factories all over the world churning out gold by the ton. There is an important respect in which that process behaves like a scientific phenomenon, even though, as we said, transmuting lead into gold is impossible.
I first started talking about this difference between science and magic several years ago, and initially, the way I phrased it was to say that scientific phenomena are egalitarian with regards to people, while magical phenomena are preferential. For example, in the Harry Potter universe, there are clearly two classes of people. There are magic users who are special and muggles who are ordinary. By contrast, the universe that we live in does not divide people into classes like that.
Then someone asked me, “What if the class of magic users contains everybody? If magic spells work for every single person in the world, does that mean they’re science?” And to me, the answer was still no. That feels like magic, so I had to rephrase. So here’s my revised formulation. Not only does a scientific phenomenon make no distinction between people, it also makes no distinction between people and mechanisms. You don’t need a person to pass a magnet through a coil of wire in order to generate electric current. A windmill or a water wheel can do the job just as well. But magic always requires a human being for an incantation to work. A person has to recite it. You cannot use a tape recorder or play an mp3 for a spell to work, a person has to make the hand gesture to cast it. You can’t use a bunch of sticks driven by levers and pulleys.
“Magic is evidence that the universe knows that you’re a person… The scientific worldview is one where the universe is purely mechanistic. A magical worldview is one where the universe responds to you in a personal way.”
Here’s another way to say it: Magic is evidence that the universe knows that you’re a person. Magic is an indication that the universe recognizes that people are different from things and that you are an individual who is different from other people. Yeah, some people say that the scientist’s way of viewing the world is cold and impersonal. I’m not sure that I would agree that it’s cold, but I would agree that it is impersonal.
The scientific worldview is one where the universe is purely mechanistic. A magical worldview is one where the universe responds to you in a personal way. Let’s go back to the example. Transmutation of lead into gold in medieval alchemy was a physical analog of the spiritual purification that the alchemist himself underwent. Transmuting a base metal like lead into a noble metal like gold was a reflection of the alchemist’s soul being transformed from the base into the noble. Spiritual purification of this sort is not something that happens with a furnace.
There are parts of medieval alchemy that were purely impersonal and did not rely on the practitioner’s spirit, and those are what eventually became chemistry, the parts of alchemy that had a personal component, that relied on some aspect of the practitioner spirit are the ones that didn’t work and were eventually discarded. This transition from alchemy to chemistry is just one example of a much broader trend. We have moved away from believing in a personal universe to realizing that we live in a mechanistic one.
In the past, most people assume that we lived in a personal universe. Things happened to us because of something you did. It might have been God punishing you or rewarding you for your actions, but it didn’t have to be it might have been elves or forest sprites, but it didn’t have to be those either. In some unspecified way, the universe was aware of you, and it responded accordingly. But during the Enlightenment, scholars became aware that most everything could be explained by natural laws. They described the universe as a gigantic clockwork mechanism, and that meant that the things that happened to you were simply happenstance without a greater meaning behind them. We began to understand that the universe was impersonal.
The mechanistic, impersonal nature of the universe gained practical importance during the Industrial Revolution. Before industrialization, everything was handmade, so every artifact was the product of an individual’s care and attention. The only way to get a really high quality object was to have a master craftsman put a lot of effort into making it, which meant that the outcome of the manufacturing process was entirely dependent on the practitioner. After the Industrial Revolution, that was no longer the case. High quality textiles no longer required skilled weavers. They could be churned out by mechanical looms powered by water wheels or steam engines, interchangeable parts meant that a factory could produce accurate rifles without a master gunsmith sweating over each and every one. Nowadays you can buy a smartphone that works amazingly well, even though no one who worked on the assembly line was emotionally invested in its manufacturer at all. All of that is possible because of the impersonal nature of the universe.
Remember those two hypothetical stories I mentioned above. I said both stories could be set in the present, but if we try moving them away from the present, we encounter some resistance. The first story, the one with the factories in mass production, could easily take place in the future, but it cannot be set too far in the past. It has to take place after the Industrial Revolution. The second story, the one with a few highly skilled individuals who have limited output, could be set at any point in history, but that mode of production was the norm before the Industrial Revolution, so the story fits most easily in that period. We don’t see a lot of artisanal labor nowadays, and it is likely that we will see less of it in the future.
It’s worth noting that in the world of most fantasy stories, no Industrial Revolution occurs no matter how much time passes. For example, in The Lord of the Rings, there are few individuals that have palantirs or seeing stones, which are sort of like two-way crystal balls, allowing you to see and talk to anyone else who has one. Some people who make the argument that sufficiently advanced technology is indistinguishable from magic will say that smartphones are the modern day version of palantirs, but the palantirs used in The Lord of the Rings are literally thousands of years old. In all of that time, no one has figured out a more efficient way to make them. In fact, it is explicitly stated that Gandalf cannot make one, and neither can Sauron. It’s safe to say that palantirs will never be made in a factory, so they will never become widely available, and that is a crucial difference between smartphones and palantirs.
Palantirs are the quintessential artisanal product. They can only be made one at a time, by hand, by individuals who possess special skills. The universe of Middle Earth recognizes certain people as special, and that is what has kept a revolution in palantir technology from occurring there.
Here’s another way to understand why mass production is incompatible with magic. Along with the characteristics of magic that I mentioned before—magic only works for certain individuals, or if your intentions are good, or if you’ve made a sacrifice—there are also characteristics of an interaction between one person and another. When you ask someone for a favor, you don’t expect that they will do a favor for anyone. You expect that they will say yes to people whom they’ve known for a while, but say no to those who are strangers. You expect that they will only do favors for people they consider trustworthy, and if it’s a really big favor you expect that you will need to show you’re serious and not just messing around.
I’m not saying that every act of magic is literally asking a deity for a favor. What I’m saying is: thinking about magic as an interaction with another person helps explain some of the ideas we have about magic. Belief in magic is a kind of anthropomorphic projection onto the universe as a whole. It’s the idea that the universe recognizes you’re a person, because that’s what people do. But mass production cannot be understood this way because people don’t behave this way. No one would grant a favor once a second, every second, day in and day out, 365 days a year. When mass production works, anthropomorphic production stops being a useful model for understanding how the universe operates. You are clearly dealing with a mechanistic system and not a person.
I’d also like to address one of the more common features of magic systems, that which is the idea that magic comes with a cost, from an extra-textual point of view. The reason that magic has to have a cost is so that you as the author have an answer to the question of, “Why can’t your characters use magic to solve all of their problems?” Fiction requires that your characters have problems, and if magic can solve all of their problems, you don’t have a story.
One of the ways to avoid this is to design a magic system where magic has a cost, one that prevents your characters from overusing it, but that is the author’s rationale. The characters are not aware of it. The characters will need a different explanation for why they cannot use magic all the time. Rules about the cost of magic are sometimes phrased in a way that resemble the laws of physics, like the conservation of momentum, every action has an equal and opposite reaction, or the conservation of energy, that you can’t get something for nothing. And sometimes these rules are cited as a way that magic and science are alike. But these resemblances are purely superficial. They don’t actually indicate any underlying similarity between magic and science, because the rules of magic are not really conservation laws. They are prescriptions against excess. They are moral injunctions. They carry the message that actions have consequences and nothing comes for free.
Consider the expression you can’t get something for nothing. There are two contexts in which I encounter this phrase. The first is that when you’re engaged with a transaction with another person, you have to give something to get something. If you try to cheat them, you will you risk getting penalized. The other context is that it is impossible to build a perpetual motion machine because the outputs of the system have to equal the inputs. These two contexts have nothing to do with each other. The conservation of energy has no moral dimension. It does not prevent abuses of power or ensure a balance between good and evil. It is purely a coincidence that the same phrase can be applied to both. When a magic system has a cost, it is an attempt to turn moral injunctions into physical laws. It reinforces the idea of a personal universe, rather than having anything to do with thermodynamics.
Now let’s return to the subject of science fiction and fantasy. I’m not going to try and give complete definitions of these two genres, because that’s a much bigger topic than we have time for today. I’ll just say that science fiction does not have to be about science and fantasy does not have to be about magic, so there are countless examples of these two genres for which this talk does not apply at all. I also want to be clear that I’m not saying it’s impossible for a story to incorporate aspects of both genres. I’m just saying that there are two directions in which stories can lean. And so, as a shorthand for the purposes of this discussion, I’m going to refer to stories about imaginary science as science fiction, and I’m going to refer to stories about magic as fantasy. But I acknowledge that this is a radical oversimplification.
“Belief in magic is a kind of anthropomorphic projection onto the universe as a whole.”
Let’s consider a story in which there are spaceships capable of faster than light travel. I would be inclined to call this a science fiction story, not because faster than light travel is possible, but because the story depicts a universe that resembles our own in an important way, that universe operates according to purely mechanistic laws. The characters inhabiting that universe can construct machines that operate according to those laws, and over time, those machines become easier to construct and more widely available, so eventually the majority of people can take advantage of them. That doesn’t mean it’s a good story, or even good science fiction. It just means that at a certain level, the story reflects a modern understanding of how the natural world behaves.
On the other hand, there are plenty of stories that have ostensibly futuristic settings for which this is not the case. For example, Star Wars includes people who can use the force, which is pretty clearly magic. The laws of the Star Wars universe contain exceptions for certain special individuals, and for those individuals, intentions play a role in the way that their special powers manifest. So I would say that Star Wars is, in an important respect, a fantasy story, because it depicts a universe that is personal. That doesn’t mean that it’s a bad story, it just means that at a certain level, it reflects a pre modern understanding of how the natural world behaves.
And Star Wars is far from the only example of this. There are countless stories where the protagonist is a kind of Chosen One, a savior with a special destiny and special abilities, someone who is different from everyone else. This type of story is very appealing, which is why we see it in so many different settings, ranging from pseudo-medieval settings with dragons to distant planets with spaceships. For this type of story, I would agree that the specific details of the setting are not that important, the fact that their hero is carrying a laser pistol covered with blinking LEDs instead of a sword encrusted with jewels doesn’t make that big a difference. All stories about a chosen one are, in a certain respect, fantasy stories.
So what difference does it make if a story reflects a modern or a pre-modern understanding of the natural world. That is, what difference does it make if a story is science fiction or fantasy? What do we get out of a story set in a mechanistic universe, and how does it differ from what we get out of a story set in a personal universe?
One of the strengths of science fiction is how it can help us think through the implications of an idea. I like it when science fiction stories are based on real science, but I think they can be interesting even when they are built on imaginary science, if they do a good job of extrapolation.
If you start with a question: What if we had a gadget that could do such and such? A science-fictional approach leads you to follow up with the questions: What if everyone had one of those gadgets? How would that change society? How would it change our values or the ways that we interact with each other?
I think of science fiction as a post-industrial-revolution form of storytelling, because stories about the ways that new technology can transform society only make sense to people who lived after the Industrial Revolution. In the Middle Ages, technological advances spread so slowly that no one ever saw their world transformed by a new invention within their lifetime. But now that sort of thing is happening every day. Science-fiction stories engage with that reality, and stories about imaginary technologies can help us think about the ways that real technologies affect us.
Fantasy stories do something a little different. Fantasy is good at taking metaphors and making them literal. It turns subjective truths into objective truths. Stories about magic tell you that how you feel matters, and that’s a good thing for a story to do. Having noble intentions or concentrating very hard are worthwhile goals, even if they don’t affect the physical universe. Stories about a universe that recognizes you’re a person are valuable because the modern world makes people feel unimportant. In particular, capitalism excels at making people feel unimportant. Working on an assembly line takes a lot of the joy out of working, but just because the universe does not pay special attention to us, we don’t have to accept the idea that we are undeserving of special attention.
Speaking of capitalism, it’s also worth thinking about the difference between a personal universe and a mechanistic universe, through the lens of Karl Marx. Marx talked about how factories and machines created a sense of alienation between workers and their labor. Unlike master craftsmen, who took pride in their work, assembly line workers repeating monotonous tasks all day long, gain no satisfaction from their labor. This alienation that Marx was talking about is a direct byproduct of living in a mechanistic universe. In a universe where magic works, that type of alienation cannot happen because magical labor is inextricably tied to an individual practitioner. To the extent that a fantasy story contains magic, the universe of that story resists the alienation of labor, and this is another reason why fantasy stories are appealing.
The story might not make it explicit, but in a universe built around magic, you will never reach a point where iPhones are made by semi-skilled laborers in a factory in China, because the laws of the universe prevent that from working. Wondrous objects can only be made by gifted individuals. Even in a contemporary magic-realist story where the magic is very understated, there is a sub-textual message that human beings possess some special quality that modern society cannot take away from them. When people say that their job makes them feel like a cog in the machine, they are experiencing the alienation that Marx was talking about.
Fantasy stories are a way of imagining that the universe itself does not see you as a cog in the machine, even if capitalism does. Ever since the Industrial Revolution, capitalism has moved in the direction of replacing human workers with automation, and the mechanistic nature of the universe is what makes that possible. However, something being possible is entirely different from it being right. The physical laws of the universe don’t distinguish between people and machines, but that does not mean that we as a society ought to treat people like machines. Corporate executives talk about capitalism like it’s a law of the universe, but it is not. The universe makes dehumanizing labor practices possible, but that does not mean that the universe endorses those practices, and this is why I think it’s worth making a distinction between imaginary science and magic, because they serve two different roles in fiction.
The scientific worldview has become dominant in the last two centuries, and it has helped us achieve amazing things, but it also runs the risk of making people feel insignificant. It’s important for us to understand how the universe works and how technology affects our lives, but it’s also important for us to be reminded that our desires and our intentions matter. Stories about science help us think about the physical world, while stories about magic help us think about our worth as individuals, and we benefit from both kinds of stories. Thank you.