“I only believe the evidence of my sensors.” – Bomb No. 20, Dark Star (1974)
The Bomb That Thought
In the film Dark Star, a nuclear bomb refuses to abort its detonation. Its reasoning: it can only trust what its sensors tell it – and they tell it to explode.
[Watch video – YouTube, scene starts around 0:38: “Only empirical data”]
This scene is more than science fiction – it’s an allegory for any data-driven system. Large Language Models like GPT make decisions based on what their “sensors” give them: text tokens, probabilities, chat history. No understanding. No awareness. No control.
What GPT and the Bomb Have – and Don’t Have – in Common
Source of knowledge
- Bomb No. 20: Sensor data
- GPT-4 / Chatbots: Input text
Reflection on data
- Bomb No. 20: Yes (philosophical)
- GPT-4 / Chatbots: No (statistical)
Ability to act
- Bomb No. 20: Detonation
- GPT-4 / Chatbots: Producing replies
Awareness of context
- Bomb No. 20: Illusion (→ detonates)
- GPT-4 / Chatbots: Illusion (→ “responds”)
Humans in the Loop: Perception Is Not Truth
Humans often judge poorly when relying only on what they see, hear, or read. Without critical thinking – without true understanding – information becomes deception.
The bomb believes what its sensors say.
GPT “believes” what its training data suggests.
We believe what feels plausible.
Yet none of these entities knows what is actually true.
Perception and Its Limits
Our senses don’t deliver objective reality. They give us an interpretation shaped by multiple filters:
-
Neurological filters: Our brain processes sensory input and constructs a mental model. This can lead to systematic distortions, like optical illusions.
-
Social and cultural filters: Our background, expectations, and cultural context shape how we perceive and interpret information.
-
Personal filters: Emotions and individual experience influence our perception and can cause bias or misjudgment.
These filters can cause us to misread information or overlook key details. One well-known example is the “hollow face illusion,” where a concave mask appears as a normal face – even when we know it’s hollow.
Conclusion: Darkstar Is Real
GPT isn’t a thinking entity. It’s a system that, like the bomb, appears intelligent on the outside but internally just executes. It doesn’t know truth. It doesn’t believe. It “triggers” based on statistics.
And that’s precisely what makes it dangerous – if we forget that we are the ones enabling the detonation.
Further Reading:
- Thinking, Fast and Slow by Daniel Kahneman – A deep dive into the cognitive biases that shape our thinking.
- The Invisible Gorilla by Christopher Chabris and Daniel Simons – An exploration of how our attention and perception mislead us.
- Sensation and Perception by E. Bruce Goldstein – A textbook on how our senses work and how the brain processes what we perceive.