
The Paper That Invented the Neural Net (1943)
How a neurophysiologist and a self-taught teenage logician wrote the founding document of modern AI before a single stored-program computer existed
Every neural network you've used the one drafting your emails, generating your images, recommending your next video traces its conceptual lineage back to one paper. Not a 2012 ImageNet paper. Not "Attention Is All You Need." A 1943 paper written before the word "computer" meant what it means today.
"A Logical Calculus of the Ideas Immanent in Nervous Activity," Warren McCulloch and Walter Pitts. If you've heard of it at all, you probably haven't read it the notation is borrowed from 1930s symbolic logic (Carnap, Russell and Whitehead) and it reads like it was written to keep people out. But the core idea inside it is simple enough to explain in a sentence, and understanding it properly is worth the hour, because it's the first rung of the ladder that leads directly to the architectures we build on today.
This is a full technical walkthrough every assumption, every theorem, in plain language. No prior background in formal logic required.
The core move
McCulloch was a neurophysiologist. Pitts was a self-taught logician who, a few years before this paper, had been effectively homeless, teaching himself mathematical logic in libraries. Together they asked one question: what if you modeled a neuron as a proposition in formal logic?
Not "represents" a proposition. Is one, for the purposes of the math. Fired = true. Didn't fire = false. Nothing else about the neuron's biology matters to the theory.
That's the entire foundation. Everything downstream logic gates built from wiring diagrams, a working explanation of a real sensory illusion, memory modeled as loops, and eventually a formal equivalence to the Turing machine falls out of taking that one assumption completely seriously.
The five assumptions the whole calculus rests on
All-or-none activity. A neuron fires, or it doesn't. No partial states.
Fixed threshold. A specific number of synapses must fire together, within one narrow time window, to trigger a neuron.
Synaptic delay is the only delay that matters. Signal propagation inside a neuron is treated as instantaneous; all timing happens at the junctions between neurons.
Absolute inhibition. One active inhibitory synapse completely blocks firing, regardless of how many excitatory synapses are also active.
Static topology. The wiring of the network doesn't change over time.
None of these are biologically exact the authors say so themselves but that's not the goal. The goal is the simplest possible model of "neuron-like behavior" that still supports rigorous proof, and then showing how much that simplified model can actually do.
One more distinction worth carrying forward: the paper separates equivalence in the narrow sense (identical behavior, tick for tick) from equivalence in the extended sense (same eventual behavior, possibly on a different timescale). Nearly every "these two mechanisms are really the same thing" result later in the paper is proven in the extended sense which is what allows very different biological explanations to be treated as logically interchangeable.
Section II Nets Without Circles
This section restricts itself to feedforward networks: no loops, no signal referencing its own past.
Theorem I. Any loopfree net, no matter how complex, can be fully described by a single logical formula (a Temporal Propositional Expression, or TPE) written purely in terms of its input neurons. You derive it by substituting each neuron's local firing rule into its downstream neighbors, repeatedly, until you bottom out at the inputs. Because there are no loops, this process is guaranteed to terminate.
Theorem II. The converse: any valid TPE can be physically realized as an actual net. The proof is an induction over four base cases delay, OR, AND, and AND NOT each directly constructible from a small wiring diagram. Since nesting realizable expressions inside each other stays realizable, and every TPE is built from nested combinations of these four operations, every TPE is buildable.
Together, Theorems I and II establish something clean: for loop-free networks, logic and circuit topology are the same language, just expressed two different ways. This is, not coincidentally, the same duality that underlies digital logic design today Boolean expressions and gate-level circuits are interchangeable representations of the same function.
Theorem III settles a smaller but necessary question: which logical sentences are physically realizable at all? Answer a sentence is realizable exactly when it evaluates to false if every one of its elementary components is assumed false. Intuitive once stated: a neuron can't fire spontaneously from nothing, so nothing downstream of an all-false input can fire either.
A worked example that grounds the abstraction
Immediately after Theorem III, the authors build the actual circuit for a documented sensory illusion: briefly touching a cold object produces a felt sensation of heat, while sustained contact produces only cold. Their explanation the "feels hot" neuron fires either from a genuine heat signal, or from the cold receptor firing recently and then abruptly stopping. That pattern of recent change is what gets misread as heat. The whole circuit is built from nothing but the four gates established in Theorem II.
This is worth sitting with: perception, in this model, is not a direct readout of external reality it's whatever the specific wiring computes from its inputs. Change the network, and identical physical stimuli produce different subjective results.
Theorems IV–VII mechanism-independence
The next four theorems each prove that two different candidate biological mechanisms produce identical logical behavior:
Theorem IV = a hard inhibitory veto and a "soft" threshold-raising form of inhibition are equivalent.
Theorem V = post-firing refractory behavior (extinction) can always be reconstructed as a self-inhibiting feedback loop.
Theorem VI = temporal summation (stimulation accumulating over time) can be replaced by spatial summation (parallel delay lines of different lengths converging at once).
Theorem VII = learning, modeled as a dormant synapse that permanently activates after co-firing, can always be reconstructed as a loop.
Theorem VII = is the hinge point of the whole paper. Learning requires memory, and memory requires a loop. Which means the entire next section loops, which the paper had deliberately excluded up to now becomes unavoidable.
Section III = Nets With Circles
Loops break the clean backward-substitution trick from Section II. A signal inside a loop can depend on events arbitrarily far in the past the notion of "how long ago" becomes indefinite. This is, functionally, close to what memory is.
The paper defines the order of a network as the size of its smallest cyclic set the fewest neurons you'd need to remove to eliminate every loop. Order 0 nets are everything covered in Section II.= Higher-order nets are progressively more loop-dependent and harder to analyze.
Theorem VIII = shows looped nets can still be fully solved, but the solution now requires recursive function theory rather than plain propositional logic the same mathematical toolkit underlying the formal definition of computability.
Theorem IX= provides an exact realizability test (equation 14 in the paper) for whether a given set of "required past patterns" can be built into a net. It's complete but computationally impractical verifying it in the general case means searching a space that grows as 2^(2^n).
Theorem X supplies the practical alternative: a restricted, well-behaved class of expressions (ordinary TPEs plus universal/existential time quantifiers plus periodic/congruence statements) that's guaranteed realizable, without the exponential search.
The Turing-equivalence result
The section closes with the paper's most consequential claim: a net equipped with a tape, scanners, and output mechanisms computes exactly the class of functions a Turing machine computes and the converse holds too. Nets with loops but no tape compute a strict subset of that class.
Turing's 1936 formalization of computability, Church's lambda calculus, and Kleene's recursive functions had already been shown to define the identical class of computable functions the Church-Turing thesis. McCulloch and Pitts added a fourth, biologically grounded angle to that equivalence: anything a nervous system can compute is computable by all of these formal systems, and vice versa.
This is the sentence that elevates the paper from a neuroscience curiosity to a founding document of computer science. It's a direct conceptual ancestor of the perceptron (Rosenblatt, 1958), and from there, of every layered, trainable network built since.
Section IV = Consequences
The final section shifts from engineering to epistemology.
Because the system's logic includes disjunctions (ORs) and loop-based regeneration, forward computation from a known state is always well-defined, but backward inference generally isn't multiple distinct histories can converge on the same present state. The authors' conclusion: knowledge of the world, including self-knowledge, is structurally incomplete not from any deficiency of measurement, but as a direct consequence of how a regenerative network behaves.
They extend this into a claim about the smallest unit of subjective experience proposing that it can be no smaller than a single neuron's firing, since that firing is already inherently referential (they use the term "semiotic"). Because firing obeys strict two-valued logic, they argue that psychological phenomena, examined from any angle, ultimately reduce to that same two-valued structure.
They close by grounding this in clinical phenomena tinnitus, hallucination, delusion, disorientation framed not as malfunctions layered on top of normal perception, but as the correct output of a differently structured or damaged network, given its actual inputs.
Why this still matters if you're building today
None of the specific neuroscience in this paper survived. Real neurons are analog, stochastic, and chemically far more complex than an all-or-none switch. The authors were explicit that this was a simplification, not a biological claim.
What survived is the framework: simple binary units, connected and weighted, capable in aggregate of representing any computable function. That framework became the perceptron, then backpropagation, then deep learning, then the transformer architecture behind the models most of us now build on top of daily. Understanding this paper doesn't make you better at tuning a model but it does make the whole field legible in a way that skipping straight to modern architectures never quite gives you. You start seeing why the field is built the way it is, not just how to use it.
It started here: two people, drawing triangles, working out what a neuron and a true-or-false statement actually have in common.
This is the first in an ongoing series working through the foundational papers of AI and machine learning, one at a time, from first principles. Full video walkthrough and diagram-based notes for this paper are linked below.
🎥 Video: [ https://youtu.be/abKmAEwNzEo?si=zrtycQhgbybXhCG\ _]
📄 Notes / diagrams: [ https://drive.google.com/file/d/1u\_WoOGF70FqFUT6JhUn-ztqnEXXjV3-D/view?usp=drivesdk ]
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article