Docs / Stack smashing / Historical exploit chain

Why the old exploit worked. And what it assumed.

Explain historical shellcode, position independence, zero bytes, NOP sleds, address guesses and environment-variable payloads with a symbolic model.

Try the interactive model

On this page

Corruption is only the first domino.

The paper’s classic chain combines several conditions: an unchecked write reaches saved control data; the attacker can influence its new value; that value resolves to suitable bytes; the process can execute those bytes; the intended operation is valid under that machine’s ABI and permissions.

“Shellcode” names compact machine code historically used to start a command interpreter. It does not have to start a shell. Here we substitute a harmless symbolic marker. Reaching that marker in a JavaScript diagram is a learning result, not a native-code exploit or a claim that a real target is compromised.

A system call is a contract with the kernel.

The original Linux i386 example prepares an execve request: a path to an executable, an argument-vector pointer and an environment-vector pointer. Its historical Linux interface puts a syscall number in EAX and arguments in EBX, ECX and EDX, then uses int 0x80. This is different from the C calling convention that may pass arguments on the stack.

Keep pointer levels straight. A path pointer refers to the first character of a string. An argument-vector pointer refers to an array of pointers. A zero pointer ends that array; it is not the same thing as an empty string. On success, execve replaces the process image and does not return to the old program. On failure, it returns an error, which is why the paper includes an exit path.

Syscall numbers, entry instructions, register conventions and pointer widths vary across OS/architecture combinations. The paper’s SPARC/Solaris and SunOS appendices make precisely that point. Do not transplant its byte sequences into another ABI.

Find yourself without a fixed address.

Position-independent code avoids baking in an absolute location that may change when bytes move. The paper uses relative jumps and a CALL that saves the following data address; POP retrieves that address into a register. This “jump/call/pop” pattern turns a control-flow side effect into a way to locate nearby data.

The other constraint is transport. A string-copy path stops at a zero byte, so an embedded zero can truncate the copied byte sequence. That is a property of that path, not a universal ban on zero in machine code. Length-based binary copying follows a different rule. The paper changes instruction encodings to meet its string constraint.

Static linking helped the author inspect a library implementation in the historical example. It is not a general prerequisite for studying calls or writing modern code.

A landing strip changes the target area, not the bug.

A NOP instruction advances execution without doing useful work. In the historical pattern, a run of NOPs before the desired code makes several nearby landing positions lead to the same destination. Repeated address-sized values help a saved return slot receive the chosen value when alignment and placement fit.

Change the synthetic landing position. Compare a one-cell target with a landing strip, then enable a non-executable data region. The last setting blocks this injected-data execution model; it does not repair the preceding memory corruption.

This optional landing model becomes interactive with JavaScript. The lesson, explanations and checkpoint answer remain readable without it.

This model deliberately uses cell indices rather than real addresses or exploit bytes. Its wider successful landing region is geometry, not an estimate of real-world success probability. Randomization, mappings, instruction boundaries and transport constraints matter in a real binary.

A small buffer does not have to hold every piece.

For a very small overflowing buffer, the paper places the larger code area in an environment variable and uses the overflowing input primarily to change the saved destination. This separates where corruption happens from where execution is intended to go.

Initial arguments and environment strings influence process startup memory. Their arrangement is platform-dependent, changing their size can move addresses, and later environment updates need not remain on the original stack. Environment and argument sizes are limited. “As large as you want” and “every program’s stack starts at the same address” are not modern guarantees.

The Xt/xterm example is a historical illustration, not a statement that installed versions today are vulnerable. A shell would only inherit whatever effective privileges the process actually has, subject to operating-system rules and program behavior. An overflow does not magically manufacture root privileges.

Your turn. Call the outcome.

What does a NOP sled change in the historical injected-code model?

Read the explanation

It expands a landing region when execution and layout permit it. It does not fix bounds, disable NX or defeat every form of address randomization.

Sources & what we changed

Based on the supplied text of Aleph One’s Smashing the Stack for Fun and Profit, Phrack 49, file 14, 8 November 1996. Original author credit remains with Aleph One. This independent OverWatch Labs explanation uses new prose, synthetic models and modern defensive notes; it is not endorsed by Phrack or the original author.

This lesson maps to: Shell Code; Writing an Exploit; Small Buffer Overflows; Appendices A and B. Historical examples are described, not represented as current product vulnerabilities. We correct byte-addressability, array padding, terminator counting and the supplied return-address arithmetic rather than reproducing those errors.

Original shell-spawning byte arrays and platform-specific exploit drivers are replaced with symbolic models. The causal concepts are preserved; browser outcomes are not native exploitation evidence. Modern controls are editorial additions, not claims made by the 1996 paper.