Deep Engineering
Intermediate·Published·50 MIN

io.Reader and io.Writer: a contract of two methods

Interview questions on io climb a ladder: what a Reader is — how to write a read loop — why Read returned less than asked — how io.Copy copies and how much memory it takes — why bufio and what happens without Flush — what is wrong with defer f.Close(). A run answers each step: a loop that looks at the error before the bytes loses the last chunk; io.Copy sends 100,000 bytes in a single Write if the source has WriterTo; and a function with bufio and defer f.Close() returns nil and leaves an empty file on disk.

Full technical treatment

TL;DR

io.Reader and io.Writer are interfaces with one method each: Read(p) puts up to len(p) bytes into p, Write(p) writes p and, if it wrote less than len(p), must return an error. Both return (n int, err error) — how many bytes went through and what happened. Everything rests on these two signatures: a file, a network connection, gzip, bufio, bytes.Buffer, sha256, and io.Copy, which connects any source to any destination.

Read may return data and an error in the same call — and they must be handled in that order: first the n bytes, then err. The documentation allows two endings of a stream: the last chunk with nil and then (0, EOF) — or the last chunk right together with EOF. A loop with if err != nil { break } before handling n collects the whole string "hello, world" on a strings.Reader, and "hello, wor" on a reader of the second kind: two bytes vanished without a single error.

Beyond that comes what separates real understanding from having read about it. Read does not have to fill the buffer: with a buffer of 8 the reader returned 4 bytes, and filling the buffer completely is guaranteed by io.ReadFull — and on short input it answers io.ErrUnexpectedEOF. io.Copy takes a 32 KiB buffer, but first looks for WriterTo on the source and ReaderFrom on the destination: from a *bytes.Reader 100,000 bytes went out in one Write, without a single memory allocation. bufio.Writer turned 1000 writes into 3, but the third went down only on Flush, and the write error also arrived only on Flush. And a function with defer f.Close() and bufio without Flush returned nil and left a 0-byte file on disk. The lesson measured the price of the buffer in one configuration: 10,000 writes of 16 bytes to a file in a temporary folder, go1.24.7, linux/amd64 — 4248 µs without bufio and 65 µs with it. The mechanism explains why there is a difference; how large it is for you has to be measured on your own machine.

Where to start
Before this lesson it is enough to understand
  • how to declare a type and a method on it — that is the lesson on structs;
  • what a byte slice []byte is, its length and capacity; the "how to use it" level is enough, the details are in the lesson on slices;
  • that a type implements an interface if it has all of the interface's methods — there is nothing to declare; the details are in the lesson on interfaces, which comes later in the track;
  • that an error in Go is an ordinary error return value compared with nil — the details are in the lesson on errors;
  • that defer postpones a call until the function returns — the details are in the lesson on defer, panic and recover.
You do not need to know in advance
  • what exactly Read promises and why the error comes together with data;
  • io.ReadFull, io.Copy, WriterTo and ReaderFrom, the wrappers MultiWriter, TeeReader, LimitReader, Pipe;
  • bufio.Writer, Flush and the error that gets lost in defer f.Close().

What is really being asked

The ladder is usually this one:

  1. "What is io.Reader?" — the warm-up; everyone remembers the signature.
  2. "Write a loop that reads a stream to the end" — this checks in which order the candidate looks at n and at err.
  3. "Why did Read return less than the buffer size?" — the short read and io.ReadFull.
  4. "How does io.Copy copy, and how much memory does it need?" — a 32 KiB buffer and two interfaces that let it do without one.
  5. "Why bufio, and what happens if you forget Flush?"
  6. "What is wrong with defer f.Close()?" — about the error returned by a write that happens on close.

The ladder has one spine: Read and Write promise only one thing — how many bytes went through in this call and what happened. Neither "the whole buffer", nor "the end separately from the data", nor "written to disk" is part of the contract. Everything you get caught on in an interview is an attempt to read more into the contract than is written there.

Base: two methods and what they promise

Fragment of io/io.go — the interface definitions, comments omitted:

GO
type Reader interface {
    Read(p []byte) (n int, err error)
}
 
type Writer interface {
    Write(p []byte) (n int, err error)
}

The caller brings the buffer: Read writes into p, Write reads from p. That is why neither method has to allocate memory for the data itself, and also why the documentation forbids both of them to retain p after returning. The main sentence of the Read contract:

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered.

— Package io — Reader
language contractThe contract guarantees of Read, Write, ReadFull, Copy and bufio.Writer are written in the package documentation (go doc, go1.24.7); the run of bench/goio/internals.go illustrates them rather than establishes them. Buffer sizes, how the pipe and ReadFrom slice the chunks, and the large-write bypass in bufio are the behavior of a particular version's implementation; where it matters, that is marked separately.

"Up to len(p)" are the key words. Write promises something else: if it wrote less than the whole slice, it must explain why — "Write must return a non-nil error if it returns n < len(p)". The contracts are asymmetric, and this table is worth memorizing right away — all the mechanisms below rest on it:

Read(p)Write(p)
how many bytes0 to len(p), whatever is therelen(p); fewer only with an error
n > 0 together with err != nilhappens: handle n firsthappens, including with n == len(p); if fewer than len(p) bytes were written, the error is mandatory
end of dataio.EOF — not a failure, a signal—
buffer pmust not be retainedmust not be retained or modified

That is enough for a Junior-level answer to the first question: io.Reader is an interface with one method Read(p []byte) (n int, err error): it puts up to len(p) bytes into p and returns how many it put. io.Writer is the same for writing: if it wrote less than len(p), it must return an error. Any type with such a method — a file, a connection, a buffer — fits wherever the interface is expected. Why an interface needs no "implements" declaration and what lives in an interface variable is covered in the lesson on interfaces.

The Write rule works in one direction: a short write without an error breaks the contract, while a full write with an error does not. A writer may accept all the bytes and still report a problem, and the caller checks err even when n == len(p). The run checks both sides.

Output of bench/goio/internals.go, section 9; labels translated:

9) WRITE: A FULL WRITE TOGETHER WITH AN ERROR
─────────────────────────────────────────────
  the writer returns len(p) and errQuota — the Write contract allows this:
    io.Copy (generic loop)      -> 12, err=quota exceeded
    io.Copy (*strings.Reader)   -> 12, err=quota exceeded
    bufio: Flush                -> quota exceeded
  the writer returns len(p)/2 and nil — the contract is broken:
    io.Copy (generic loop)      -> 6, err=short write

io.Copy and bufio pass the error of a full write up: all 12 bytes are counted, and still the copy returned quota exceeded. A short write without an error was turned into short write by the io.Copy loop — more on that in Mechanism 3.

What this was checked on: the runs — go1.24.7 and go1.25.1, linux/amd64; the documentation quotes were checked against go doc on 14.10.2026. By that date Go 1.26 and 1.27 had already been released; the lesson was not run on them.

Mechanism 1: data arrives together with an error

The first question people get wrong is the order. The Read documentation allows a reader two ways to end a stream:

When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call.

— Package io — Reader

The run reads the stream "hello, world" with a 5-byte buffer using four readers. strings.Reader, *os.File and io.Pipe return the last chunk with nil and EOF on the next call with n=0; iotest.DataErrReader from the standard library returns EOF in the same call as the last chunk: it was made precisely to test code against that behavior.

Output of bench/goio/internals.go, section 1, one block per reader; labels translated:

  stream "hello, world", 12 bytes, 5-byte buffer
  strings.NewReader
    Read #1 -> n=5 "hello"    err=nil
    Read #2 -> n=5 ", wor"    err=nil
    Read #3 -> n=2 "ld"       err=nil
    Read #4 -> n=0 ""         err=EOF
  *os.File
    Read #1 -> n=5 "hello"    err=nil
    Read #2 -> n=5 ", wor"    err=nil
    Read #3 -> n=2 "ld"       err=nil
    Read #4 -> n=0 ""         err=EOF
  io.Pipe, writer: Write("hel"), Write("lo, wor"), Write("ld")
    Read #1 -> n=3 "hel"      err=nil
    Read #2 -> n=5 "lo, w"    err=nil
    Read #3 -> n=2 "or"       err=nil
    Read #4 -> n=2 "ld"       err=nil
    Read #5 -> n=0 ""         err=EOF
  iotest.DataErrReader(strings.NewReader)
    Read #1 -> n=5 "hello"    err=nil
    Read #2 -> n=5 ", wor"    err=nil
    Read #3 -> n=2 "ld"       err=EOF

Now the loop people write most often — "read, check the error, process".

Fragment — the function readBroken from bench/goio/internals.go, in essence:

GO
for {
    n, err := r.Read(buf)
    if err != nil {
        break
    }
    out = append(out, buf[:n]...)
}

Output, section 2; labels translated:

2) A LOOP THAT LOOKS AT THE ERROR FIRST
───────────────────────────────────────
  if err != nil { break } before processing n bytes:
    strings.NewReader          -> "hello, world"
    *os.File                   -> "hello, world"
    io.Pipe ("hel", "lo, wor", "ld") -> "hello, world"
    DataErrReader              -> "hello, wor"  (2 bytes lost)

On the first three readers the loop works; on the fourth it loses "ld" — the last chunk arrived together with EOF, and break threw it away unprocessed. And there is no error: the function returned the short string as if nothing had happened.

That is why such a loop passes tests easily: they are usually run on strings.NewReader or on a file — the kind of reader on which it is correct. It breaks on another source that is entitled to hand over the tail together with EOF, and it breaks silently. The documentation says how it should be done:

Callers should always process the n > 0 bytes returned before considering the error err.

— Package io — Reader

Fragment — the function readCorrect from bench/goio/internals.go, in essence:

GO
for {
    n, err := r.Read(buf)
    out = append(out, buf[:n]...) // bytes first
    if err == io.EOF {
        return out, nil
    }
    if err != nil {
        return out, err
    }
}

Output, section 2 (continued); labels translated:

  n bytes first, then the error:
    strings.NewReader          -> "hello, world"
    io.Pipe ("hel", "lo, wor", "ld") -> "hello, world"
    DataErrReader              -> "hello, world"
  io.ReadAll(DataErrReader)    -> "hello, world" err=nil

Two follow-up clarifications. io.EOF is compared with ==, and that is not sloppiness: the documentation requires that "Read must return EOF itself, not an error wrapping EOF, because callers will test for EOF using ==". Wrapping and errors.Is are about errors propagating up the stack; those are covered in the lesson on errors. And you usually do not need to write your own loop: in the run io.ReadAll collected the whole string from DataErrReader too, and the io.Copy loop in the source first writes the nr > 0 bytes and only then looks at the error (io/io.go, lines 430 and 448 on go1.24.7). A successful io.Copy returns nil, not EOF.

The correct loop and a write through bufio with a checked Flush fit into a short program. The full file bench/goio/cases/example/main.go (its comments translated here) can be copied into an empty folder next to a go.mod (module example, go 1.24) and run with go run .:

GO
// The lesson's full short example: a read loop by the Read contract and
// a write through bufio with a checked Flush.
//
// Run in the folder with this file and go.mod: go run .
package main
 
import (
	"bufio"
	"fmt"
	"io"
	"os"
	"strings"
	"testing/iotest"
)
 
// readAll reads r to the end: the n bytes first, then the error.
func readAll(r io.Reader) (string, error) {
	var out []byte
	buf := make([]byte, 5)
	for {
		n, err := r.Read(buf)
		out = append(out, buf[:n]...)
		if err == io.EOF {
			return string(out), nil
		}
		if err != nil {
			return string(out), err
		}
	}
}
 
func main() {
	// DataErrReader returns the last chunk together with io.EOF.
	s, err := readAll(iotest.DataErrReader(strings.NewReader("hello, world")))
	fmt.Printf("readAll: %q, err=%v\n", s, err)
 
	w := bufio.NewWriter(os.Stdout)
	n, err := io.Copy(w, strings.NewReader("line through bufio\n"))
	if err == nil {
		err = w.Flush() // without Flush the line would stay in the buffer
	}
	fmt.Printf("io.Copy: %d bytes, err=%v\n", n, err)
}

Output of bench/goio/example.sh; the header translated:

go1.24.7 linux/amd64 | full example: go run .

readAll: "hello, world", err=<nil>
line through bufio
io.Copy: 19 bytes, err=<nil>

Mechanism 2: the short read — Read does not have to fill the buffer

The second half of the same "up to len(p)":

If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

— Package io — Reader

In the run the buffer holds 8 bytes, and the stream is longer.

Output of bench/goio/internals.go, section 3; labels translated:

3) SHORT READ: READ DOES NOT HAVE TO FILL THE BUFFER
────────────────────────────────────────────────────
  HalfReader.Read(buf[8])      -> n=4 "hell" err=nil
  OneByteReader.Read(buf[8])   -> n=1 "h" err=nil
  io.Pipe, the writer writes "hel", "lo, wor", "ld"; the reader uses a buffer of 8:
    Read -> n=3 "hel" err=nil
    Read -> n=7 "lo, wor" err=nil
    Read -> n=2 "ld" err=nil
    Read -> n=0 "" err=EOF

iotest.HalfReader and iotest.OneByteReader are test wrappers that return less on purpose. The pipe shows the same thing with no intent at all: each Read got exactly what one Write on the other side put in — 3, 7 and 2 bytes, although the buffer held 8. In Mechanism 1 the same stream, read with a 5-byte buffer, came in chunks of 3, 5, 2 and 2: one 7-byte Write was split across two Reads. Chunk boundaries are set by the source, not by the buffer.

Hence the classic mistake with a fixed-length header.

An illustrative fragment, not part of the run:

GO
hdr := make([]byte, 8)
n, err := r.Read(hdr) // n may turn out to be 4

Code that goes on to parse all 8 bytes will read half garbage. For "exactly this many bytes" there is io.ReadFull:

ReadFull reads exactly len(buf) bytes from r into buf.

— Package io — ReadFull

Output, section 3 (continued); labels translated:

  io.ReadFull(HalfReader, buf[8]) -> n=8 "hello, w" err=nil
  io.ReadFull(5 bytes, buf[8])    -> n=5 "hello" err=unexpected EOF
  io.ReadFull(0 bytes, buf[8])    -> n=0 err=EOF
  errors.Is(ErrUnexpectedEOF, EOF) -> false

HalfReader returns half at a time, and ReadFull keeps calling it until it has 8. The two lower cases are more interesting: EOF and ErrUnexpectedEOF are different answers. EOF means "there was no data at all" — the stream ended exactly on a record boundary, and that is a normal end. ErrUnexpectedEOF means "a record started and was cut off" — the input is truncated. errors.Is does not confuse them, and a loop reading fixed-length records is written exactly that way: io.EOF — exit, io.ErrUnexpectedEOF — report broken input.

Mechanism 3: io.Copy — a 32 KiB buffer or someone else's method

io.Copy(dst, src) is the very loop from Mechanism 1, written once. But before running it, Copy asks whether the two sides can copy by themselves:

If src implements WriterTo, the copy is implemented by calling src.WriteTo(dst). Otherwise, if dst implements ReaderFrom, the copy is implemented by calling dst.ReadFrom(src).

— Package io — Copy

What it looks like in code — io/io.go, lines 407–419:

implementation detail · Go 1.24The order of the checks is written in the io.Copy documentation; the buffer size 32 * 1024 and the shrinking for a LimitedReader are the io/io.go source of a particular version (lines 410, 414 and 418 on go1.24.7 and go1.25.1), not a guarantee.
GO
func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
	// If the reader has a WriteTo method, use it to do the copy.
	// Avoids an allocation and a copy.
	if wt, ok := src.(WriterTo); ok {
		return wt.WriteTo(dst)
	}
	// Similarly, if the writer has a ReadFrom method, use it to do the copy.
	if rf, ok := dst.(ReaderFrom); ok {
		return rf.ReadFrom(src)
	}
	if buf == nil {
		size := 32 * 1024
		if l, ok := src.(*LimitedReader); ok && int64(size) > l.N {

The run checks every branch with counters: a wrapper around the source counts Read calls and records len(p), a wrapper destination counts Write calls. The wrappers deliberately hide every method except their own — otherwise Copy would find WriterTo on the *bytes.Reader inside.

Output of bench/goio/internals.go, section 4; labels translated:

4) IO.COPY: A 32 KB BUFFER OR SOMEONE ELSE'S METHOD
───────────────────────────────────────────────────
  copying 100000 bytes
  Read only -> Write only:       Read ×5, len(p)=32768; Write ×4 [32768 32768 32768 1696]; copied 100000
  *bytes.Reader (WriterTo):     Read ×0; Write ×1 [100000]; copied 100000
  -> *bytes.Buffer (ReaderFrom): Read ×10, first len(p)=[512 512 1024]; copied 100000
  io.LimitReader(src, 100):     Read ×1, len(p)=100; Write ×1; copied 100
  total heap allocated (TotalAlloc) per io.Copy, average over 1000:
    generic path (io.Copy buffer) allocations: 1, 32.0 KiB
    via WriterTo                  allocations: 0, < 1 KiB

Four lines — four branches of the source. The generic path: a buffer of 32,768 bytes, one allocation per Copy call, four Writes and a fifth Read that returned EOF. WriterTo: *bytes.Reader handed the destination the whole slice in one Write, Read was never called, no memory was allocated. ReaderFrom: now *bytes.Buffer does the reading — in its own chunks, starting at 512 bytes. LimitReader: the buffer shrank to 100 bytes, because there is nothing more to read anyway.

Who in the standard library can bypass the buffer this way.

Output, section 5:

5) WHO IN THE STANDARD LIBRARY IMPLEMENTS WRITERTO AND READERFROM
─────────────────────────────────────────────────────────────────
  *strings.Reader    WriterTo: true  ReaderFrom: false
  *bytes.Reader      WriterTo: true  ReaderFrom: false
  *bytes.Buffer      WriterTo: true  ReaderFrom: true
  *bufio.Reader      WriterTo: true  ReaderFrom: false
  *bufio.Writer      WriterTo: false ReaderFrom: true
  *os.File           WriterTo: true  ReaderFrom: true
  *net.TCPConn       WriterTo: true  ReaderFrom: true
  *io.PipeReader     WriterTo: false ReaderFrom: false
  *io.LimitedReader  WriterTo: false ReaderFrom: false
  io.Discard         WriterTo: false ReaderFrom: true

The practical takeaway — a wrapper hides the methods of what it wraps. The counter from the run, io.LimitReader, any type of yours with a single Read switch off the source's WriterTo: Copy sees only the methods of the outer type. The destination's ReaderFrom still works — in the run onlyReader → *bytes.Buffer still went through ReadFrom. That follows from how a type assertion to an interface works, not from anything special about io.

The io wrappers do not accumulate data. MultiWriter hands every write to all its writers in turn, TeeReader writes what it read into a writer right inside Read, LimitReader cuts the stream off, Pipe connects code that writes with code that reads. About Pipe the documentation says it plainly: "The data is copied directly from the Write to the corresponding Read (or Reads); there is no internal buffering".

Output, section 6; labels translated:

6) COMPOSITION: WRAPPERS DO NOT ACCUMULATE DATA
───────────────────────────────────────────────
  stream of 1048576 bytes
  MultiWriter(a, b):  a got 1048576 bytes in 32 Writes, b — 1048576 in 32
  TeeReader(r, sha256): wrote 1048576 bytes, hash matches sha256.Sum256: true
  LimitReader(r, 1000): wrote 1000 bytes
  Pipe, one Write of 1048576 bytes: the reader got 1048576 bytes in 32 Writes of 32768
  total heap allocated (TotalAlloc) per copy of 1 MiB, average over 200:
    io.Copy(w, *bytes.Reader)      allocations: 0, < 1 KiB
    io.Copy(MultiWriter(w1, w2))   allocations: 2, < 1 KiB
    io.Copy(w, TeeReader)          allocations: 2, 32.0 KiB
    io.Copy(w, io.Pipe)            allocations: 6, 32.5 KiB
    io.ReadAll(*bytes.Reader)      allocations: 27, 5118.3 KiB
  writer returned n < len(p) and nil: io.Copy -> 6, err=short write

A megabyte through TeeReader or a pipe costs roughly one io.Copy buffer — 32.0 and 32.5 KiB, and TeeReader computed the SHA-256 of the same stream in the same pass. Through MultiWriter from a *bytes.Reader — under a kilobyte: the source's WriterTo passed the whole slice in one write, and MultiWriter handed it to two writers. For comparison, the bottom line: io.ReadAll of the same megabyte allocated more than five megabytes — it collects everything into one growing slice (more in "Deeper"). A stream that can be processed on the way is better processed on the way.

The last line is about the Write contract. The writer returned half and nil, breaking the rule n < len(p) → error, and the generic io.Copy loop turned it into io.ErrShortWrite by itself (io/io.go, lines 443–446 on go1.24.7). The source in the run is wrapped so that the loop is reached: a bare *strings.Reader has WriterTo, and it would have made the same check. Without that check, half the data would have disappeared silently.

Mechanism 4: bufio.Writer — writes go down in chunks, and the tail only after Flush

bufio.Writer is a writer on top of a writer: it accumulates writes in a buffer and hands them down in large chunks. A counter underneath shows how many Writes reach the underlying writer.

Output of bench/goio/internals.go, section 7; labels translated:

7) BUFFERING: HOW MANY WRITES REACH THE UNDERLYING WRITER
─────────────────────────────────────────────────────────
  1000 writes of 10 bytes = 10000 bytes
  without bufio:                 Write below ×1000 bytes arrived: 10000
  bufio.NewWriter, before Flush: Write below ×2    bytes arrived:  8192, in the buffer: 1808
  after Flush:                   Write below ×3    bytes arrived: 10000 [4096 4096 1808]
  default buffer size: 4096 bytes
  one Write of 10000 bytes into an empty buffer: Write below ×1 [10000]

A thousand writes became three. But the third went down only on Flush: before it the underlying writer had received 8192 bytes, and 1808 sat in the buffer. If the program ends or the function returns without Flush, those bytes will be nowhere — bufio.Writer does not know it has been abandoned. The documentation says so directly:

After all data has been written, the client should call the Writer.Flush method to guarantee all data has been forwarded to the underlying io.Writer.

— Package bufio — Writer

The last line of the block is an implementation detail: a write larger than the buffer into an empty buffer is not copied by bufio but handed down directly (bufio/bufio.go, lines 680–683 on go1.24.7, the comment "Large write, empty buffer").

The other side of the buffer — the error is deferred too. /dev/full on Linux is a device on which every write ends with "no space left on device".

Output, section 7 (continued); labels translated:

  bufio.Writer on top of /dev/full (every write is ENOSPC):
    100 x Write(10 bytes) without error: 100 of 100
    Flush -> write /dev/full: no space left on device
    Write after the error -> write /dev/full: no space left on device
    without bufio, the very first f.Write -> write /dev/full: no space left on device

A hundred writes "succeeded": they landed in the buffer and never reached the device. Only Flush returned the error — and after it bufio.Writer answers with the same error on any call:

If an error occurs writing to a Writer, no more data will be accepted and all subsequent writes, and Writer.Flush, will return the error.

— Package bufio — Writer

Hence the rule that is actually being tested: with bufio, checking the error of every Write is not enough — you have to check Flush. The stickiness of the error is even convenient: you can write without checks and check once — on Flush.

Mechanism 5: Close writes too — and its error gets lost in defer

The last step of the same idea: for many writers the last write happens not in Write but on close. The most vivid example is gzip.Writer:

Close closes the Writer by flushing any unwritten data to the underlying io.Writer and writing the GZIP footer.

— Package compress/gzip — Writer.Close

Output of bench/goio/internals.go, section 8; labels translated:

8) CLOSE WRITES TOO — AND ITS ERROR GETS LOST
─────────────────────────────────────────────
  defer f.Close(), bufio without Flush, 100 lines: returned nil, file size 0 bytes
  gzip without Close: bytes in the underlying writer: 10
    reading back -> unexpected EOF
  gzip after Flush, without Close: bytes in the underlying writer: 62
    reading back -> unexpected EOF
  gzip after Close: bytes in the underlying writer: 75
    reading back -> 2600 bytes, err=nil

The first line is a function people write all the time.

Fragment — the function writeReportDeferOnly from bench/goio/internals.go, in essence:

GO
func writeReport(path string, lines int) error {
    f, err := os.Create(path)
    if err != nil {
        return err
    }
    defer f.Close()
    w := bufio.NewWriter(f)
    for i := 0; i < lines; i++ {
        fmt.Fprintf(w, "line %d\n", i)
    }
    return nil
}

The hundred lines of the run take 1590 bytes, less than the default bufio buffer (section 10 of the run: "100 report lines from section 8: 1590 bytes; default bufio buffer: 4096 bytes"). All of them stayed in the buffer, the file was closed empty, and the function returned nil. Not a single error check fired, because there was no error — there was a lost write. With gzip it is the same on another level: without Close the underlying writer holds only the 10-byte header, even after Flush the archive cannot be read — there is no footer, and the reader answers unexpected EOF.

Now about defer f.Close() itself. Close returns an error, and defer throws that result away. The run shows that there is somewhere for an error to come from.

Output, section 8 (continued); labels translated:

  first f.Close() -> nil
  second f.Close() -> close <folder>/twice.txt: file already closed
  f.Write after Close -> write <folder>/twice.txt: file already closed
  writing to /dev/full:
    defer w.Flush(); defer f.Close()     -> the function returned nil
    Flush checked, Close in defer with err -> the function returned write /dev/full: no space left on device

The function that deferred both Flush and Close returned nil when writing to a device with no space. The one that checked Flush and captures the Close error into a named result returned the real cause. Here the error came from Flush: capturing the Close error is insurance for the case when Close writes by itself, as with gzip.Writer; this run does not show it in action.

Fragment — the function writeToChecked from bench/goio/internals.go under a shorter name:

GO
func writeTo(path string) (err error) {
    f, err := os.OpenFile(path, os.O_WRONLY, 0)
    if err != nil {
        return err
    }
    defer func() {
        if cerr := f.Close(); err == nil {
            err = cerr
        }
    }()
    w := bufio.NewWriter(f)
    if _, err = w.WriteString("report\n"); err != nil {
        return err
    }
    return w.Flush()
}

Why defer can change the return value and why that needs a named result is covered in the lesson on defer, panic and recover. What matters here is the rule: for a writer, the last write is Flush or Close, and their errors are checked just like the error of Write. For a file opened for reading only, defer f.Close() without a check is common practice: there is nothing left to write. Whether Close on a regular file can return a write error was not tested by this run: on a local disk the first Close returned nil. And the guarantee that the data reached the disk comes not from Close but from f.Sync() — according to the documentation it "commits the current contents of the file to stable storage", and its error is checked too.

Deeper: why a small Write is expensive — and what memory costs

implementation detail · Go 1.24That f.Write on an *os.File is one write system call per call comes from the internal/poll/fd_unix.go source of a particular version, not a guarantee of package os.
measured observationbench/goio/practice.go, go1.24.7 linux/amd64, GOMAXPROCS=2, best of 15 rounds; the file is in a temporary folder, writes of 16 bytes, and creating the file is not part of the measurement. This is an observation of one configuration, not a portable number: the ratio depends on the file system, the kernel, the write size and the machine — on yours it has to be measured. Call and allocation counters are an observation of the go1.24.7 implementation, not a language contract. Memory — bench/goio/internals.go, section 6; under go1.25.1 the output is the same, apart from a tenth of a kilobyte for io.ReadAll.

The counter from Mechanism 4 tells how many calls reach the bottom; the measurement tells what that means for a file. 10,000 writes of 16 bytes straight into an *os.File and through a bufio.Writer.

Output of bench/goio/practice.go, problem 2; labels translated:

  10000 x f.Write(16 bytes) to *os.File           4248 µs, 10000 Writes reached the file
  the same writes via bufio.Writer + Flush        65 µs, 40 Writes reached the file
  ratio                                          65
  per write: 425 ns against 7 ns

The counter in the same run shows that all 10,000 Write calls reached the file, and each f.Write on an *os.File goes into the write system call (internal/poll/fd_unix.go, FD.Write, line 381 on go1.24.7: syscall.Write): 425 ns per write. How the price depends on the size of one write was not tested — the measurement has a single size, 16 bytes. bufio turns 10,000 calls into 40; its 65 µs include both those 40 calls and copying 10,000 records into the buffer — 7 ns per write on average. Hence the rule: many small writes to a file or a socket go through a buffer. Where Write writes to memory anyway, as with bytes.Buffer, the buffer is superfluous; and bufio.NewWriter on top of a bufio.Writer with a large enough buffer, according to the documentation, simply returns that same writer.

The second price is memory, and here the trap is the opposite. io.Copy keeps one 32 KiB buffer in memory, however much data flows through (with wrappers in the run — no more than 32.5 KiB). io.ReadAll keeps everything.

An output line of bench/goio/internals.go, section 6; labels translated:

    io.ReadAll(*bytes.Reader)      allocations: 27, 5118.3 KiB

This is the sum of all allocations per call (TotalAlloc), not memory held at the same time. ReadAll starts with a 512-byte slice and grows it by appending (io/io.go, lines 709–726 on go1.24.7); every growth is a new array, and the previous one becomes garbage. At the end one slice remains.

Output of bench/goio/internals.go, section 10; labels translated:

  io.ReadAll(1048576 bytes): len=1048576, cap=1114112 — the final slice

The final buffer is about 1.06 MiB; the remaining ~4 MiB of the 5118.3 KiB are intermediate arrays that the garbage collector will reclaim. How many of them are alive at the same time at the peak, the lesson did not measure; but they had to be allocated. And when the other side sets the size of the stream — a request body, another service's response — ReadAll without a limit hands it the decision of how much memory your process will take. If you really need all of it in memory, limit the size (io.LimitReader); if not, process the stream on the way with io.Copy and wrappers.

A summary table that is handy to keep in your head:

tasktoolwhat to check
read a stream to the endio.Copy, io.ReadAll, your own loopn first, then err
read exactly N bytesio.ReadFullEOF versus ErrUnexpectedEOF
many small writesbufio.Writerthe Flush error
writing with compression or a buffergzip.Writer, bufio.Writerthe Close / Flush error
a stream to two placesio.MultiWriter, io.TeeReaderthe error of any of the writers
limit the inputio.LimitReaderthat truncated input is not an error but EOF

How to answer in an interview

Short answer: io.Reader is an interface with one method Read(p []byte) (n int, err error): it puts up to len(p) bytes into p and returns how many it put, plus an error. io.Writer is the same for writing, but having written less than len(p) it must return an error, and the error is checked even after a full write. When reading, you handle the n bytes first and then look at err, and io.EOF is the normal end of a stream. For ready-made tasks — io.Copy, io.ReadFull, bufio.

That is enough to answer correctly. What follows is what gets added if the interviewer digs.

If the interviewer digs deeper

On the read loop — the order of n and err. "The documentation allows the last chunk to come together with EOF. A loop that checks err first works on a strings.Reader, but on iotest.DataErrReader it returned "hello, wor" instead of "hello, world" — with no error. So buf[:n] first, then err."

On "why did it read less" — the short read. "Read promises up to len(p), not len(p): with a buffer of 8 the reader returned 4. For a fixed size — io.ReadFull; on truncated input it returns ErrUnexpectedEOF, on empty input EOF, and those are different cases."

On io.Copy — the buffer and the interfaces. "First WriterTo on the source, then ReaderFrom on the destination, and only then a 32 KiB buffer. From a *bytes.Reader 100,000 bytes went out in one Write with no allocations; a wrapper with a single Read switches that path off."

On bufio — Flush and the deferred error. "A thousand writes of 10 bytes — three Writes below, and the last one only on Flush. The write error also arrives only on Flush: on top of /dev/full all hundred Writes returned nil."

On defer f.Close() — writing on close. "For gzip.Writer and bufio the last write is in Close and Flush. A function with defer f.Close() and bufio without Flush returned nil and left an empty file. So I check Flush explicitly, and for a writer I capture the Close error into a named result."

Next they ask

Next they ask

What is wrong with io.ReadAll for an HTTP request body?

Short answer

It keeps the whole body in memory and allocates more than it reads: in the run a megabyte cost 5118.3 KiB of allocations in total over 27 growth steps (the final slice has a capacity of 1114112 bytes, the rest are intermediate arrays). And the body size is set by the client. So the body is either limited — io.LimitReader or http.MaxBytesReader in the handler — or read as a stream: a JSON decoder, io.Copy into a file, bufio.Scanner line by line.

io.LimitReader has a quirk worth mentioning: it ends truncated input with an ordinary EOF, and the code will not find out that the data was longer. If that matters, read one byte more than the limit and check whether you reached it. How this works in a server is in the lesson on the HTTP server.

Next they ask

When do you need io.Pipe?

Short answer

When one library can only write to an io.Writer and another can only read from an io.Reader. The classic example is compressing data on the fly and handing it to an HTTP client as a request body: gzip.Writer writes to a PipeWriter in a separate goroutine, and the request reads from the PipeReader.

There is no buffer in a pipe: each Write waits until the data is taken. In the run the reader received one 1-megabyte Write in 32 chunks of 32768 bytes — exactly the size of the io.Copy buffer that was reading from the pipe. Two things are mandatory: the writer must close the pipe (Close or CloseWithError), or the reader will wait forever, and writing and reading must happen in different goroutines — otherwise the very first Write will block the program.

Next they ask

How do you test that code reads a stream correctly?

Short answer

With the testing/iotest package — that is what it is for. DataErrReader returns the last chunk together with EOF and catches the loop from Mechanism 1; HalfReader and OneByteReader return less than asked and catch code that expects a full buffer; ErrReader returns a given error on every Read; TimeoutReader returns iotest.ErrTimeout on the second read. Wrapping a test's input in them costs one line.

There is also the reverse check — iotest.TestReader(r, content): it tests your reader — reads it in chunks of different sizes until EOF and compares what it read with the expected content.

Common misconceptions

Claim

If Read returned an error, there is no data in that call

Actually

There is, and it must be handled first: the documentation allows returning n > 0 together with an error, including io.EOF. A loop that checked err before n collected "hello, wor" instead of "hello, world" on iotest.DataErrReader — with no error.

Claim

Read fills the whole buffer if there is enough data

Actually

It promises "up to len(p)", not len(p): with a buffer of 8 the reader returned 4 bytes, the pipe 3, 7 and 2, exactly one Write each. Exactly N bytes are read by io.ReadFull.

Claim

io.Copy always copies through its own 32 KiB buffer

Actually

Only if the source has no WriterTo and the destination no ReaderFrom. From a *bytes.Reader 100,000 bytes went out in one Write, Read was never called, zero allocations. And conversely: a wrapper with a single Read method hides the source's WriterTo.

Claim

If no Write returned an error, the data is written

Actually

Not with bufio.Writer: on top of /dev/full all hundred writes returned nil, and no space left on device came only from Flush. Without Flush the last 1808 bytes of 10,000 never went down at all.

Claim

defer f.Close() is all a file needs

Actually

Not for a file being written. A function with defer f.Close() and bufio without Flush returned nil and left a 0-byte file; gzip without Close cannot be read — unexpected EOF. A writer's Flush and Close errors are checked, and defer throws them away.

Claim

io.ReadAll is the simplest and cheapest way to read a stream

Actually

Simplest — yes, cheapest — no: a megabyte cost 5118.3 KiB of allocations in total, because the slice grows by appending; the final slice is about 1.06 MiB, the rest are intermediate arrays for the garbage collector. io.Copy through wrappers in the run kept no more than a little over one 32 KiB buffer — 32.5 KiB with a pipe, under one with MultiWriter.

Practice

Two problems. Answer first, then check against the real output: in both, the correct answer is taken from a run of the script rather than assigned.

Practice · predict the output

DataErrReader returns the last chunk together with io.EOF. The loop checks the error before printing the bytes. Then — io.ReadFull on input shorter than the buffer, and bufio.Writer without Flush. What will this code print?
r := iotest.DataErrReader(strings.NewReader("hello, world"))
buf := make([]byte, 5)
for {
n, err := r.Read(buf)
if err != nil {
	break
}
fmt.Print(string(buf[:n]), "|")
}
fmt.Println()
n, err := io.ReadFull(strings.NewReader("hello"), make([]byte, 8))
fmt.Println(n, err)
var out bytes.Buffer
w := bufio.NewWriter(&out)
fmt.Fprint(w, "report")
fmt.Println(out.Len())

Practice · estimate

10,000 writes of 16 bytes to a file: straight into an *os.File and through a bufio.Writer with Flush at the end. How many times slower is writing without the buffer?
times

Knowledge check

Question 1 of 5

Read returned n = 2 and err = io.EOF. What do you do with those two bytes?

Sources & further reading

6 SOURCES

  1. Package io — ReaderOfficial documentation. The Read contract. How many bytes: «Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered». The short read: «If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more». Data together with an error: «It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call». And the rule for the caller: «Callers should always process the n > 0 bytes returned before considering the error err». Checked against go doc io.Reader, go1.24.7.https://pkg.go.dev/io#Reader
  2. Package io — Writer, EOF, ReadFullOfficial documentation. The Write contract is stricter: «Write must return a non-nil error if it returns n < len(p)». How to compare with EOF: «Read must return EOF itself, not an error wrapping EOF, because callers will test for EOF using ==». ReadFull: «ReadFull reads exactly len(buf) bytes from r into buf» and «If an EOF happens after reading some but not all the bytes, ReadFull returns ErrUnexpectedEOF». Checked against go doc, go1.24.7.https://pkg.go.dev/io#ReadFull
  3. Package io — Copy, Pipe, TeeReaderOfficial documentation. What Copy copies with: «If src implements WriterTo, the copy is implemented by calling src.WriteTo(dst). Otherwise, if dst implements ReaderFrom, the copy is implemented by calling dst.ReadFrom(src)». Why a successful Copy does not return EOF: «A successful Copy returns err == nil, not err == EOF». For TeeReader: «There is no internal buffering - the write must complete before the read completes». For Pipe: «The data is copied directly from the Write to the corresponding Read (or Reads); there is no internal buffering». Checked against go doc, go1.24.7.https://pkg.go.dev/io#Copy
  4. io/io.go — copyBufferGo source code. Lines 407–419 on go1.24.7 (the same numbers on go1.25.1): first the src.(WriterTo) check on line 410, then dst.(ReaderFrom) on line 414, and only then the buffer — line 418: size := 32 * 1024; if the source is a *LimitedReader with a smaller remainder, the buffer shrinks to it (lines 419–425). The comment on the first check: «Avoids an allocation and a copy».https://github.com/golang/go/blob/go1.24.7/src/io/io.go#L407-L450
  5. Package bufio — WriterOfficial documentation. Why Flush: «After all data has been written, the client should call the Writer.Flush method to guarantee all data has been forwarded to the underlying io.Writer». The error is sticky: «If an error occurs writing to a Writer, no more data will be accepted and all subsequent writes, and Writer.Flush, will return the error». The default buffer size is defaultBufSize = 4096, bufio/bufio.go, line 19 on go1.24.7.https://pkg.go.dev/bufio#Writer
  6. Packages compress/gzip and os — CloseOfficial documentation. gzip.Writer.Close writes: «Close closes the Writer by flushing any unwritten data to the underlying io.Writer and writing the GZIP footer». os.File.Close: «Close will return an error if it has already been called». Checked against go doc, go1.24.7.https://pkg.go.dev/compress/gzip#Writer.Close