Strings, runes and bytes: why len does not count characters, and a substring holds megabytes

What is really being asked

The topic looks like a warm-up, and that is the trap: the questions almost always come in pairs, where the second checks whether the reason behind the first was understood.

1. "What does len(s) return for a string with CJK characters in it?" — followed by: "and why?"
2. "What is a rune?" — followed by: "is a rune the same as a character?"
3. "Why are strings immutable?" — a question about substrings, not about safety.
4. "What does s[2:5] cost?" — followed by: "and what stays in memory?"
5. "What is wrong with += in a loop?" — followed by: "and at three pieces?"

The topic has a single spine, and it is worth saying out loud: a string is a header over an immutable run of bytes. Immutability makes a substring free, a free substring retains the array, and immutability is also what makes joining quadratic. Three different questions, one property.

Base: a string is bytes

A string holds text, and memory can only hold numbers. So between the text and the memory sits an agreement: which number writes down which mark. Such an agreement is called an encoding, and in Go there is only one — UTF-8.

It is deliberately uneven: frequent marks are written in one byte, the rest in two, three or four. A Latin letter takes one byte, a CJK character three, an emoji four. The number the encoding assigns to a mark is called a code point; in Go it has its own name — a rune.

Take a string of three marks of different lengths — "A世🙂" — and walk the whole entrance to the topic with it.

The length returns bytes. There are three marks, and the length is eight: one plus three plus four. A string's length in Go is not the number of marks but the room the text occupies.

Ranging walks marks. for range over a string steps through code points and yields "A", "世" and "🙂" — three steps, not eight. But the number it hands you alongside each mark is the byte offset where that mark starts: 0, 1 and 4.

Indexing yields a byte. s[0] is not the first mark but the first byte: 65, which happens to be the whole letter "A", because that letter is one byte long. And s[1] is 228, only the first third of the CJK character: the other two thirds sit in s[2] and s[3].

That is the whole entrance. Three answers to "how many are there" — eight bytes, three marks, eight positions to index — and none of them is the string's "real" number: which one is right depends on why you are asking.

That is already enough to answer the basic interview question: a string's length in Go counts bytes rather than characters, because the text is stored in UTF-8 where a mark takes between one and four bytes. Everything below is about where that difference cuts: why a rune is still not a visible mark, how slicing a string splits a mark in half, and what follows from what a string variable is made of.

Mechanism 1: one string, three frames of reference

The lesson carries that same string — "A世🙂" — through every mechanism. It is chosen to cover all three UTF-8 lengths at once: the Latin letter takes one byte, the CJK character three, the emoji four.

This is not pedantry. On a purely Latin string not one bug in this topic shows up: len matches the number of characters, s[0] gives the first character, range indices run consecutively. Which is exactly why such bugs live in the code until the first non-Latin input — and surface in production rather than in tests.

A run of bench/gostring/practice.go prints:

8        len(s) — bytes
3        utf8.RuneCountInString(s) — runes
65       s[0] — a byte, and also the whole letter "A"
228      s[1] — a byte, and it is a THIRD of the CJK character
3        len([]rune(s))
0 1 4    byte offsets from range
1        len(string(s[0]))
2        len(string(s[1]))

Three points are what matter here.

len counts bytes. Eight against three is one and the same string in two frames of reference, and neither of them is the "real" one. The right answer to "how many characters are there" depends on why you are asking.

s[i] is a byte, not a character. And here is the pair the string was chosen for: s[0] gives 65, and that really is the whole letter "A"; s[1] gives 228, and that is the first of three bytes. The first case works by accident — and thereby masks the second.

The last two output lines follow from that. string(s[i]) does not decode the byte: it converts a number into the rune with that code. From 65 comes "A", 1 byte long — right by coincidence. From 228 comes "ä", 2 bytes long — a character with no relation to the original string.

range yields byte offsets: 0 1 4, not 0 1 2. That is in the specification directly:

The practical consequence: an index from range is fit for slicing the string (s[i:]) but is not a character number. A character counter goes in a separate variable.

Mechanism 2: bytes, runes and graphemes — three different numbers

"A rune is a character" is the most durable simplification in the topic, and it is worth demolishing right away, before it works its way into decisions. A run of bench/gostring/internals.go:

The two middle rows are the same mark on screen with different rune counts. é exists in two forms: precomposed (U+00E9) and decomposed (e plus a combining acute U+0301). So a rune is not a character, and a rune count answers "how many characters" correctly only when the text is known to be in the precomposed form — which text from the outside world does not guarantee.

The last row takes the distinction to its limit: eighteen bytes, five runes, one visible mark. Those five runes are three emoji and two zero-width joiners.

What a person calls a character is called a grapheme cluster, and splitting into clusters is not in the standard library at all. That is the substantive interview answer: "count the characters" in the general case is solved by a separate package, while len and []rune answer two other questions.

The three levels are worth holding in mind as three:

bytes      the unit of storage        len(s)
runes      Unicode code points        utf8.RuneCountInString(s)
graphemes  what a person sees         a separate package

Mechanism 3: slicing cuts by bytes

String slice indices are byte indices, which means s[:n] with an arbitrary n can cut a UTF-8 sequence in half. From the run:

The second row is a CJK character cut in half. The program did not crash and said nothing: a string in Go is a sequence of arbitrary bytes, and invalid UTF-8 inside one is legal.

Note the rune counter on that row: it counted two runes where the second does not exist. Iteration substitutes the replacement character U+FFFD for every invalid byte — so the corruption turns not into an error but into a question mark in a diamond that users will notice.

The practical consequence is narrow and common: you cannot truncate text by length with a byte slice. A hundred-character preview cut as s[:100] will sooner or later split a character. Cut on the boundaries range gives — it walks exactly the starts of runes.

Mechanism 4: what is in the variable — and why a substring is free

The measurement prints the header sizes:

Those eight bytes of difference explain half the topic: a string has no capacity, because nothing can ever be appended to it.

Hence, immediately: s[0] = 'x' does not compile, and that is not caution for its own sake. Writing into a string would corrupt every substring looking into the same array — and they look into it precisely because a string is immutable. The property holds itself up.

A substring is free — and therefore expensive

Here is the pair they want to hear together.

The substring s[10:20] costs zero allocations: a new header points into the same array. And for exactly that reason:

Ten bytes retain eight megabytes. The collector reclaims an array whole or not at all; a reference into the array holds all of it.

Where this actually happens: a large service response is parsed, an identifier is taken out of it and put into a long-lived cache — and the response stays in memory along with the identifier. The symptom is characteristic: memory grows while the object count shows nothing.

The rule that follows is a narrow one: strings.Clone is needed only where a short piece outlives a long string. Cloning everything means paying in copies for a problem you do not have.

Mechanism 5: joining in a loop — why this is not about the speed of an operation

Before looking at the numbers it is worth deriving the answer from what has already been said. A string is immutable — that is the lesson's first claim. So "appending to a string" is impossible in principle: every += builds a new string and copies everything accumulated so far into it. On step k it copies k pieces, and over n steps the sum from 1 to n — the work is quadratic.

The prediction from that is unambiguous: the ratio against Builder must grow with the number of pieces. The growth, not the ratio itself: a single number says nothing, because it depends on the length of the loop.

The prediction holds: ×2.3 → ×23.3 → ×111.8. A growing ratio is the signature of quadratic work; if the work were linear, the column would stand still.

Allocations at a thousand pieces: 999 against 15 for Builder and 1 for Builder with Grow.

Builder writes into a growing buffer and hands it over as a string without a copy; Grow removes the intermediate expansions as well, when the final size is known in advance.

And the caveat without which the rule does harm. At two or three pieces a + b + c is faster than a Builder and reads better: the compiler joins such an expression with a single call. A Builder is for where the pieces are many and their number is not known in advance — that is, in a loop.

Deeper: conversions — what copies and what does not

From here the claims change kind, and the difference is worth saying. The contract says only that string and []byte are different types; everything below is what the current compiler does, and numbers taken on one particular machine.

This is where the topic carries its most outdated piece of common wisdom.

First. len([]rune(s)) — the very expression people are told to replace with utf8.RuneCountInString — allocates nothing. Measured:

The compiler recognises the shape len([]rune(s)) and counts runes in place without building a slice. The advice repeated in articles describes a compiler that has long since gone. You pay where the slice is actually needed — when it is kept and indexed.

Second. Some []byte → string conversions are free:

The rule is simple: a copy is needed when the string outlives the expression. If it is needed only inside one expression and never escapes, the compiler does not build it. Hence the practical consequence: looking up a map[string]T by a key held in a []byte can be done directly, with no unsafe and no manual workarounds.

How to answer in an interview

Short answer: a string in Go is a sequence of bytes in UTF-8, not a sequence of characters. That is why the length counts bytes, indexing yields a byte, and for range walks code points and hands back byte offsets. Add the layout as a second sentence: a string is a pointer and a length over immutable bytes, and both the free substring and the quadratic join follow from immutability.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

On len, answer with a number and a reason. "Bytes. For "A世🙂" it is eight against three runes: the letter takes one byte, the CJK character three, the emoji four."

On s[i], name the type and give both halves. "byte. In that same string s[0] is 65 and the whole letter "A", while s[1] is 228 and a third of the CJK character. And string(s[1]) gives not the character but an unrelated "ä": it converts a number into the rune with that code rather than decoding the byte."

On characters, separate the three levels. "Bytes, runes and graphemes are three different numbers. A decomposed é is two runes for one visible mark, an emoji family five runes for one. Splitting into graphemes is not in the standard library."

On runes, add the second half. "A rune is a code point, but not a character on screen: é may be two runes, and then the counter shows two for one visible mark."

On substrings, name both sides at once. "Zero allocations — and retention of the whole array. Ten bytes held eight megabytes for me; strings.Clone fixes it."

On joining, talk about growth, not about the ratio. "Quadratic: every += copies what has accumulated. At a thousand pieces it came out ×109 against a Builder. But at two or three pieces the plus is faster."

Next they ask

Common misconceptions

Practice

Two problems. Answer first, then check against the real output: in both, the correct answer comes from a run of the script, not from an assertion.

fmt.Println(len(s))
fmt.Println(utf8.RuneCountInString(s))
fmt.Println(s[0])
fmt.Println(s[1])
fmt.Println(len([]rune(s)))

var offsets []string
for i := range s {
offsets = append(offsets, strconv.Itoa(i))
}
fmt.Println(strings.Join(offsets, " "))

fmt.Println(len(string(s[0])))
fmt.Println(len(string(s[1])))

8
3
65
228
3
0 1 4
1
2

3
3
65
228
3
0 1 2
1
1

8
3
65
228
3
0 1 4
1
1

8
8
65
228
8
0 1 2 3 4 5 6 7
1
2

Knowledge check