Skip rune decoding for ASCII bytes in ToChars

utf8.DecodeRune already fast-paths ASCII, but it is too complex to inline
(cost 201 against a budget of 80), so a mostly-ASCII line pays one call
per byte just to be told the byte is ASCII.

Only the run after the first non-ASCII byte reaches the decode loop, so
the gain depends on where that byte falls.

- 70-rune ASCII line: 145ns -> 42ns in the decode loop
- Ingestion of 1.4M mostly-ASCII paths behind a Hangul prefix, where the
  loop covers the whole line: 377ms -> 233ms
- The same paths with the Hangul at the end, where it covers six bytes:
  207ms -> 196ms
- Break-even sits at ~100% non-ASCII runes: still 1.01x at 95%. Only a
  line holding no ASCII byte at all loses, by ~0.14ns per rune, ~5% of
  the loop
This commit is contained in:
Junegunn Choi
2026-08-08 11:08:01 +09:00
parent 759b7c3283
commit 793e58b558
+7
View File
@@ -77,6 +77,13 @@ func ToChars(bytes []byte) Chars {
runes[i] = rune(bytes[i])
}
for i := bytesUntil; i < len(bytes); {
// utf8.DecodeRune has an ASCII path of its own, but it is too complex
// to inline, so a mostly-ASCII line pays one call per byte for it.
if b := bytes[i]; b < utf8.RuneSelf {
runes = append(runes, rune(b))
i++
continue
}
r, sz := utf8.DecodeRune(bytes[i:])
i += sz
runes = append(runes, r)