A high-performance Go package for measuring the monospace display width of strings, UTF-8 bytes, and runes.
go get github.com/clipperhouse/displaywidthpackage main
import (
"fmt"
"github.com/clipperhouse/displaywidth"
)
func main() {
width := displaywidth.String("Hello, 世界!")
fmt.Println(width)
width = displaywidth.Bytes([]byte("🌍"))
fmt.Println(width)
width = displaywidth.Rune('🌍')
fmt.Println(width)
}For most purposes, you should use the String or Bytes methods. They sum
the widths of grapheme clusters in the string or byte slice.
Note: in your application, iterating over runes to measure width is likely incorrect; the smallest unit of display is a grapheme, not a rune.
If you need the individual graphemes:
import (
"fmt"
"github.com/clipperhouse/displaywidth"
)
func main() {
g := displaywidth.StringGraphemes("Hello, 世界!")
for g.Next() {
width := g.Width()
value := g.Value()
// do something with the width or value
}
}There is one option, displaywidth.Options.EastAsianWidth, which defines
how East Asian Ambiguous characters
are treated.
When false (default), East Asian Ambiguous characters are treated as width 1.
When true, they are treated as width 2.
You may wish to configure this based on environment variables or locale.
go-runewidth, for example, does so
during package initialization.
displaywidth does not do this automatically, we prefer to leave it to you.
You might do something like:
var width displaywidth.Options // zero value is default
func init() {
if os.Getenv("EAST_ASIAN_WIDTH") == "true" {
width = displaywidth.Options{EastAsianWidth: true}
}
// or check locale, or any other logic you want
}
// use it in your logic
func myApp() {
fmt.Println(width.String("Hello, 世界!"))
}This package implements the Unicode East Asian Width standard (UAX #11), and handles version selectors, and regional indicator pairs (flags). We implement Unicode TR51. We are keeping an eye on emerging standards.
clipperhouse/displaywidth, mattn/go-runewidth, and rivo/uniseg will
give the same outputs for most real-world text. Extensive details are in the
compatibility analysis.
If you wish to investigate the core logic, see the lookupProperties and width
functions in width.go. The essential trie generation logic is in
buildPropertyBitmap in unicode.go.
cd comparison
go test -bench=. -benchmemgoos: darwin
goarch: arm64
pkg: github.com/clipperhouse/displaywidth/comparison
cpu: Apple M2
BenchmarkString_Mixed/clipperhouse/displaywidth-8 5917 ns/op 285.11 MB/s 0 B/op 0 allocs/op
BenchmarkString_Mixed/mattn/go-runewidth-8 14321 ns/op 117.80 MB/s 0 B/op 0 allocs/op
BenchmarkString_Mixed/rivo/uniseg-8 22430 ns/op 75.21 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 5979 ns/op 282.15 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/mattn/go-runewidth-8 23914 ns/op 70.54 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/rivo/uniseg-8 20320 ns/op 83.02 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/clipperhouse/displaywidth-8 78 ns/op 1630.93 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/mattn/go-runewidth-8 1166 ns/op 109.81 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/rivo/uniseg-8 1664 ns/op 76.91 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3196 ns/op 226.54 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/mattn/go-runewidth-8 4746 ns/op 152.55 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/rivo/uniseg-8 6594 ns/op 109.80 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3348 ns/op 503.84 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Mixed/mattn/go-runewidth-8 5345 ns/op 315.60 MB/s 0 B/op 0 allocs/op
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3390 ns/op 497.58 MB/s 0 B/op 0 allocs/op
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15335 ns/op 110.01 MB/s 0 B/op 0 allocs/op
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 257 ns/op 498.09 MB/s 0 B/op 0 allocs/op
BenchmarkRune_ASCII/mattn/go-runewidth-8 262 ns/op 488.63 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1327 ns/op 545.49 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Emoji/mattn/go-runewidth-8 2219 ns/op 326.32 MB/s 0 B/op 0 allocs/op
BenchmarkTruncateWithTail/clipperhouse/displaywidth-8 3858 ns/op 45.88 MB/s 192 B/op 14 allocs/op
BenchmarkTruncateWithTail/mattn/go-runewidth-8 8158 ns/op 21.70 MB/s 192 B/op 14 allocs/op
BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8 3439 ns/op 66.59 MB/s 0 B/op 0 allocs/op
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8 10430 ns/op 21.96 MB/s 0 B/op 0 allocs/op
Here are some notes on how to make Unicode things fast.
