std::collation
Collation: ordering text as a reader of a language orders it. < on
String compares UTF-8 bytes, and collation is never implicit, so
"Zorn" < "apple" and "Ärlig" sorts after every ASCII word. A
Collator orders strings by the Unicode Collation
Algorithm with the CLDR rules of a locale: in German ä sorts with a,
in Swedish after z, and in Polish ł is a letter after l.
Make a collator once and reuse it: sort with
Iter.sort_with(names, &Collator.compare(sv, &1, &2)), or keep each
string’s Collator.sort_key and compare those. A program carries the
collation data of the locales its nomi.toml lists; a collator for a
locale it does not carry orders as that locale’s nearest listed parent,
or as the root order every language starts from. The Chinese and Korean
tailorings, 160 to 250 KB each, are carried only when the program lists
those locales. Case mapping for a language and caseless matching are
String.uppercase_in, String.lowercase_in, String.capitalize_in and
String.fold_case (std::strings).
Import with import std::collation.
Exports
Section titled “Exports”The order
Section titled “The order”The order is the Unicode Collation Algorithm (UTS #10) over CLDR’s root
collation, with the locale’s CLDR tailoring laid over it (UTS #35 part 5):
German keeps ä with a, Swedish puts å, ä and ö after z,
Polish puts ł after l, Czech ch after h, Lithuanian y with i,
Chinese sorts Han by pinyin before Latin, and Traditional Chinese by
stroke count. Strings are compared in NFD, so canonically equivalent
strings are equal at every strength below Identical. A Han character a
tailoring does not list sorts by its code point, the core ideographs
before the rest, and an unassigned code point after every assigned one.
The locale’s tag chooses among a language’s collations and sets the
options: -u-co- names a CLDR collation type (de-u-co-phonebk sorts ü
as ue; es-u-co-trad sorts ch and ll as letters; zh-u-co-stroke),
and -u-ks- (level1 to level4, identic), -u-kn-true,
-u-kf-upper or -u-kf-lower, and -u-ka-shifted set what strength,
numeric, case_first and ignore_punctuation set. The settings apply
in order: the tailoring’s own (Danish puts upper case first), then the
tag’s keywords, then each argument of Collator.new that is not its
default.
A locale whose collation the program does not carry orders as its nearest
carried parent along the collation chain (the collations component of
CLDR’s parent locales, else the main ones short of root, else the tag less
its last subtag: zh-Hant takes zh’s stroke order), else in the root
order.
type Collator
Section titled “type Collator”type Collator String
opaque — construction surface is private to its defining module
Compares strings as readers of one language order them. Make it with
Collator.new. A Collator is Equatable and Hashable, and its
Debug is the call that makes it.
The locale’s tag can ask for a different order of the language: a
-u-co- keyword picks a CLDR collation type (de-u-co-phonebk sorts ü
as ue, es-u-co-trad sorts ch after c and ll after l, as
letters of their own, zh-Hant sorts by stroke),
and -u-kn-true orders digits by value, -u-kf-upper puts upper case
first and -u-ks-level1 compares base letters only, as the arguments
to Collator.new do.
sv = Collator.new(Locale"sv")
assert Iter.sort_with(["ö", "z", "a", "å"], &Collator.compare(sv, &1, &2)) == ["a", "z", "å", "ö"]
Collator.new
Section titled “Collator.new”fn new(locale: Locale, strength: Strength = Strength.Tertiary, numeric: Bool = False, case_first: CaseFirst = CaseFirst.Default, ignore_punctuation: Bool = False) -> Collator
type function on Collator
A collator for locale. strength is how fine a difference counts;
numeric orders a run of decimal digits, of any script, by its value
(file9 before file10); case_first moves upper case before or
after lower case; ignore_punctuation is UTS #10’s shifted
alternate: spaces and punctuation are ignored at the first three
levels and compared at the quaternary, so they count only when the
strings are otherwise equal (de luge with deluge). An argument
left at its default keeps what the locale’s tag and its collation
say.
de = Collator.new(Locale"de")
assert Collator.compare(de, "Ärger", "Arzt") == Less
loose = Collator.new(Locale"en", strength: Strength.Primary)
assert Collator.compare(loose, "résumé", "RESUME") == Equal
files = Collator.new(Locale"en", numeric: True)
assert Collator.compare(files, "file9", "file10") == Less
Collator.compare
Section titled “Collator.compare”fn compare(collator: Collator, a: String, b: String) -> Ordering
type function on Collator
Orders a and b as the collator’s locale does.
sv = Collator.new(Locale"sv")
assert Collator.compare(sv, "Zorn", "Åberg") == Less
assert Collator.compare(Collator.new(Locale"de"), "Zorn", "Åberg") == Greater
Collator.sort_key
Section titled “Collator.sort_key”fn sort_key(collator: Collator, text: String) -> Bytes
type function on Collator
The bytes that order text under the collator: two strings’ keys
compare, under Bytes’ Comparable order, as Collator.compare
compares the strings. Compute each
string’s key once to sort or index many strings, or to store the
order. A key is only meaningful beside keys from the same collator
and the same version of Nomi.
pl = Collator.new(Locale"pl")
assert Collator.sort_key(pl, "łoś") > Collator.sort_key(pl, "lis")
assert Collator.sort_key(pl, "łoś") < Collator.sort_key(pl, "mysz")
Collator.inspect
Section titled “Collator.inspect”impl Debug for Collator
fn inspect(collator: Collator) -> String
impl Debug.inspect
The call that makes the collator, its arguments that are not defaults included.
assert Debug.inspect(Collator.new(Locale"sv")) == "Collator.new(Locale\"sv\")"
assert Debug.inspect(Collator.new(Locale"en", strength: Strength.Primary, numeric: True))
== "Collator.new(Locale\"en\", strength: Strength.Primary, numeric: True)"
Collator.equal
Section titled “Collator.equal”impl Equatable for Collator
fn equal(a: Collator, b: Collator) -> Bool
impl Equatable.equal
Collator.hash
Section titled “Collator.hash”impl Hashable for Collator
fn hash(value: Collator) -> Int
impl Hashable.hash
enum Strength
Section titled “enum Strength”enum Strength Primary Secondary Tertiary Quaternary Identical end
How fine a difference a collator tells apart. Each level breaks the ties of the one before it.
Variants
-
PrimaryBase letters only:
a,áandAare equal. -
SecondaryBase letters and accents:
aandAare equal,áis not. -
TertiaryLetters, accents and case: the default.
-
QuaternaryAlso what
ignore_punctuationignores, and Japanese hiragana from katakana. -
IdenticalEvery difference: strings equal at every level compare by their code points.
Strength.inspect
Section titled “Strength.inspect”impl Debug for Strength
fn inspect(value: Strength) -> String
impl Debug.inspect
Strength.equal
Section titled “Strength.equal”impl Equatable for Strength
fn equal(a: Strength, b: Strength) -> Bool
impl Equatable.equal
enum CaseFirst
Section titled “enum CaseFirst”enum CaseFirst Default Upper Lower Off end
Where upper case sorts beside lower case.
Variants
-
DefaultAs the locale’s collation puts it: Danish and Maltese upper case first, most languages lower case first.
-
UpperUpper case before lower case.
-
LowerLower case before upper case.
-
OffAs the tertiary weights fall, with no rule for case: lower case first for most letters, even in Danish.
CaseFirst.inspect
Section titled “CaseFirst.inspect”impl Debug for CaseFirst
fn inspect(value: CaseFirst) -> String
impl Debug.inspect
CaseFirst.equal
Section titled “CaseFirst.equal”impl Equatable for CaseFirst
fn equal(a: CaseFirst, b: CaseFirst) -> Bool
impl Equatable.equal