Skip to content

Commit f2afb7b

Browse files
derek73claude
andauthored
Read Portuguese given names a particle completes, in two tiers (#1118) (#1120)
* Read Portuguese given names a particle completes, in two tiers (#1118) rules/P10 read only Spanish-measured words, so Maria das Graças Silva and Maria de Lurdes Costa read the phrase as the start of the surname. Re-measured over the whole ORCID extract in Portuguese-speaking countries: eight words reach the 90% bar and are marked by default (lurdes, céu, glória, tarso, ribamar, kássia, amparo); seven more, below the bar but well above a surname's share, are marked default = false in pt.toml, read under locales.PT and locales.PT_BR alone (graças, graça, cássia, fatima, piedade). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the design-docs review of #1118 - Mark unaccented fatima on its Spanish counts (91 to 0), on by default in es.toml as fátima is, where it had sat in pt.toml's pack tier on its Portuguese counts alone; María de Fatima López read family 'de Fatima López' beside María de Fátima López's middle 'de Fátima'. - State the pack tier's actual cut (75% up to the bar, ten given records) in decisions/P10's Standing and the vocab README, and record that an unaccented spelling is measured on its own (kassia parts from kássia). - Label the ORCID agreement rows as the capped corpus and give the population that could move; re-measure every row after the fatima move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address the second design-docs review of #1118 - Standing: an unaccented spelling under ten given records takes its twin's reading (ceu, graca, gracas), as the Log and pt.toml say. - Record that a strict Portuguese choice does not read the words es.toml marks (de Fátima, de Fatima, do Socorro, de Lourdes), accepted here. - Pin unaccented Fatima's default reading with a rules/P10 example; it fails with es.toml's reading removed. - State the 75% cut as a chosen threshold, with the words either side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * decisions/P10: link the strict Portuguese question (#1122) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 8c1cc70 commit f2afb7b

16 files changed

Lines changed: 189 additions & 13 deletions

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -242,7 +242,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na
242242
- particles — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name (the particles whose word has no `given` reading, each stating `may_be_given = false`; a `given` reading on the word is what makes a particle possibly given, the phase 1 flag `may_be_given = true` refused since 2026-10-25). What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. Behind a title-particle (`St`, `Freiherr`, P4's Except) it does chain, and under a family-first order the fold cuts that chain as it cuts a run (`St de Mesnil Jean` → family `de Mesnil`, given `Jean`, #828). The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `Van Gogh` is given `Van`, family `Gogh` by default and family `Van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three, save one written in lower case in a mixed-case name under a family-first order, which the fold takes and which records none (`van Gogh` → family `van Gogh`, rules/P11, #670); a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). The internal `Lexicon._sets.particles_ambiguous`, the particles holding a `given` reading, is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either
243243
- bound given names — `BOUND_GIVEN_NAMES` (`bound_given` readings, e.g. "abdul", "abu"; one that is also a particle needs a `given` reading, a build check); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (ported from v1's `_join_bound_first_name`), and `BOUND_PHRASES` (#694, rules/P12, the two-word given names a bound word's `completed_by` lists, Arabic `عبد` before the names of God), each pair joined past the first name word, where P5's join does not reach
244244
- given names written short — `GIVEN_NAME_ABBREVIATIONS` (#789, a `given` reading carrying `same_as`, the full name: "chas", "wm", Filipino "ma"), the given names written short that rules/H2's shape yields to; nothing else reads it
245-
- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules`
245+
- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", "lurdes", the Spanish and Portuguese words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default, and a few Portuguese ones, "graças", only under a Portuguese choice, #1118), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules`
246246
- conjunctions — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479, rules/P8, `may_be_initial`, refused by the build on anything but a cased single letter), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set
247247
- maiden markers — `MAIDEN_MARKERS` (`birth_marker` readings, e.g. "née", "geb.") routing the following name to `maiden`; with an `addresses = "given"` title, the one kind a phrase may be (`z domu`)
248248
- `config/_ko_surnames.py`, private since 3.0 (`config/surnames.py` through 2.x, 2.8.2 warning of the move) and read as `Lexicon.default().surnames` — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271), still written in Python with an import-time `assert_normalized`: a segmentation list, outside the per-word model, like the locale packs' lists.

‎docs/customize.rst‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1187,12 +1187,15 @@ particle's words would otherwise run on into the family, the parser
11871187
reads them as a middle name, with or without the switch, if the
11881188
vocabulary marks the word behind the particle as a given name written
11891189
there (``after_particle``), and the switch then takes them into the
1190-
given name. The Spanish ones are marked, those that ORCID's researchers
1191-
from Spain and Spanish-speaking America write in the given box at least
1192-
nine times for each ten they also open a family box with it (``del
1193-
Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``: 23 words in
1194-
all on 2026-11-03). A phrase too often a surname to mark, as Portuguese
1195-
``do Carmo`` is, stays the family, the same shape as ``"João da Silva
1190+
given name. The words marked are those that ORCID's researchers write
1191+
in the given box at least nine times for each ten they also open a
1192+
family box with it: from Spain and Spanish-speaking America (``del
1193+
Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``), and from
1194+
Portugal and Brazil (``de Lurdes``, ``do Céu``, ``da Glória``, ``de
1195+
Tarso``). Portuguese words written as given names less surely
1196+
(``das Graças``, ``de Cássia``) are read only under ``locales.PT`` and
1197+
``locales.PT_BR`` (see :ref:`language-packs`). A phrase too often a
1198+
surname to mark, as Portuguese ``do Carmo`` is, stays the family, the same shape as ``"João da Silva
11961199
Santos"``, and so does one opening with a saint (``"Domingo Santa María
11971200
González"``):
11981201

‎docs/design/decisions/O7-two-given-names.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@
4343
- **Open:** which particle phrases complete a given name, and whether that is a reading on the word or a list the rule kind holds (the particle-chain bullet above), #704 (decided 2026-11-03, below), and whether this rule reads O6's given-name list for it (`María del Carmen` against `João da Silva`), #959 (declined 2026-11-03, below); deriving `DEVIATES` from the rules (decisions/vocabulary-model's 2026-10-29 4f entry, item 6), which this rule leaves as it found it, having no surface in the name to declare, #754.
4444
- 2026-10-30 #959 — THE WORD O6'S GIVEN-NAME LIST LEAVES IS THIS RULE'S. rules/O6 now leaves a listed given name where it stands (decisions/O6's entry of this date), so with both switches on the middle it leaves is this rule's to take where it is the unit beside the given name: `José Luis Pérez` reads given `José Luis`, family `Pérez`, where with the list empty O6 takes `Luis` first and the name keeps one given name. Where the given part holds two units before it, this rule takes the first and the listed word stays a middle (`Ana María Luis Pérez`: given `Ana María`, middle `Luis`), so a name of four words reads two and two only where O6 took its third (found by the #959 design-docs review). Nothing in this rule moved; its precedence clause states the reading with an example. Whether this rule reads the list itself is Open above.
4545
- 2026-10-30 #753, phase 3g, decided in chat — NEITHER `es` NOR `pt` TURNS THIS SWITCH ON. ORCID cannot score it: its given box holds the middle names too, so the switch moves no record's agreement (0 fixed, 0 broken in every Spanish- and Portuguese-speaking country measured). PSC, whose form has a middle-name box, scores it against the filer's own split, and filers put the second given name in that box: under a `es` choice the switch alone takes Spanish filers from 31.8% to 15.7% (1 fixed, 980 broken), and beside `locales.ES`'s `two_family_names` from 82.0% to 54.7%; Portuguese filers fall from 61.1% to 28.5%. CLDR's `es` renderings read it right (the 2026-10-30 #704 entry, 6 → 8 of 8). The two sources disagree about a convention, not about a misreading, and the pack follows the boxes callers fill: a caller whose data writes the compound given name whole turns the switch on beside the pack. Recompute: decisions/O6's phase 3g recipe with `PolicyPatch(two_given_names=True)` added.
46-
- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`).
46+
- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete. Its Portuguese clause, that no Portuguese-speaking word reaches the bar, is superseded by decisions/P10's 2026-11-06 #1118 entry: over the whole ORCID extract eight words do, and six more are marked for a Portuguese choice alone; and the unaccented `fatima`, which this entry did not weigh, passes on its Spanish counts.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`).
4747
- **The shape: a property on the word behind the particle.** The particle cannot carry it (`del` opens `del Carmen` and `del Río` alike), and the question is a vocabulary one (the 2026-10-30 entry's particle-chain bullet). Weighed: (A) a phrase reading (`text = "del carmen"`), most precise, telling Spanish `de la Luz` from Portuguese `da Luz`, but a third kind of phrase reading beside a birth marker and a given-addressing title; (C) a list the rule holds as a `Policy` field or `[rules]` key, vocabulary in `Policy` that no choice of languages or strict parser reaches; (D) O6's given-name list (`Lexicon.given_names`), no new data, but it would split `de la Cruz` (175 family boxes in ORCID against 38 given), `de León` (151 against 5) and `de la Rosa`, and is empty by default and has no Portuguese list. B was chosen: `after_particle = true` on a `given` reading, one word, compiled into the internal `PARTICLE_GIVEN_NAMES`. Its cost against A is a word whose phrase reads both ways by particle, which is left out (`luz`).
4848
- **The reach.** Given-first, no family comma (a comma, and both family-first orders, already read `Silva, Maria do Carmo` given `Maria do Carmo` through the middle path), the given name one word with no middle between it and the chain, the chain opening with particles and then the marked word, and a name word left after them for the family, one with a letter that is no connective standing alone (a family of particles alone never reaches the path: assign leaves a trailing particle the family by itself, the phrase a middle, found by the second #1057 review), and no Turkic marker behind the phrase under that rule, as for the middle path (both found by the #1057 review: `María del Carmen y` and `Ali del Carmen oglu` had moved): `María del Carmen` alone keeps family `del Carmen`, as `Juan Carlos` keeps family `Carlos`. A given name of two words (`María-José del Carmen García`) keeps what it has, and a middle between leaves the chain to the family, this rule taking that middle instead (`Ana María del Carmen García` reads given `Ana María`). With rules/O6 on too, O6 leaves the phrase where it stands (decisions/O6, this date), so `María del Carmen de la Vega` reads given `María del Carmen` with both on, where the 2026-11-03 #959 entry in decisions/O6 recorded family `del Carmen de la Vega`; and the family the split leaves keeps its chained words as one unit, so `María del Carmen García López` gives no `surname2` (accepted in the rule).
4949
- **The words** (es.toml; each records its counts): `carmen`, `ángeles` and `angeles`, `pilar`, `rosario`, `valle`, `caridad`, `socorro`, `mar`, `rocío` and `rocio`, `lourdes`, `mercedes`, `maría` and `maria`, `milagro`, `milagros`, `cisne`, `fátima`, `concepción`, `nieves`, `consuelo`, `jesús` — 23. Measured 2026-11-03 on ORCID's `raw.jsonl` of the 2026-10-27 fetch (20,000 records a country at most): for each word, the records whose given box ends with a particle run (`Lexicon.default()`'s particles, NFC lower case) and the word, against those whose family box opens with a particle run and the word and holds more words after them, the shape that competes; kept where the given side is at least 90% with ten given records or more, counted over Spain and Spanish-speaking America (ES MX AR CO CL PE VE EC UY BO PY CR GT CU DO HN SV NI PA PR AD) and, separately, Portugal, Brazil and Portuguese-speaking Africa (BR PT AO MZ CV GW ST TL). `carmen` 1,261 to 5, `valle` 235 to 26 (exactly 90.0%), `rosario` 260 to 27. Left out: no Portuguese-speaking word reaches the bar (`carmo` 21 to 6, `rosário` 21 to 8, `conceição` 25 to 25, `fátima` 32 to 4, which the Spanish counts carry, 27 to 1); unaccented `jesus`, 203 to 16 in Spanish-speaking countries but 67 to 41 in Portuguese-speaking ones; `luz` (12 to 6, and Portuguese `da Luz` 4 to 5), `dios`, `paz`, `cruz`; and two that pass by count but are no compound given name, `c.` (an initial) and `piero` (Italian `del Piero`, 10 to 0). The Catalan phrases (`dels Àngels`, `de les Neus`, `de la Mercè`) barely occur, so no Catalan word is marked. The PSC register agrees where it can, its filers putting `del Carmen` in the middle-name box 21 times and the surname box 2, `del Mar` 11 and 0, `del Pilar` 10 and 1. `dels` stays out of `ca.toml` (the reason void since decisions/P10's 2026-11-04 entry, which holds the question Open): the property moves nothing with the switch off, so a `dels` particle would still move `Maria dels Àngels Puig` from middle `dels Àngels` to family `dels Àngels Puig` by default (the #704 comment of 2026-10-31).

0 commit comments

Comments
 (0)