Address Parser in 2026: Tokenization, Ambiguity, Fallbacks

Parsing recovers structure that was present in the text: the postal code, the region code, the city, and the street line, in that order of reliability. It cannot recover information that was never written down. A string with no unit designator does not contain a unit number, and no amount of inference should invent one.

It also cannot confirm that the fields are true. Parsing "123 Main St, Springfield, IL 62701" produces coherent fields, and the address may still be fictional or the city may not match the ZIP. Parsing is a text operation. Verification is a data operation, and the two should never share a function name or a success message.

Stage one is normalization: trim, collapse internal whitespace, replace non-breaking spaces, normalize punctuation variants, and decide whether to uppercase. Keep the original untouched for the fallback. Stage two is segmentation: split on commas when they exist, otherwise split on the country grammar. Stage three is classification: label each segment as postal, region, city, street, or unit using the country rule set. Stage four is assignment: map labels to fields, resolve conflicts, and score the result.

Keeping the stages separate makes the failure modes separable. A segmentation bug looks different from a classification bug, and each stage can be unit tested with its own corpus. It also lets you stop early: if the country is unknown, skip classification entirely and return the segments with per-field confidence marked as unknown.

Field order is the single biggest source of wrong parses, because the same tokens move around. The United States, Canada, the United Kingdom, and Australia put the postal code after the city. Germany, France, the Netherlands, and Belgium put it before the city. Japan puts it first, before the prefecture. Brazil places the postal code after the state with a separator.

That is why the parser must know the country before it classifies anything. Inferring the country from the string itself is circular: the postal pattern can suggest one, but many patterns overlap. Ask for the country from the form, the locale, or an explicit selector, and mark the parse as low confidence when it is missing rather than guessing from a two-letter token that might be a state or a country.

Secondary unit designators are the most common ambiguity. "Apt", "Ste", "Unit", "Fl", and "Rm" are reliable when followed by an identifier, but "Unit 5" can also be a street name, and "Building 3" can be the primary address of a campus. A parser should extract a unit only when the designator is followed by a number or a short alphanumeric token and appears after the street number.

Street suffixes are the second trap. A token like "Way", "Court", or "Park" can be a suffix, a street name, or a city. Directionals have the same problem: "N" can be a predirectional, a street name abbreviation, or a region. Use position relative to the house number and the comma structure, and when two readings are both plausible, keep the original segment and lower the confidence instead of choosing.

Score each field, then the record. A field is high confidence when it matched an anchored country pattern and its position agreed with the expected order. It is medium when the pattern matched but the position was inferred, and low when it was assigned by elimination. A record is only high confidence when every required field is high and there are no unplaced segments.

Set a threshold and enforce it. Above the threshold, accept and store. Below it, keep the original string and flag the record for review rather than writing a guess into the structured columns. Expose the score and the reasons in the API response, because the reviewer needs to know which field failed, not just that something did.

A parse result should have explicit outcomes, not a boolean. Missing country means the grammar cannot run. Missing postal code matters in countries that have one and is normal in Hong Kong and the UAE. An unplaceable segment means the comma structure and the grammar disagree, and the record needs human eyes.

Give each outcome a stable code so callers can branch without parsing prose. Return the normalized input, the original input, the field values with confidence, and the list of unplaced segments. That payload lets a UI highlight the exact token that failed instead of showing a generic error at the top of the form.

Start with a small hand-verified corpus, one to three addresses per country, each with the expected field values written by someone who reads that country's mail. Twenty to forty cases is enough to catch structural regressions and small enough to review in a pull request. Store it as data, not as assertions buried in test code.

Then mutate each case mechanically: remove commas, change case, collapse double spaces, insert a non-breaking space, swap two lines, abbreviate a suffix, and append a unit. The expected output for a mutation is not always identical, but it should stay within a defined tolerance, and the confidence should fall rather than the fields silently changing. Track precision and recall per field so a fix for city does not quietly break postal.

Do not parse freeform notes, delivery instructions, or anything a human wrote as prose. "Leave at the side gate, second house after the blue one" has no fields, and extracting tokens from it produces confident garbage. Keep it as a text blob attached to the order.

Do not parse scripts your rule set does not cover. A Japanese address in Kanji needs a Japanese grammar; running a Latin-script tokenizer over it will split on the wrong boundaries and lose the prefecture. Do not parse addresses that have no street number or that use a building name as the primary locator, which is common in parts of Asia and in campus addresses. In all of these cases the correct output is the original string plus a clear "not parsed" status.