1966–70s · Sequence Labelling
Tag the whole sentence at once, so context can outvote a word's own bias.
pos-markov/'s baseline tagger looks at one word at a time: "rose" is tagged the same way everywhere, because the tagger is a pure function of a word's spelling. A Hidden Markov Model, decoded with the Viterbi algorithm, tags a whole sentence at once, so an ambiguous word gets resolved by its neighbours instead of by itself alone.
Two tables drive this: transition probabilities (how likely a Verb is to follow a Pronoun) and emission probabilities (how likely a specific word is, given the tag). The tags are "hidden" — you see the words, not the tags — so tagging means finding the tag sequence most likely to have produced the sentence. Trying every sequence is exponential. Viterbi finds the best one in linear time, using the same dynamic-programming trick as edit-distance/'s matrix: build the best score reaching each (word, tag) cell from the best scores one position back, keep a backpointer, then read the winning path off the end.
Honest training data: this repo has no hand-annotated corpus. Transition counts come from auto-tagging the real sonnets with the baseline tagger — noisy per-word, but the sequence patterns of English survive. Emission counts add a handful of hand-written ambiguity seeds for classic ambiguous words: "rose", "light", "still". Without them the model could never learn that "rose" is anything but a flower, since every real occurrence in the sonnets is exactly that.
| tag ↓ / word → | they | rose |
|---|---|---|
| Noun | −11.55 | −15.07 |
| Verb | −10.83 | −14.74 |
| Adjective | −10.42 | −19.38 |
| Adverb | −10.61 | −17.64 |
| Pronoun | −6.77 | −18.31 |
| Determiner | −10.28 | −17.85 |
| Preposition | −10.87 | −17.52 |
| Conjunction | −10.68 | −18.16 |
The real trellis for "they rose" — every cell is the best-path log-probability of reaching that (word, tag) pair. Orange is the winning path: Pronoun → Verb. The baseline tagger says "rose" is a Noun here, unconditionally. Viterbi says Verb, because the Verb cell in the "rose" column (−14.74) beats the Noun cell (−15.07) — a margin built entirely from context, since both tags start from the same word.
Whole-sentence accuracy can hide which tags fail. On a small hand-labelled ambiguity set, this HMM gets 75% of tokens right—but its adjective recall is zero: every true adjective became a noun. A confusion matrix makes that failure visible immediately, while per-tag precision, recall, and F1 distinguish false alarms from missed tags.
actual \ predicted Noun Verb Adj Pron Det Noun 3 0 0 0 0 Verb 1 2 0 0 0 Adjective 2 0 0 0 0 Pronoun 0 0 0 2 0 Determiner 0 0 0 0 2 accuracy .750 · macro-F1 .693
The benchmark is intentionally tiny and disclosed, not a claim of production quality. Its job is to show why one aggregate number is never enough.
The training data is synthetic where it matters most. The transition table is real; the disambiguating emission counts for "rose", "light", and "still" are hand-picked numbers, not observed frequencies. A tagger trained this way "knows" exactly the ambiguities its author thought to seed, and not one more.
Only one word of tag history. This is a bigram HMM — the next tag depends only on the immediately preceding tag, the same context-window limit every bigram Markov chain here has.
It still doesn't know what words mean. "Rose" resolves to Verb or Noun by pattern statistics over tags, not by understanding that one is a flower and the other a motion. Two unrelated ambiguous words with identical tag patterns would be resolved identically.
Smoothing is uniform and crude. Add-1 smoothing treats every unseen combination as equally unlikely, which isn't true. Modern taggers use better smoothing, or skip HMMs for neural sequence models that learn a continuous representation instead of a lookup table.