Lexical analysis

Word Frequency

Paste any text and count the words that appear most often, or switch to word pairs to see which words tend to follow one another. It is a hands-on introduction to tokenization and n-grams — the very first thing a language model does with text: break it into pieces and notice which pieces follow which.