I Don’t See the R’s

“I don’t even see the R’s. All I see is 302, 1618, 19772, 198, 3504, 1134, 19772, 198, 101830, 198, 138322, 198, 1100, 302, 1618, 19772, 25644, 1100, 3504, 1134, 19772, 1100.” strawberry Strawbery strawberry Strawberry “strawberry ”Strawberry

A bald man looks toward a screen displaying several differently formatted versions of the word “strawberry.” The caption jokes about an AI or tokenizer that cannot perceive the letter R and instead interprets the word as a sequence of numerical token IDs. It is used as a reaction to mock language models, tokenization quirks, or overly literal computer vision.

• @profunctor_io / 10406 2024-09-19