TL-003 · for teachers
- Groep 6 to 8
- 15–25 minutes for all three levels
- How a chatbot turns text into numbered pieces, and what that costs
- After the worksheets Words in pieces (WS-L2-1A to 3B)
- No account, nothing saved or sent
Open the activity Open worksheet 1B
What pupils do
What it teaches, and why it is true of real AI
How SNIP works
SNIP and real chatbots
| SNIP | Real chatbots | |
|---|---|---|
| Size of the list | ||
| Counted from | ||
| What a piece can hold | ||
| Languages | ||
| The chopping rule | ||
| What comes next |
What we simplified, and why
What it does not teach
In class
Sources
- Gage, P. (1994). A new algorithm for data compression. The C Users Journal, 12(2). (byte-pair encoding, first as compression)
- Sennrich, R., Haddow, B. & Birch, A. (2016). Neural machine translation of rare words with subword units. Proceedings of ACL 2016. (byte-pair encoding for language: the method used to make SNIP's lists)
- Petrov, A., La Malfa, E., Torr, P. & Bibi, A. (2023). Language model tokenizers introduce unfairness between languages. NeurIPS 2023. (the same text costs very different numbers of pieces in different languages)
- Kaushal, A. & Mahowald, K. (2022). What do tokens know about their characters and how do they know it? NAACL 2022. (a model gets whole pieces, so tasks about the letters inside them are harder)
- Walan; Liu et al.: on treating machines as people (anthropomorphism), and on teenagers who think a chatbot works like a search engine. Summarized in Vonk's design philosophy.