!). Changelog: 30 Dec 2022 — Use Medium’s new code block for syntax highlighting 12 Jan 2022 — Improve clarity 5 Jan 2022 — Fix typos and improve clarity What do BERT, RoBERTa, ALBERT, SpanBERT, DistilBERT, SesameBERT, SemBERT, SciBERT, BioBERT, MobileBERT, TinyBERT and CamemBERT all have in common? And I’m not looking for the answer “BERT” 🤭. Answer: self-attention 🤗. We are not only talking ab