Attention in transformers, step-by-step | Deep Learning Chapter 6
3Blue1Brown · 5 位 HN 用户推荐 · 2024–2026
Unfortunately, that's it. I kept scrolling for a while longer through pages and pages of thank-yous. And that's typically my experience with comments on thoughtful videos. It's not usually like the HN comments where you can actually get a lot of value from the comments alone, they're just... shallow. Sometimes nice shallow, but still shallow. [0] [link]
— lolinder, 20243/ Andrej Karpathy has really good series of intros: [link] Let's build GPT: from scratch, in code, spelled out. [link] GPT with Andrej Karpathy: Part 1 [link] 4/ 3Blue1Brown: But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning [link] Attention in transformers, visually explained | Chapter 6, Deep Learning [link] Full 3Blue1Brown Neural Networks playlist [link]
— MAXPOOL, 2024most human person's experience with basic education ( [link] ). Though it is obvious that humans can do more than that too. And while "voting" that may be a close-enough metaphor for the multi-layer perceptron layers, it is not what is happening in the attention layers. I suggest checking out 3blue1brown's videos on gradient descent [link] and "How might LLMs store facts" [link] versus how attent
— superkuh, 2024And also why it is so much harder to determine what it's "thinking". If you aren't familiar with hyper words, this is an amazing series: [link]
— onlyrealcuzzo, 2026If you want the actual maths instead of handwaving, I recommend: [link] For something gentler, 3Blue1Brown: [link] (this is part of a series)
— wren6991, 2026