Transformers, the tech behind LLMs | Deep Learning Chapter 5
3Blue1Brown · 11 位 HN 用户推荐 · 2024–2026
3Blue1Brown 用可视化讲解 GPT 与 Transformer 原理,直观易懂
To quote GPT-4, "AI's 'creativity' is more about remixing and reinterpreting the information it has been fed". I think the degree to which a person perceives human-like creativity from AI/LLMs is inversely proportional to their understanding of how they work. Here's a great video that can help demystify GPTs: [link]
— CharlesW, 20243Blue1Brown has a fantastic video on GPT that I'd highly recommend called "But what is a GPT? Visual intro to transformers" [1]. You'll need to have some background in mathematical reasoning, but not too much. He does a great job of drawing pictures to show the mathematical transformations that happen under the hood. [1] [link]
— Wilduck, 20243/ Andrej Karpathy has really good series of intros: [link] Let's build GPT: from scratch, in code, spelled out. [link] GPT with Andrej Karpathy: Part 1 [link] 4/ 3Blue1Brown: But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning [link] Attention in transformers, visually explained | Chapter 6, Deep Learning [link] Full 3Blue1Brown Neural Networks playlist [link]
— MAXPOOL, 20243Blue1Brown has a pretty great video series walking through Transformers: [link]
— humansareok1, 2024In practice most LLM systems use this distribution (along with a “temperature” multiplier) to make a weighted random choice among the tokens, giving the illusion of nondeterminism. But there’s no fundamental reason you couldn’t for example always choose the most likely token, yielding totally deterministic output. This is an excellent and accessible series going over how transformer systems work
— semiquaver, 2025This video explains the direction-encodes-trait topic very well IMO: [link]
— manmal, 2025[1] [link] [2] [link]
— matthewolfe, 2025Good intro here that talks about how embeddings work! [link]
— FaisalAbid, 2025I really recommend watching this section of this video. Embeddings do encode plenty of "human knowledge" into the vector values and their relations to each other. [link]
— gf000, 2026LLMs are explicitly designed to handle, and also possibly 'learn' from different tokens encoding similar information. I found this video from 3blue1brown very informative: [link]
— dannyw, 2026It's chapter 5. Start at chapter 1 if you want more background on neural nets and backprop. [link]
— robrenaud, 2026