❝HN 片单

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy · 17 位 HN 用户推荐 · 2023–2026

Karpathy 两小时现场编码从零搭建 GPT,被广泛视为理解 Transformer 的最佳教程

  1. I recommend this video on the subject. "Let's build GPT: from scratch, in code, spelled out." [link]
    — amrb, 2023
  2. Great compilation, would be great to see Vision Transformer (ViT) included. Andrej Karpathy's GPT video is a must have companion for this [link] I was going nuts trying to grok Key, Query,Position and Value until Andrej broke it down for me.
    — yonz, 2023
  3. For those interested I would also check out Andrej Karpathy's YouTube video on building GPT from scratch: [link]
    — lspears, 2023
  4. Very impressive. Recently I watched this really amazing lecture on building GPT from scratch from Karpathy, I was blown away: [link]
    — durdn, 2023
  5. - Whether the text generated by LLMs is factual or not is purely coincidental. I would highly recommend watching and working though the code (if possible) of NanoGPT by Andrej Karpathy. [link]
    — qazpot, 2023
  6. I agree that Karpathy's YouTube video is an excellent resource for understanding Transformers from scratch. It provides a hands-on experience that can be particularly helpful for those who want to implement the models themselves. Here's the link to the video titled "Let's build GPT: from scratch, in code, spelled out": [link]
    — pankajdoharey, 2023
  7. This MIT lecture on recurrent neural networks provides a lot of context on why attention was so groundbreaking, and how things were solved before: [link] And for the Attention Is All You Need paper itself, Andrej Karpathy's "GPT From Scratch" talk is a really great walkthrough of how to make this work in practice, all condensed into two hours of live coding: [link]
    — btown, 2023
  8. Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on the topic. Here is the link: [link]
    — wesammikhail, 2023
  9. I love all the series about writing your own X in 100 lines of code. It gives you the understanding of technology and removes a lot of unnecessary details. The great examples of this are 'A from-scratch tour of Bitcoin in Python' [link] and 'Let's build GPT: from scratch, in code, spelled out' [link] from Andrej Karpathy
    — Alpi, 2023
  10. [0] [link] , [link] [1] [link]
    — FranklinMaillot, 2023
  11. [0]: The original paper: [link] [1]: Full walkthrough for building a GPT from Scratch: [link]
    — techbruv, 2023
  12. +1, Andrey is an amazing educator! I'd also recommend his [link] and checking out his github repos. MinGPT, for example, implements a small gpt model that's compatible with HF API, whereas more modern nanoGPT shows how to use newer features such as flash attention. The quality of every video, every blog post is just so hig
    — villedespommes, 2024

← HN 片单