❝HN 片单

The spelled-out intro to neural networks and backpropagation: building micrograd

Andrej Karpathy · 14 位 HN 用户推荐 · 2022–2025

Karpathy 从零手写 micrograd,用代码直观讲透神经网络与反向传播

  1. I recommend watching Andrej Karpathy's recent videos on backprop and neural nets. Makes it really simple and intuitive. The spelled-out intro to neural networks and backpropagation: [link]
    — Geee, 2022
  2. Starts here [0] and most recent is building GPT from scratch in code. [link]
    — ian0, 2023
  3. h Hinton -- love the man but extremely mathematically technical for foundation! D:). Karpathy might have a really good one, I think I saw a zero-to-hero course from him a little while back that seemed really good. Andrej (practically) got me into deep learning via some of his earlier work, and I really love basically everything that I've seen the man put out. I skimmed the first video of his from
    — tysam_and, 2023
  4. I highly recommend the video from karpathy [link] - This explains the same idea spelled out in code. It really helped me understand the mathematical expression graph of neural networks. He uses graphviz to show the expressions and initially calculate gradients manually and then implement back propagation.
    — gavi, 2023
  5. Andrej Karpathy's "zero to hero" series [1] was how I learned the fundamentals of this stuff. It's especially useful because he explains the why and provides intuitive explanations instead of just talking about the what and how. Would recommend it if you haven't checked it out already. [1] [link]
    — radq, 2023
  6. Interesting video by Andrej Karpathy building an autograd engine, quite insightful: [link]
    — Cef111, 2023
  7. Watch Karpathy's recent lectures. They're gold. Start here[1] with micrograd[2]. It doesn't use linear algebra/matrices to start, but the principles are the same. The matrix multiplication is how the weights of the connections between neurons and the input values are combined (to form an activation value that then may lead to that neuron "firing" or not, depending on whether it passes some thresho
    — paulsmith, 2023
  8. In short you need to understand: vector, linear combination, cross product, partial derivative, chain rule and finding global minimum. If you have basics of linear algebra it’s easy to grok this video. [link]
    — machiaweliczny, 2023
  9. imo linear algebra is the main thing to refresh yourself on, as most of NN is based on matrix multiplication. The calculus is mostly in the gradient descent which in practice is abstracted away, so having a few decades old understanding might be good enough. For practical implementation Andrej Karpathy has a good series of videos going over a lot of the concepts starting here [link]
    — sprobertson, 2024
  10. [4] [link] [5] [link]
    — rottc0dd, 2025
  11. In the same vein, Karpathy's video series "Neural Networks from zero to hero"[0] touches upon a lot of this and intuitions as well. One of the best introductory series (even if you ignore the neural net part of it) and brushes on gradients, differentiation and what it means intuitively. [0] [link]
    — throwaway7783, 2025
  12. People think that deepseek/mistral/meta etc are democratizing AI, but its actually Karpathy who teaches us :) so we can understand them and make our own. [1] [link]
    — jackdoe, 2025

← HN 片单