Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman
Lex Clips · 3 位 HN 用户推荐 · 2023–2023
I found Andrej Karpathy's explanation of transformers to be very insightful: [link]
— chaoz_, 2023Check this explanation from Karpathy which provides really good intuition [link]
— jgotti92, 2023For example, Andrej Karpathy pointed out that the paper had another significant improvement hidden in it: during training the gradients can take a "shortcut" so that the bottom layers are trained faster than in typical deep learning architectures. This enables very large and deep models to be trained in a reasonable time. Without this trick, the huge LLMs seen these days would not have been possib
— jiggawatts, 2023