Post

#Article #NeuralNetwork #Optimizer
Issue Date: 2025-10-28 [Thread Memo] 最近の最適化に関する研究についての見解, Seunghyun Seo, 2025.10 Comment

関連:
- [Paper Note] Weight Decay may matter more than muP for Learning Rate Transfer in Practice, Atli Kosson+, arXiv'25, 2025.10
- [Paper Note] Robust Layerwise Scaling Rules by Proper Weight Decay Tuning, Zhiyuan Fan+, arXiv'25, 2025.10
- [Paper Note] WHEN DOES SECOND-ORDER OPTIMIZATION SPEED UP TRAINING?, Ishikawa+, ICLR'24 Tiny Paper
- [Paper Note] Fantastic Pretraining Optimizers and Where to Find Them, Kaiyue Wen+, arXiv'25



#Article #NLP #LanguageModel #Prompting
Issue Date: 2024-09-08 A few prompt engineering tips that Ilya Sutskever picked up at OpenAI, Ilya Sutskever, 2024.09