Product LLM acceleration read one paper Flash-attention: compute attention by blocks YouTube Upload five videos this week and start to try codeforces problems. Codeforces problems always c...

Backgroud There are two common kinds of bound which limited the speed of training in deep learning. Memeory-bound: time spent on memeory-access is bottlenecked Computation-bound: time spent o...

Product LLM acceleration Matrix Multiplcation: Read more LoRA: start reading paper YouTube Upload four videos this week and receive 5 Subscribers Blog Update current blog to jekyll-theme-chi...

Introduction Inspiration: the change in weights during model adaptation have a low “intrinsic rank” Description: Change small matrices A and B when fine-tune, adding A * B to weight W, which sign...

Background After read “Manual Autograd” in unsloth’s blog, I try to parse model and found more related point where we can optimize. torchview is a great similar tool to use. torchview what torc...

Product Voice correction: failed, technology is not enough mature to meet this demand. PopTranslate: First available product. Cost of product with server is higher than expectation. unsloth:...

Life I stay home for 14 days this time, it’s long. I spent the first seven days with my families and last seven days with friends. Everyone had different life, what are the same is that everyone...

Product process PopTranslate This is a multi stages story. 1) Lost my free $300 in Google Account In order to join the YouTube Premium family account, I need to change my location from Canada t...

Product process PopTranslate Summary of promotion list of ways to publish promotion and is there response? ruanyifeng: Yes Product hunt: Yes Reddit: Yes HackerNews: No Twitter: No ...

Theory of Fast Cross Entropy Loss As a start of LLM acceleration project, I learnt Fast Cross Entropy Loss in unsloth, it shows a better realize solution of cross entropy loss than default pytorch...