Summary Methods There are two main methods to acclerate LLM and another tricky methods low-rank: reduce dimension of matrix block: compute matrix with block trick: update model structure o...

Collect, summary and adjust to get the following tutorial from multi-sources in Reference. How to learn Very quickly identify what the foundational knowledge is Build a personal curriculum to...

LLM Inference It’s a great milestone when I finish a mature and iterable project, even it’s still in the first stage. There’re plenty AI infra repositoreis and projects in github, but most of the...

LLM Inference It’s a great milestone when I finish a mature and iterable project, even it’s still in the first stage. There’re plenty AI infra repositoreis and projects in github, but most of the...

This is the first week to work as a LLM inference engineer, the real work content is what I want in the past. I don’t have much experience in this field, But it’s still acceptable. To do what I l...

This is the second time period that I need to seek a job position in my life till now. I don’t have much experience to take a interview in the past, I think I learn a lot this time. First lesson I...

Happy New Year! Feel more happiness this year, really enjoy.

Product AutoSwitch Translate Final make this extension, which can automatically switch target language when source language is same with target language. What surprised me was I only spen half day...

What I do keywods: Jan-May: work Travel: wuhan, huangshan IELTS Products: Chinese blog collection slack to discord IELTS speaking assistant PopTranslate ...

Product Triton Try to debug the Liger-Kernel but found it works well except bfloat type. Learning news about LLM translation and try to realise the whole one in this kind of project, like cut cros...