Jay's Blog

Here I document my learning/research notes, as well as some of my calligraphy stuff.

Can Repeating Transformer Layers Help?

Repeating a block adds computation without adding another copy of its weights.

14 min read · October 05, 2026

2026 · looped-transformer hybrid-attention qwen llm-inference · deep-learning
Recurrent State and Depth: Two ‘new’ LLM Architectures

Notes and hands-on experiments on recurrent-state hybrid attention and recurrent-depth looped Transformers.

12 min read · September 20, 2026

2026 · transformer linear-attention gated-deltanet hybrid-attention looped-transformer llm-inference · deep-learning
TIRx GEMM - Detailed Technical Notes

Detailed technical notes for the nine-stage TIRx GEMM optimization journey on NVIDIA Blackwell.

25 min read · August 15, 2026

2026 · cuda gpu tirx blackwell gemm performance · technical-notes
Python Syntax, PTX-Level Control - A GEMM at 98% of cuBLAS on Blackwell

A nine-stage TIRx GEMM optimization journey on NVIDIA B200, reaching 97.9% of cuBLAS throughput.

13 min read · August 15, 2026

2026 · cuda gpu tirx blackwell gemm performance · technical-notes
CUDA foundation

Notes on CUDA hardware, memory architecture, coalescing, shared memory, atomics, warps, streams, and error checking.

68 min read · June 19, 2026

2026 · cuda gpu parallel-computing systems · technical-notes