- formatting
- images
- links
- math
- code
- blockquotes
- external-services
•
•
•
•
•
•
-
Kimi Attention Residual: Rethinking Layer Connections in Transformers
A breakthrough in residual architecture that applies attention mechanisms across layers, not just tokens
-
Action, Planning, and Memory: The Foundation of LLM Agents
Exploring three foundational frameworks for LLM agents - ReAct (action and reasoning), Tree of Thoughts (planning and search), and Reflexion (memory and learning) - that transform passive language models into capable autonomous agents.
-
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Notes on Mamba paper by Albert Gu and Tri Dao - exploring selective state space models as an alternative to transformers with fixed-size internal states and efficient inference.
-
mHC: Manifold-Constrained Hyper-Connections
Notes on DeepSeek's mHC paper - extending residual connections with Hyper-Connections constrained on double stochastic matrix manifolds for stability. Covers implementation with Sinkhorn-Knopp algorithm and performance improvements.
-
ReAct: Synergizing Reasoning and Acting in Language Models
Notes on ReAct - a paradigm for LLM agents that combines reasoning traces and task-specific actions. Covers prompt engineering, fine-tuning approaches, and implementation of zero-shot ReAct agents with LangChain.