Grok Mountain’s Substack

Grok Mountain’s Substack

Home
Notes
Archive
About
Nvidia GPUs: Powering the Future of AI with the H100 and the upcoming B200
Exploring the Nvidia H100 and the soon to be released B200 GPU, their SXM form factors, and the specialized servers driving AI innovation.
 • 
Grok Mountain
1
Trump's Bargaining Chip: How AI GPUs Could Lead to Peace with Putin
Could AI GPUs be Trump’s key to peace with Putin? This theory explores how tech might stabilize Ukraine, end conflict, and shift global alliances.
 • 
Grok Mountain
Why Grok 10 Will Rely on Reinforcement Learning for 90% of Its Training
Following the trends of plateauing world knowledge and larger and larger AI data centers, it stands to reason that the future of AI model training will…
 • 
Grok Mountain

February 2025

The Singularity Unveiled: A Leap Beyond Humanity and the Chips That Could Take Us There
Exploring the possibility of super-intelligent AI breaking through the plateau of Moore's Law by designing the next generation of microchips.
 • 
Grok Mountain
Inside the Colossus: How NVIDIA and xAI Teamed Up to Build a Supercomputer for Grok 3
NVIDIA provides technology beyond GPUs. In building Colossus, xAI partnered with NVIDIA to utilize their high-end networking and software solutions to…
 • 
Grok Mountain
The AI Arms Race: U.S., China, and the Path to Grok 3
History of the AI arms race, where control of Nvidia's H100 chips plays a major role in giving the US an advantage over China.
 • 
Grok Mountain
Origin of Grok 3 - the Colossus Data Center in Memphis
How Musk leapfrogged the LLM competition by procuring 200,000 NVIDIA H100 GPUs and building a cutting edge data center in Memphis
 • 
Grok Mountain
Understanding the Context Window: A Train Station Analogy
How training data must be broken into sequences of tokens, adhering to the size of the context window. These sequences are then processed in batches…
 • 
Grok Mountain
An Android's Journey Through University: Understanding Neural Networks and the Transformer Block
Explaining the layers of neural networks and Transformers, and how hidden state vectors are created and enriched as they pass through these layers…
 • 
Grok Mountain
The Emergence of Intelligence in LLMs via Complex Interactions of Weights
How weights (parameters) in LLMs are individually meaningless, but collectively across many layers of the neural network, give rise to intelligent…
 • 
Grok Mountain
Introduction to Neurons in Transformers
Exploring the definition of "neurons" in neural networks, from the broad definition to the strict, as this term is sometimes used in various ways in the…
 • 
Grok Mountain
Transformers and Holography: How AI Models Capture the 'Whole in Every Part'
Explaining how the FFN is able to process each token independently due to it taking as input contextually aware vectors, thanks to the holographic…
 • 
Grok Mountain
© 2025 Grok Mountain
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture