Clik here to view.
What are KV caches really?
KV caching lies sufficiently deep within the guts of how transformers work that really understanding how it can be used goes beyond any simple analogy. This is why the Internet is full of explanations...
View ArticleClik here to view.
ISC'26 recap
Last month was the 2026 ISC High Performance Conference in the beautiful (and sweltering) Hamburg, Germany. It was my fifth time attending in-person, and it has fast become my favorite HPC community...
View ArticleClik here to view.
AI doesn't need giant supercomputers after all
I attended the 2026 Salishan Conference on High Speed Computing last month, and it was a week well spent in coastal Oregon hearing what many of the world's experts in scientific computing are worried...
View ArticleClik here to view.
GTC 2026 recap
I recently attended GTC26, NVIDIA's flagship annual conference, and it was a blazing hot week full of equal parts glitz and technology. And when I say blazing hot, I mean that literally--San Jose was...
View ArticleClik here to view.
HPC in an AI world: swimming upstream with more conviction
Dan Reed recently published an essay, HPC In An AI World, that summarizes a longer-form statement piece he co-authored with Jack Dongarra and Dennis Gannon called Ride the Wave, Build the Future:...
View ArticleClik here to view.
SC'25 recap
The annual SC conference was held last week, drawing over 16,000 registrants and 560 exhibitors to St. Louis, Missouri to talk about high-performance computing, artificial intelligence,...
View ArticleClik here to view.
Lessons learned from three years in cloud supercomputing
I recently decided to leave Microsoft after having spent just over three years there, first as a storage product manager, then as a compute engineer. Although I touched many parts of Azure's...
View ArticleClik here to view.
ISC'25 recap
I had the pleasure of attending the 40th annual ISC High Performance conference this month in Hamburg, Germany. It was a delightful way to take the pulse of the high-performance computing community and...
View ArticleClik here to view.
GTC 2025 recap
I attended GTC for the first time this year and got to experience what turned out to be one of the strangest, over-the-top conferences I've ever attended. It took many typical conference activities...
View ArticleClik here to view.
LLM training without a parallel file system
The illustrious Jeff Denworth recently posted a hot take across social media, claiming that training large language models (LLMs) doesn't require massive, expensive parallel file systems:As someone...
View ArticleClik here to view.
SC'24 recap
The premiere annual conference of the high-performance computing community, SC24, was held in Atlanta last week, and it attracted a record-shattering number of attendees--nearly 18,000 registrants, up...
View ArticleClik here to view.
FASST will be DOE's opportunity to adapt, align, or...
These are some personal thoughts I’ve had in response to Notice of Request for Information (RFI) on Frontiers in AI for Science, Security, and Technology (FASST) Initiative.The premise of the RFI...
View ArticleClik here to view.
A critique of the call for public AI
As I spend more time in the AI infrastructure business, I've been thinking more and more about the government's role in AI. There's no shortage of opinions and position papers on the topic, and sadly,...
View ArticleClik here to view.
How has life after leaving the Labs been going?
June 2024 marked two years since I left my job at one of the world's most prestigious government HPC centers for a job in one of the world's largest technology corporations. In that time, the world of...
View ArticleClik here to view.
ISC’24 recap
I had the great pleasure of attending the ISC High Performance conference this month, marking the fifth time I've attended what has become one of my top must-attend industry conferences of the year....
View ArticleOn the road to Hamburg for ISC'24
I have the great fortune of being able to attend to the 2024 ISC High Performance conference in Hamburg next week where I will be both speaking and listening throughout the contributed program and the...
View ArticleClik here to view.
A closer look at "training" a trillion-parameter model on Frontier
A paper titled "Optimizing Distributed Training on Frontier for Large Language Models" has been making its rounds over the last few weeks with sensational taglines saying the authors trained a...
View ArticleClik here to view.
SC'23 Recap
The largest high-performance computing industry conference of the year, SC23, was held in Denver last week. This year's conference attracted over 14,000 attendees and 438 exhibitors, finally breaking...
View ArticleClik here to view.
SC'22 Recap
The biggest annual conference in HPC, the SC conference, was recently held in Dallas, Texas in its second hybrid incarnation since being all-remote for the pandemic. This year attracted over 11,000...
View ArticleClik here to view.
Life and leaving NERSC
When word started to spread that I was leaving my job at NERSC for Microsoft, a lot of people either directly or indirectly attributed my decision to being one motivated by money. Rationalizing my...
View Article
/pic9814888.png)
/fit-in/580x348/filters:format(webp)/english-betterindia/media/media_files/2026/09/17/balaghat-3-2026-09-17-18-46-40.png)