Nvidia BlueField-4 STX adds a context memory layer to storage to close the agentic AI throughput gap

Nvidia's BlueField-4 STX reference architecture inserts a dedicated context memory layer between GPUs and traditional storage ...
VentureBeat

Breaking through AI’s memory wall with token warehousing

Shimon Ben-David, CTO, WEKA and Matt Marshall, Founder & CEO, VentureBeat As agentic AI moves from experiments to real production workloads, a quiet but serious infrastructure problem is coming into ...

New memory architecture targets AI inference bottlenecks

Lightbits Labs Ltd. today is introducing a new architecture aimed at addressing one of the most stubborn bottlenecks in large ...
The Next Platform

AWS Taps Nvidia NVSwitch For Liquid Cooled, Rackscale GPU Nodes

Since the advent of distributed computing, there has been a tension between the tight coherency of memory and its compute within a node – the base level of a unit of compute – and the looser coherency ...
TweakTown

Meta's huge 16,384 NVIDIA H100 AI GPU cluster: HBM3 memory crashed half of Llama 3 training

Meta released a new study detailing its Llama 3 405B model training, which took 54 days with the 16,384 NVIDIA H100 AI GPU cluster. During that time, 419 unexpected component failures occurred, with ...
Wired

A Flaw in Millions of Apple, AMD, and Qualcomm GPUs Could Expose AI Data

As more companies ramp up development of artificial intelligence systems, they are increasingly turning to graphics processing unit (GPU) chips for the computing power they need to run large language ...
InfoQ

Pitfalls of Unified Memory Models in GPUs

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...