All news
AI

Nvidia researchers argue huge AI context windows are a dead end

The Decoder

A model's context window is the text it can hold in view while answering, and it has grown from 8,192 tokens in GPT-4 to about a million today. Nvidia researchers say keeping everything in view is often slow and costly, and that in many tests periodic summaries plus targeted lookups matched full-context results. They propose models that work from short notes instead.

Summary written with AI by HowTechMoves, based on reporting by The Decoder.

All tech news
Loading related stories