Why Modern LLMs Are Decoder-only: Architectural Evolution and Considerations from Seq2Seq to GPT
「If we’re on the path to the Turing machine, Seq2Seq itself makes more sense than a Decoder." This is a sentence I wrote in my notes in 2022. Back then, I was experimenting with an early version of GPT-2, and I kept wondering: What exactly is this thing? In comparison, models like BART or T5, based on the Seq2Seq concept, seemed much more reasonable. Unexpectedly, a few years later, in this AI arms race, it’s the Decoder-only architecture that has claimed the MVP title. ...