GPT-3 Parameters vs TPU Power

noooo you can't just add more parameters and data to a model! gpt-3 is no different from eliza, since it's just glorified pattern matching and curve fitting, not true intelligence which requires a symbolic representation of the input which connectionist models will never be able to do! also, the data needed is almost an entire percent of the total possible data we could collect from common crawl alone, and the hardware needed to train gpt-3 is so unfathomably massive that it requires in total almost as much money as the entire hourly US military budget. haha tpu go brrr

A crying, bespectacled Wojak complains that GPT-3 is merely glorified pattern matching and cannot achieve true intelligence, while a smug Wojak points to a massive room of computing hardware and says, “haha tpu go brrr.” The joke mocks the contrast between elaborate theoretical objections and the brute-force scaling of data and compute that made large language models successful. EN: A crying, bespectacled Wojak complains that GPT-3 is merely glorified pattern matching and cannot achieve true intelligence, while a smug Wojak points to a massive room of computing hardware and says, “haha tpu go brrr.” The joke mocks the contrast between elaborate theoretical objections and the brute-force scaling of data and compute that made large language models successful.

• @profunctor_io / 5023 2020-07-24