GPT-3 Solves Letterstring Analogies

When I was a graduate student in Douglas Hofstadter's AI lab, we used letterstring analogy puzzles (if abc goes to abd, what does ijjk go to?) as a testbed for intelligence. My fellow student Melanie Mitchell devised a program, Copycat, that was quite good at solving these puzzles. Copycat took years to write. Now Mitchell has tested GPT-3 on the same puzzles, and has found that it does a reasonable job on them (e.g. giving the answer ijjj). It is not perfect by any means and not as good as Copycat, but its results are still remarkable in a program with no fine-tuning for this domain.

A text screenshot describes how GPT-3 was tested on letterstring analogy puzzles previously used to evaluate AI intelligence. It compares GPT-3's reasonable results with Copycat, a specialized program created by Melanie Mitchell, and emphasizes that GPT-3 achieved this without fine-tuning for the domain. The meme is shared as a reaction to unexpectedly impressive artificial intelligence capabilities. EN: A text screenshot describes how GPT-3 was tested on letterstring analogy puzzles previously used to evaluate AI intelligence. It compares GPT-3's reasonable results with Copycat, a specialized program created by Melanie Mitchell, and emphasizes that GPT-3 achieved this without fine-tuning for the domain. The meme is shared as a reaction to unexpectedly impressive artificial intelligence capabilities.

• @ithueti / 2612 2020-07-31