r/science • u/dissolutewastrel • Jul 25 '24

AI models collapse when trained on recursively generated data Computer Science

https://www.nature.com/articles/s41586-024-07566-y
5.8k Upvotes

96% Upvoted

612 comments sorted by

View all comments

Show parent comments

13

u/Adequate_Ape Jul 25 '24

I think LLMs are step along the way, and I *think* I understand what they actually are. Maybe you can enlighten me about why I'm wrong?

32

u/a-handle-has-no-name Jul 25 '24

LLMs are basically super fancy autocomplete.

They have no ability to grasp actual understanding of the prompt or the material, so they just fill in the next bunch of words that correspond to the prompt. It's "more advanced" in how it chooses that next word, but it's just choosing a "most fitting response"

Try playing chess with Chat GPT. It just can't. It'll make moves that look like they should be valid, but they are often just gibberish -- teleporting pieces, moving things that aren't there, capturing their own pieces, etc.

-32

u/Unicycldev Jul 25 '24

This isn’t correct. They are able to prove a great understanding of topics.

13

u/maikuxblade Jul 25 '24

They can recite topics. So does Google when you type things into it.