I started worrying about the impact of the LLMs being trained on more and more AI output over time, it seems like eventually they will be generating text based on their own generated text. What a mess...
Hi Ann, yes, I think that's already starting to happen, and most companies know that's a bad idea. Eventually, when a language model is being trained on AI-generated texts, it will collapse and will be unable to generate anything useful. What I find wild is that Elon Musk announced he wants to retrain his language model on sources that are selected or generated to represent his worldview; in this way, he says he wants GenAI to be less left-biased. This post explains the craziness well: https://garymarcus.substack.com/p/1984-but-with-llms
These essays have been fascinating, thank you for the time and effort you've put in. They've helped me get a much bwtywr foundation under my unease (tending toward horror sometimes!) about AI chat bots. I plan to read them all again and digninto the references too.
I started worrying about the impact of the LLMs being trained on more and more AI output over time, it seems like eventually they will be generating text based on their own generated text. What a mess...
Hi Ann, yes, I think that's already starting to happen, and most companies know that's a bad idea. Eventually, when a language model is being trained on AI-generated texts, it will collapse and will be unable to generate anything useful. What I find wild is that Elon Musk announced he wants to retrain his language model on sources that are selected or generated to represent his worldview; in this way, he says he wants GenAI to be less left-biased. This post explains the craziness well: https://garymarcus.substack.com/p/1984-but-with-llms
These essays have been fascinating, thank you for the time and effort you've put in. They've helped me get a much bwtywr foundation under my unease (tending toward horror sometimes!) about AI chat bots. I plan to read them all again and digninto the references too.
Thank you so much! Writing these essays has been quite enjoyable because the writing helped me understand how AI chatbots work even better!