When LLMs started making inroads into the technology space over the last few years, training relied heavily on publicly accessible internet data. This data, in various forms including audio, video, images, and text, is full of subtleties and nuances, resulting in rich training for LLMs. However, the original creators of this data are becoming aware of their rights and are starting to restrict access or set commercial terms. As a result, there is a growing inclination to train the next generation of LLMs on synthetic data generated by existing models.
When it comes to many industries, especially those with specialized or highly regulated applications like manufacturing automation, generating sufficient real-world training data can be challenging, time-consuming, and expensive. Synthetic data offers a viable solution by simulating various scenarios and conditions that may be difficult or impossible to replicate in the real world. Synthetic data can be used to create a wide range of scenarios, including edge cases and anomalies. For rare events (e.g., equipment failures), synthetic data augments the limited real-world examples.
Researchers warn (Reference 1) that increasing use of synthetic data could lead to a phenomenon called ‘Model Collapse’. The paper argues that the indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. The paper defines ‘Model collapse’ as a degenerative process affecting generations of learned generative models, in which the data they generate end up polluting the training set of the next generation. Being trained on polluted data, they then mis-perceive reality.
Slowly but steadily the information on the web will start getting flooded by the one generated by the current or earlier versions of LLMs. Which will in turn will go as input to the training. The recursive nature of this process can result in models drifting further away from the real-world distribution, compromising their accuracy in representing the world. We must remember that the outputs from LLM are not always perfect. With such recursive degradation the content starts losing the diversity, subtleties and nuances that is characteristic of human generated data. As a result, the subsequent generations of LLMs start producing content that is increasingly homogenous, lacking the richness and variety of human experiences and less connected to the real world.
This problem has a potential to become more acute in specialized applications like manufacturing automation. Consequence of model collapse can be severe and costly in such cases. Some examples of fallout are – The model’s ability to accurately predict or classify manufacturing data (e.g., sensor readings, product quality) may deteriorate. The model might start making more mistakes in tasks like defect detection, predictive maintenance, or process optimization. The model may become overly specialized to the synthetic data it was trained on, struggling to generalize to real-world manufacturing scenarios. The model may find it difficult to adapt to changes in manufacturing processes or conditions. If the model makes incorrect decisions in critical applications like quality control or safety monitoring, it could lead to product defects, equipment failures, or even accidents. Frequent errors and breakdowns caused by model collapse can increase maintenance and repair costs. As the model’s performance degrades, trust in its capabilities may erode, leading to reluctance to rely on it for critical tasks.
While model collapse is a significant concern in the field of language models, experts have proposed several strategies to mitigate its risks, as listed below. But as with many other things, devil will be in details. It will never be a trivial exercise.
- Diverse and High-Quality Training Data: Ensure that the model is trained on a diverse and representative dataset that accurately reflects the real world. Prioritize human generated content over synthetic data to help the models remain grounded in reality.
- Regular Evaluation and Monitoring: Continuously monitor the model’s performance using appropriate metrics to detect signs of degradation. Implement systems to identify potential issues before they escalate.
- Data Augmentation Techniques: Use synthetic data augmentation techniques judiciously, ensuring that it complements real-world data and doesn’t introduce biases.
- Model Architecture and Training Methods: Employ different techniques to prevent overfitting. Combine multiple models to improve robustness and reduce the risk of catastrophic failure.
- Human Oversight and Intervention: Incorporate human feedback loops to guide the model’s learning and correct errors. Implement safeguards to prevent the model from generating harmful or biased content.
- Transparency and Explainability: Develop techniques to understand the inner workings of the model and identify potential vulnerabilities.
While these recommendations are expected to help mitigate the risk of model collapse, can this problem be eliminated? Is this a real problem or hypothetical? Time will tell.
It is our view at AiThoughts.org that, as language models continue to evolve, new strategies and approaches will be employed by model vendors to address this ‘model collapse’ challenge.
References:
- AI models collapse when trained on recursively generated data by Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson & Yarin Gal at nature.com (https://www.nature.com/articles/s41586-024-07566-y#citeas )
