Growth in 2023
Overview
Generative AI gained global attention in 2023 with the launch of several consumer products. ChatGPT made history by reaching 100 million monthly users in just two months, a feat that took platforms like TikTok and Instagram much longer to achieve.
Key Factors
The development of generative AI has been driven by different factors:
- Availability of training data
- Computing power to run models
- Competitive interests among public technology companies
- Breakthrough model design research
Regulation is constantly evolving as generative AI evolves, and this does not directly contribute to the development of generative AI. In some cases, regulation may slow it down!
Powering Large Models
By 2023, models required 100 million times more computing power than those from a decade earlier. Innovations in parallelization allowed for the training of larger models using specialized processors like GPUs and TPUs.
Cloud computing enabled researchers to scale resources as needed, and improved software frameworks optimized computing power usage.
Data-Driven Improvements
The explosion of data availability in recent years has provided generative AI models with more training data. Techniques for creating synthetic data have further scaled the availability of training data.
Competitive Push
Big Tech companies and governments have driven generative AI development to gain commercial or political advantages. The core of this evolution is innovation in models.
GANs and High-Quality Output
As previously mentioned, GANs and High-Quality Output consist of generator and discriminator models that compete with each other, leading to significant improvements in the quality of generated content.
Transformers
Transformers are designed to understand and process text by considering the relationships between words. They excel at grasping context and generating coherent responses.
For example, they can differentiate between meanings in sentences like:
"The animal didn't cross the street because it was too tired"
"The animal didn't cross the street because it was too wide."
User Feedback (RLHF)
Reinforcement Learning with Human Feedback (RLHF) improves models by incorporating user feedback. Users rate model responses, and this feedback is used in the retraining process to better align outputs with user preferences.
For instance, Midjourney uses user ratings to guide its image generation models toward higher-rated responses.
