Skip to main content

What is Generative AI?

Generative AI or GenAI is a branch of AI that focuses on creating new content, such as text, images, audio, video, and code etc from the existing data.

Traditional AI or ML systems analyze and interpret data. However, GenAI models learn patterns and relationships from vast amounts of training data and use that knowledge to generate entirely new content.

Machine learning has been around for decades then what has led to the raise of GenAI suddenly? Machine learning supplied the foundations; large-scale compute, Transformers, abundant data, improved training, and a natural-language interface made those foundations suddenly usable at scale.

Foundation Models​

Generative AI is powered by Foundation Models (FMs). These are large AI models trained on broad, diverse datasets so they learn general patterns that can be adapted to manay different tasks.

For example, a language foundation model may first learn from books, websites, documents, and code. After that general pretraining, it can be adapted to:

  • Answer questions
  • Summarize documents
  • Translate languages
  • Generate or explain code
  • Draft emails and reports
  • Classify or extract information

The same idea applies beyond text. A model trained on images, audio, video, or combinations of these can support tasks such as image generation, speech recognition, video understanding, or visual question answering.

A useful way to think about the process is:

Large-scale data → pretraining → foundation model → adaptation → specialized applications

“Foundation” refers to the model’s role as a reusable base layer. Instead of building and training a separate model from scratch for every task, developers choose one general-purpose model and work with it using methods such as fine-tuning, prompting, retrieval-augmented generation, or tool integration.

Foundation Model Lifecycle​

  • Data selection: FMs require training on massive datasets. So, select data from diverse sources.
  • Pre-training: Train the ML model on the selected data. During this, the foundation model's algorithm can learn the meaning, context, and relationships of the words in the datasets. The model can be further pre-trained on additional data. The goal is to expand the model's knowledge base and make it capable to understand and perform diverse tasks.
  • Optimization: Optimize the resulting FM using the methods like prompt engineering, retrieval-augmented generation (RAG) etc
  • Evalutaion: Evaluate the model's performance using appropriate metrics and benchmarks.
  • Deployment: Deploy the model on the target environment (on-prem server or cloud etc).
  • Monitor and maintain: Monitor the model's performance, collect the feedback from the users, experts. Enhance the model from the received feedback.

FM lifecycle is an iterative process. Feedback can be collected from each stage and applied at the same stage in the next iteration.

Next, we will see the most commonly used FMs now-a-days.