Master10
Artificial Intelligence Module

Large Language Models & Generative AI

Generative AI and Large Language Models (LLMs) produce text, imagery, audio, and synthetic code by modeling high-dimensional data distributions. The modern paradigm shifted with the 2017 Google Research paper 'Attention Is All You Need' by Vaswani et al., which introduced the self-attention-based Transformer architecture, eliminating the recurrence bottleneck of RNNs. Autoregressive decoder-only models (like OpenAI's GPT series) predict subsequent tokens, while masked encoder models (like Google's BERT) capture bidirectional contextual semantics. Generative image models leverage Generative Adversarial Networks (GANs, introduced by Ian Goodfellow in 2014) and latent diffusion models. Techniques like Reinforcement Learning from Human Feedback (RLHF) align model outputs with human intent.

Key Concepts & Examination Highlights

  • The Transformer architecture was introduced in the landmark 2017 paper 'Attention Is All You Need', replacing recurrence with multi-head self-attention.
  • Generative Adversarial Networks (GANs), introduced by Ian Goodfellow in 2014, utilize a generator and discriminator network locked in a minimax game.
  • Reinforcement Learning from Human Feedback (RLHF) fine-tunes foundation language models to align with safety and instruction-following criteria.
  • The self-attention mechanism enables a Transformer model to calculate dynamic attention weights between every pair of tokens in a sequence simultaneously, eliminating the sequential bottleneck of RNNs.
  • BERT (Bidirectional Encoder Representations from Transformers), introduced by Google in 2018, utilizes an encoder-only architecture pre-trained on masked language modeling (MLM) and next sentence prediction (NSP).
  • The GPT (Generative Pre-trained Transformer) series, developed by OpenAI, utilizes a decoder-only autoregressive architecture to generate coherent sequential text token by token.
  • Diffusion models generate high-fidelity synthetic images and audio by systematically adding Gaussian noise to training data (forward process) and learning to reverse the denoising steps.
  • Hallucination in large language models refers to instances where the model generates plausible-sounding but factually incorrect, ungrounded, or fabricated outputs.
  • Retrieval-Augmented Generation (RAG) combines pre-trained language models with external vector search databases to retrieve authoritative, real-time facts before generating answers.
  • Parameter-Efficient Fine-Tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA), freeze base model weights and inject trainable low-rank decomposition matrices to fine-tune LLMs efficiently.
  • Tokenization is the preprocessing step that decomposes natural language text into subword units (tokens) using algorithms like Byte-Pair Encoding (BPE) or WordPiece.
  • Foundation models are large AI models trained on vast quantities of unlabeled multimodal data that can be adapted and fine-tuned to a broad spectrum of downstream tasks.
  • Multi-Head Attention splits the key, query, and value vector representations into multiple subspaces, allowing the model to simultaneously attend to information from different representation positions.
  • Positional encoding vectors are added to input token embeddings in Transformer models to supply essential word order and spatial positional information.
  • Scaling laws in LLMs, empirically established by Kaplan et al. and Chinchilla scaling (Hoffmann et al.), demonstrate that model performance improves predictably with increases in compute budget, dataset size, and parameter count.
  • Prompt engineering involves designing and refining textual inputs (prompts) using techniques like few-shot prompting, chain-of-thought (CoT) reasoning, and role prompting to guide model output generation.
  • Temperature is a sampling hyperparameter in LLMs that controls output randomness, where lower values (e.g., 0.2) produce deterministic factual outputs and higher values (e.g., 0.8) produce more diverse text.
  • Top-pp (nucleus) sampling chooses tokens from the smallest subset of candidates whose cumulative probability exceeds threshold pp, while top-kk sampling restricts generation to the kk most probable tokens.
  • Alignment in generative AI ensures that model outputs conform to human values of helpfulness, honesty, and harmlessness through methods like Direct Preference Optimization (DPO) and RLHF.
  • Vector embeddings represent words, sentences, or documents as high-dimensional mathematical vectors, enabling semantic similarity search using cosine similarity or Euclidean distance.
  • Vector databases such as Pinecone, Milvus, Chroma, and Qdrant store high-dimensional embeddings and execute approximate nearest neighbor (ANN) searches for real-time RAG pipelines.
  • Multimodal Large Language Models (MLLMs), such as GPT-4o, Gemini, and Claude 3.5 Sonnet, natively process and generate multiple modalities including text, images, video, and audio.
  • Context window length refers to the maximum number of tokens an LLM can process in a single inference call, ranging from 8K tokens up to 1 million tokens in advanced models.
  • Synthetic data generation uses generative AI models to create artificial datasets for training new machine learning models where real-world training data is scarce or privacy-sensitive.
  • Constitutional AI, developed by Anthropic, uses an AI-driven feedback loop where models critique and revise their own outputs based on a written constitution of safety and ethical principles.
  • Scaled Dot-Product Attention computes attention weights using the formula Attention(Q,K,V)=softmax(QKTdk)V\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V, where dk\sqrt{d_k} prevents vanishing gradients.
  • Multi-Query Attention (MQA) and Grouped-Query Attention (GQA) share key and value heads across multiple query heads to drastically reduce KV-cache memory bandwidth during inference.
  • FlashAttention is an exact attention algorithm that tiles computation across GPU SRAM memory hierarchies to achieve faster transformer training with sub-quadratic memory footprint.
  • Positional embeddings have evolved from absolute sinusoidal encodings to learnable embeddings, Rotary Position Embeddings (RoPE), and Attention with Linear Biases (ALiBi).
  • RoPE (Rotary Position Embedding) encodes relative token positions by multiplying query and key vectors by rotation matrices in complex coordinate planes.
  • Byte-Pair Encoding (BPE) iteratively merges the most frequent pairs of consecutive characters or byte tokens in a corpus into single subword units.
  • WordPiece (used in BERT) and SentencePiece (used in LLaMA and T5) are tokenization algorithms that segment raw text into subwords without requiring language-specific whitespace rules.
  • The masked language modeling (MLM) objective in BERT randomly masks 15% of input tokens and trains the bidirectional encoder to predict the original hidden words from context.
  • Autoregressive language models predict the conditional probability distribution of the next token given all preceding tokens: P(w1,w2,โ€ฆ,wn)=โˆi=1nP(wiโˆฃw1,โ€ฆ,wiโˆ’1)P(w_1, w_2, \dots, w_n) = \prod_{i=1}^n P(w_i | w_1, \dots, w_{i-1}).
  • Chain-of-Thought (CoT) prompting encourages language models to generate intermediate reasoning steps before providing final answers, significantly improving mathematical and logical performance.
  • Tree-of-Thoughts (ToT) expands prompting frameworks to explore multiple reasoning branches, look ahead, and backtrack to solve complex planning tasks.
  • Quantization reduces the precision of model weights and activations from 16-bit floating-point (FP16/BF16) to 8-bit (INT8) or 4-bit (INT4), reducing VRAM memory requirements.
  • QLoRA (Quantized Low-Rank Adaptation) backpropagates gradients through 4-bit quantized base model weights into 16-bit LoRA adapter matrices, enabling fine-tuning of large models on consumer GPUs.
  • Direct Preference Optimization (DPO) optimizes language models directly on human preference pairs using a closed-form loss function without training an explicit reinforcement learning reward model.
  • KTO (Kahneman-Tversky Optimization) aligns language models using individual positive or negative signal feedback on generated outputs based on behavioral prospect theory.
  • Mixture of Experts (MoE) architectures, such as Mixtral 8x7B, activate only a sparse subset of specialized feed-forward expert subnetworks per token, maximizing parameter scale while keeping compute low.
  • Dense retrieval models in RAG pipelines use bi-encoder networks (such as sentence-transformers) to map queries and documents into shared embedding spaces for dot-product similarity search.
  • Re-ranking models in RAG use cross-encoders to jointly compute deep bidirectional attention between a user query and candidate passages, providing fine-grained relevance scoring.
  • Chunking strategies in RAG divide long source documents into semantically coherent text passages with sliding window overlaps to preserve context boundaries.
  • Diffusion probabilistic models (DDPM and DDIM) reverse a discrete-time Markov chain that adds Gaussian noise, guided by score matching or classifier-free guidance.
  • Latent Diffusion Models (LDMs), such as Stable Diffusion, execute the forward and reverse diffusion processes in a compressed lower-dimensional latent space to minimize compute costs.
  • Classifier-Free Guidance (CFG) is a sampling technique in diffusion and generative models that balances sample diversity and prompt fidelity by interpolating between conditional and unconditional generations.
  • Context caching in large language models stores precomputed KV-cache states of long system instructions and static documents to lower latency and API billing costs on repeated queries.
  • Speculative decoding accelerates LLM inference by using a small, fast draft model to generate candidate tokens in parallel, which are verified simultaneously in a single forward pass by the large target model.
  • Guardrail frameworks (such as NeMo Guardrails and Llama Guard) act as programmable programmable safety filters monitoring user inputs and LLM outputs against toxic, unsafe, or off-topic prompts.
Curriculum & Reference Sources: NeurIPS 2017 ('Attention Is All You Need'), OpenAI Research, Google DeepMind, Stanford Center for Research on Foundation Models

Sample Solved Questions & Concept Explanations

8 Verified Concept Questions
Q1.EASY

What does 'LLM' stand for in modern generative artificial intelligence?

Q2.EASY

Which neural network architecture introduced in the 2017 Google paper 'Attention Is All You Need' revolutionized modern natural language processing?

Q3.EASY

What is the common term for an unintended, factually incorrect, or fabricated response generated by a Large Language Model?

Q4.EASY

What does 'GAN' stand for in deep learning generative models?

Q5.MEDIUM

The breakthrough DeepMind AI system 'AlphaFold 2' (2020) solved which 50-year-old grand challenge in molecular biology?

Q6.MEDIUM

What does the technique 'RLHF' stand for, used to align models like GPT-4 and Claude with human intent and safety?

Q7.MEDIUM

In transformer attention mechanisms, what are the three fundamental vectors calculated for every input token?

Q8.MEDIUM

What is 'RAG' in modern generative AI and enterprise search pipelines?