AI Engineering Track
— Islamabad 2026

Working directly with OpenAI, Anthropic, and Google Gemini — and the prompt engineering techniques that make the difference between a toy prototype and a reliable production feature.

1. OpenAI API: chat completions, function calling, structured outputs, vision, and streaming with the Python SDK
2. Anthropic API: messages API, system prompts, tool use, vision, and extended thinking with Claude
3. Google Gemini API: multimodal inputs, long context, grounding, and the Gemini Python SDK
4. Streaming responses: handling token-by-token output in APIs and surfacing it to users in real time
5. Structured output: forcing models to return valid JSON using OpenAI structured outputs, Anthropic tool use, and the Instructor library
6. Vision and multimodal inputs: sending images, PDFs, and documents to LLM APIs for analysis
7. Batch API: processing thousands of requests asynchronously at lower cost with OpenAI Batch and Anthropic Batch
8. Rate limiting and retry logic: exponential backoff, request queuing, and graceful degradation
9. Provider abstraction: building a unified LLM client that can swap providers without rewriting application logic

10. System prompts: writing effective prompts that define persona, behaviour, output format, and constraints
11. Few-shot prompting: selecting and formatting examples that steer model behaviour reliably
12. Chain-of-thought: making models reason step by step before producing output
13. XML and structured prompt formatting: Anthropic's recommended approach for complex prompts
14. Prompt templating: building dynamic prompts from user input and context using Jinja2 and f-strings
15. Output formatting control: requesting JSON, markdown, tables, and code blocks reliably
16. Prompt versioning: treating prompts as code — version control and A/B testing
17. Prompt injection: understanding attack vectors and how to defend against them
18. Context window management: summarisation, truncation, and prioritisation strategies
19. Instruction following: writing prompts that models actually follow — specificity, positive framing

Embeddings are the foundation of semantic search, RAG pipelines, recommendation systems, and clustering. This phase covers generating embeddings and storing/querying them at scale using production vector databases.

1. What embeddings are: converting text, images, and data into high-dimensional vectors that encode semantic meaning
2. Embedding models: OpenAI text-embedding-3 (small/large), Cohere Embed v3, and open-source alternatives (sentence-transformers, BGE, E5)
3. Embedding dimensions and model selection: accuracy vs cost vs latency trade-offs
4. Similarity metrics: cosine similarity, dot product, and Euclidean distance — when each applies
5. Batching embedding requests: efficient bulk generation for large document corpora
6. Multimodal embeddings: text, images, and code — CLIP and OpenAI vision embeddings
7. Embedding drift: how model updates can change embedding spaces and break existing indexes

8. pgvector: vector similarity search in PostgreSQL — HNSW vs IVFFlat indexing, Eloquent-friendly queries
9. Pinecone: managed vector database — indexes, namespaces, metadata filtering, and hybrid search
10. Qdrant: open-source vector database — collections, payload filtering, and self-hosted deployment
11. Choosing a vector database: decision framework based on scale, cost, latency, and infrastructure
12. Hybrid search: combining dense vector search with sparse BM25 keyword search for better retrieval
13. Metadata filtering: narrowing searches by document type, date, user, tenant, or structured fields
14. Vector index performance: HNSW graph construction, ef_construction, and recall/latency trade-offs
15. Re-ranking: using cross-encoders (Cohere Rerank, Voyage Rerank) to improve retrieval precision
16. Amazon OpenSearch with vector engine: AWS-native vector search alternative for AWS deployments

RAG is the most important pattern in production AI engineering — it solves the core limitations of LLMs (outdated training data, hallucination, private knowledge) by retrieving relevant context at inference time. This phase covers RAG from basic implementation through to advanced production patterns.

1. Why RAG: the problem it solves, when to use it, and when fine-tuning is a better answer
2. The basic RAG pipeline: ingest → chunk → embed → store → retrieve → augment → generate
3. Document ingestion: loading PDFs, Word docs, web pages, Notion, and databases with LangChain loaders and LlamaIndex readers
4. Text chunking strategies: fixed-size, recursive character splitting, semantic chunking, document-structure-aware
5. Chunk size and overlap: how they affect retrieval quality and what to tune for different document types
6. Metadata enrichment: adding source, page number, section headers, and timestamps to chunks
7. Embedding and indexing: bulk ingestion pipelines with progress tracking and error handling
8. Query embedding and similarity search: retrieving the top-k most relevant chunks
9. Context assembly: formatting retrieved chunks into a coherent prompt context block
10. Source attribution: citing which documents the answer was drawn from

11. Query transformation: rewriting user queries with an LLM before retrieval to improve recall
12. HyDE (Hypothetical Document Embeddings): generating a hypothetical answer and using it as the retrieval query
13. Multi-query retrieval: generating multiple query variants and merging their results
14. Parent-child chunking: indexing small child chunks for precision, retrieving larger parent context
15. Contextual compression: extracting only the relevant portion of a retrieved chunk
16. Corrective RAG (CRAG): evaluating retrieval quality and falling back to web search when the knowledge base is insufficient
17. Multi-vector retrieval: indexing documents by multiple representations (summary + full text + hypothetical questions)
18. Agentic RAG: building retrieval as a tool that an agent calls dynamically
19. RAG evaluation with RAGAS: measuring retrieval quality (context precision, recall) and generation quality (faithfulness, answer relevancy)

LangChain and LlamaIndex are the two dominant frameworks for orchestrating LLM applications — managing chains of calls, tool integrations, memory, and retrieval pipelines. Both are covered so you can choose the right tool for each job.

1. LangChain architecture: chains, runnables, and the LCEL (LangChain Expression Language) pipeline syntax
2. Prompt templates, output parsers, and structured output chains
3. LangChain retrieval chains: complete RAG pipelines with LCEL
4. Conversation chains and memory: maintaining history across turns with different memory backends
5. LangChain Tools: wrapping functions, APIs, and databases as tools LLMs can call
6. LangSmith: tracing, debugging, and evaluating LangChain applications in production

7. LlamaIndex architecture: nodes, indexes, query engines, and pipelines
8. Document and node processing: readers, transformations, and metadata extractors
9. Index types: VectorStoreIndex, SummaryIndex, KnowledgeGraphIndex, PropertyGraphIndex
10. Query engines and chat engines: conversational interfaces over your data
11. Sub-question query engine: decomposing complex questions across multiple data sources
12. LlamaIndex Workflows: event-driven, step-based orchestration for complex multi-stage pipelines
13. LlamaParse: managed document parsing for complex PDFs, tables, and mixed-format documents

14. LangChain vs LlamaIndex vs building from scratch: practical decision framework with real trade-offs
15. Using both together: LlamaIndex for retrieval, LangChain for orchestration
16. When to avoid frameworks: cases where direct API calls produce simpler, more maintainable code
17. Dependency pinning and version management: keeping orchestration framework upgrades from breaking production

Agents are LLMs that can take actions — calling tools, writing and executing code, querying databases, and orchestrating other AI models. This phase covers agent architectures from simple tool-calling to complex multi-agent systems.

1. Function calling fundamentals: defining tools as JSON schemas and letting LLMs decide when and how to call them
2. Parallel tool calls: models that call multiple tools simultaneously and merge results
3. Tool design principles: naming, descriptions, and parameter schemas that LLMs use reliably
4. Built-in tools: web search, code execution, and file reading across OpenAI, Anthropic, and Gemini
5. Custom tools: wrapping REST APIs, database queries, Python functions, and external services as LLM tools

6. ReAct (Reasoning + Acting): the foundational agent loop — think, act, observe, repeat
7. OpenAI Assistants API: threads, runs, tool calls, and file search — managed agent infrastructure
8. LangGraph: stateful, graph-based agent workflows with cycles, branches, and human-in-the-loop
9. LlamaIndex Workflows: event-driven agent pipelines with explicit step definitions
10. Memory in agents: short-term (conversation buffer), long-term (vector memory), and entity memory
11. Planning agents: breaking complex goals into sub-tasks and executing in order
12. Code execution agents: agents that write Python, run it in a sandbox, and iterate on output
13. Browser agents: agents that navigate web pages and extract information (Playwright + LLM)

14. Multi-agent patterns: supervisor agents that delegate to specialist sub-agents
15. Agent-to-agent communication: how agents pass context, results, and instructions
16. CrewAI: role-based multi-agent orchestration for structured collaborative workflows
17. AutoGen: Microsoft's multi-agent conversation framework for complex task decomposition
18. Guardrails in agent systems: preventing runaway loops, cost overruns, and unintended actions
19. Human-in-the-loop: checkpoints where agents pause and request human approval before proceeding

Fine-tuning is not always the right answer — but when it is, it dramatically outperforms prompting alone. This phase covers when fine-tuning makes sense, how to do it correctly, and the alternatives that are often faster and cheaper.

1. Fine-tuning vs RAG vs prompt engineering: the decision framework every AI engineer needs
2. When fine-tuning wins: style consistency, format adherence, domain-specific terminology, and latency-sensitive use cases
3. Dataset preparation: formatting training data as instruction-response pairs, quality filtering, and diversity
4. OpenAI fine-tuning API: uploading datasets, running training jobs, evaluating fine-tuned models, and cost estimation
5. Fine-tuning GPT-4o mini for classification, extraction, and structured output tasks
6. LoRA and QLoRA: parameter-efficient fine-tuning of open-source models (Llama 3, Mistral) on consumer hardware
7. HuggingFace PEFT library: implementing LoRA fine-tuning with the Trainer API
8. Instruction tuning vs continued pre-training: understanding the difference and when each applies
9. RLHF overview: how models are aligned with human preferences — conceptual understanding
10. Deploying fine-tuned models: serving with vLLM, BentoML, or uploading to HuggingFace Hub

Production AI systems fail in ways that are hard to predict and hard to detect. This phase covers evaluation frameworks, guardrails, and safety layers that make AI features trustworthy in customer-facing applications.

1. Why LLM evaluation is hard: non-determinism, subjective quality, and the absence of ground truth
2. Evaluation metrics: faithfulness, answer relevancy, context precision, context recall, and toxicity
3. RAGAS: automated RAG evaluation — measuring retrieval and generation quality end-to-end
4. LLM-as-judge: using a strong LLM to evaluate the outputs of another — prompting patterns and limitations
5. Human evaluation: building annotation interfaces and rubrics for systematic human review
6. Regression testing: building an evaluation dataset and running it on every prompt or model change
7. LangSmith and Braintrust: platforms for logging, evaluating, and comparing LLM outputs across runs
8. Evals as code: integrating LLM evaluation into CI/CD pipelines so regressions are caught before deployment

9. Input guardrails: classifying and filtering user inputs before they reach the LLM
10. Output guardrails: validating, filtering, and post-processing LLM outputs before they reach users
11. Guardrails AI: declarative guardrail definitions with validators for PII, toxicity, and schema conformance
12. Llama Guard: Meta's open-source safety classifier for screening inputs and outputs
13. PII detection and redaction: identifying and masking personal data in inputs and outputs with Presidio
14. Jailbreak and prompt injection defence: input sanitisation and instruction hierarchy patterns
15. Content moderation: OpenAI Moderation API and custom classifiers for domain-specific policies
16. Fallback strategies: graceful degradation when models fail, time out, or produce unsafe output

Shipping an AI feature to production is different from shipping a traditional API — latency is higher, costs vary with usage, outputs are non-deterministic, and failures are often silent. This phase covers running AI systems reliably at scale.

1. AI feature architecture: synchronous vs asynchronous patterns in full-stack applications
2. Streaming AI responses to the frontend: Server-Sent Events in FastAPI and Next.js
3. Background AI jobs: document processing, embeddings, and batch inference with Celery / ARQ
4. Caching LLM responses: semantic caching with GPTCache and Redis to reduce cost and latency
5. LLM proxy layer: routing requests across providers, fallbacks, and usage tracking with LiteLLM
6. Multi-tenancy: isolating AI features, vector namespaces, and usage quotas per user or organisation

7. FastAPI on AWS ECS: containerised AI inference services with auto-scaling
8. AWS Lambda for lightweight AI features: serverless LLM calls with cold start optimisation
9. AWS Bedrock: accessing Claude, Llama, Titan, and other foundation models through AWS
10. Amazon OpenSearch with vector engine: AWS-native vector search for RAG at scale
11. Secrets management: storing and rotating API keys with AWS Secrets Manager

12. LLM observability: what to log — prompts, completions, tokens, latency, cost, and user feedback
13. Langfuse: open-source LLM observability — tracing, scoring, and dataset management
14. OpenTelemetry for AI: tracing LLM calls as spans in distributed traces
15. Cost monitoring: per-user, per-feature, and per-model spend with dashboards and budget alerts
16. Latency monitoring: p50/p95/p99 tracking and alerting on degradation
17. Hallucination monitoring: automated detection of factual inconsistencies in production
18. User feedback loops: thumbs up/down signals and using them to improve prompts and retrieval