---
title: "AI Engineering"
url: https://manasdasgupta.com/ai-engineering/
description: "AI engineering is the practice of building applications on large language models: choosing the right model, grounding it in your data with RAG and vector…"
author: Manas Dasgupta
last_verified: 2026-11-12
---

# AI Engineering

> **Bottom line:** AI engineering is the practice of building applications on large language models: choosing the right model, grounding it in your data with RAG and vector databases, building agents and multi-agent systems with frameworks such as LangChain and LangGraph, and evaluating and securing them in production. This hub collects the courses, videos and answers on that work.

## Key questions
- How does RAG work, and when should I use it?
- How do I choose the right LLM for a task?
- How do I build AI agents with LangChain and LangGraph?

## Answers
### What is AI product development?
AI product development is the practice of building product features whose behaviour depends on a machine-learning model, usually a language model. It differs from ordinary product work in three ways: output is probabilistic, quality must be measured with evaluations, and launch plans need guardrails for wrong answers.

### How do you evaluate an LLM feature before launch?
Build an evaluation set of at least 50 real inputs with expected outputs, grade responses with a mix of exact checks, model-based graders and human review, and set a pass threshold before launch. Run the same set on every prompt or model change so regressions are caught before users see them.

## Videos
### Should this feature use AI? A one-page test

Most features that teams want to build with AI do not need it. This video walks through a one-page test: is the task tolerant of occasional errors, is there data to evaluate against, and is the cost of a wrong answer acceptable? If any answer is no, use rules or search instead.

- Watch: (video link pending)
- Length: 11:20
- Published: 2026-09-01

Key points:
- Error tolerance comes first: if a wrong answer is expensive, design for human review or skip AI.
- No evaluation data means no way to know whether the feature works. Build the test set first.
- Compare against the simplest non-AI baseline, such as search, rules or templates.
- Estimate running cost per user action before committing to a design.

Chapters:
- 00:00 Why most AI features fail
- 02:10 Question 1: error tolerance
- 05:30 Question 2: evaluation data
- 08:45 Question 3: cost of a wrong answer

### Write the evaluation before the feature

An evaluation set is a list of real inputs with the answers you expect. Building it before the feature forces the team to agree on what good looks like, catches regressions when prompts or models change, and turns quality debates into numbers. Start with 50 examples; grow it from production failures.

- Watch: (video link pending)
- Length: 14:05
- Published: 2026-09-10

Key points:
- Start with 50 representative inputs, including awkward edge cases.
- Grade with a mix of exact checks, rubric-based model graders and periodic human review.
- Run the set on every prompt or model change as a regression gate.
- Add every production failure back into the set.

Chapters:
- 00:00 What an evaluation set is
- 03:00 Choosing the first 50 examples
- 07:15 Graders
- 11:40 Regression gates

## Articles
- [Build, buy or wait: an AI decision framework for technology leaders](https://manasdasgupta.com/writing/build-buy-or-wait/): Build an AI capability only when it is core to your product and you can evaluate it. Buy when it is commodity and a vendor can show evaluation results on your data. Wait when neither is true: model capability and prices change fast, and a six-month delay is often the cheapest option.
- [Evaluation-first: why AI products should be tested before they are built](https://manasdasgupta.com/writing/evaluation-first-ai-products/): Write the evaluation set before you build an AI feature. It forces agreement on what good looks like, turns quality debates into numbers, and catches regressions whenever a prompt or model changes. Start with 50 real examples and add every production failure back in.

## Courses
- [Gen AI - LLM RAG Two in One - LangChain + LlamaIndex](https://manasdasgupta.com/courses/llm-rag-langchain-llamaindex/) (Udemy)
- [Introduction to Retrieval Augmented Generation (RAG)](https://manasdasgupta.com/courses/introduction-to-retrieval-augmented-generation-rag/) (Coursera)
- [Agentic AI for Cybersecurity](https://manasdasgupta.com/courses/agentic-ai-for-cybersecurity/) (EC-Council)
- [Selecting the Right LLM with Hugging Face](https://manasdasgupta.com/courses/selecting-the-right-llm-with-hugging-face/) (Coursera)
- [Troubleshooting Issues with Generative AI](https://manasdasgupta.com/courses/troubleshooting-issues-with-generative-ai/) (EC-Council)