Manas Dasgupta
Topic hub

AI Engineering

What this hub covers

AI engineering is the practice of building applications on large language models: choosing the right model, grounding it in your data with RAG and vector databases, building agents and multi-agent systems with frameworks such as LangChain and LangGraph, and evaluating and securing them in production. This hub collects the courses, videos and answers on that work.

For
Software engineers, data professionals and ML engineers moving into AI engineering
Time
Answers 1 min · videos 10–15 min
Next step
Start with the answers
Key questions

The questions this hub answers

What is AI product development?

AI product development is the practice of building product features whose behaviour depends on a machine-learning model, usually a language model. It differs from ordinary product work in three ways: output is probabilistic, quality must be measured with evaluations, and launch plans need guardrails for wrong answers.

Full answer

How do you evaluate an LLM feature before launch?

Build an evaluation set of at least 50 real inputs with expected outputs, grade responses with a mix of exact checks, model-based graders and human review, and set a pass threshold before launch. Run the same set on every prompt or model change so regressions are caught before users see them.

Full answer
Videos

Videos

Summary always visible. Expand for key points and chapters. The player loads only when you press play.

11:20
Video · 1 Sep 2026Placeholder

Should this feature use AI? A one-page test

Most features that teams want to build with AI do not need it. This video walks through a one-page test: is the task tolerant of occasional errors, is there data to evaluate against, and is the cost of a wrong answer acceptable? If any answer is no, use rules or search instead.

Key points & chapters

Key points

  • Error tolerance comes first: if a wrong answer is expensive, design for human review or skip AI.
  • No evaluation data means no way to know whether the feature works. Build the test set first.
  • Compare against the simplest non-AI baseline, such as search, rules or templates.
  • Estimate running cost per user action before committing to a design.

Chapters

  1. 00:00 Why most AI features fail
  2. 02:10 Question 1: error tolerance
  3. 05:30 Question 2: evaluation data
  4. 08:45 Question 3: cost of a wrong answer

Course: Introduction to RAG

14:05
Video · 10 Sep 2026Placeholder

Write the evaluation before the feature

An evaluation set is a list of real inputs with the answers you expect. Building it before the feature forces the team to agree on what good looks like, catches regressions when prompts or models change, and turns quality debates into numbers. Start with 50 examples; grow it from production failures.

Key points & chapters

Key points

  • Start with 50 representative inputs, including awkward edge cases.
  • Grade with a mix of exact checks, rubric-based model graders and periodic human review.
  • Run the set on every prompt or model change as a regression gate.
  • Add every production failure back into the set.

Chapters

  1. 00:00 What an evaluation set is
  2. 03:00 Choosing the first 50 examples
  3. 07:15 Graders
  4. 11:40 Regression gates
Writing

Articles

Courses

Courses

Course
Open full page ↗