Evaluating and troubleshooting generative AI applications

AIP-C01 · Testing, Validation, and Troubleshooting · Updated

Use Bedrock model evaluation, RAG evaluation and LLM-as-a-judge, then diagnose bad retrieval, hallucinations, truncation and throttling.

This lesson is part of full AIP-C01 access

The first 2 AIP-C01 lessons are free. Full access opens every lesson with its practice questions, and the exam-style AIP-C01 question bank.

In this lesson
  • Amazon Bedrock evaluations
  • Testing practice
  • Find the failing layer
  • Observability
  • Exam traps
  • Practice questions

Get full access Try the free practice exam Log in

AIP-C01 course outline

All 6 lessons · Course overview
  1. Amazon Bedrock Knowledge Bases and RAGFoundation Model Integration, Data Management, and Compliance Free
  2. Choosing models and inference options in Amazon BedrockFoundation Model Integration, Data Management, and Compliance Free
  3. Amazon Bedrock Agents and orchestrationImplementation and Integration Full access
  4. Amazon Bedrock Guardrails and AI safetyAI Safety, Security, and Governance Full access
  5. Cost, latency and scaling for Bedrock applicationsOperational Efficiency and Optimization Full access
  6. Evaluating and troubleshooting generative AI applicationsTesting, Validation, and Troubleshooting Full access