Choosing models and inference options in Amazon Bedrock
Choosing the right model and the right way to call it decides cost, latency and reliability. The exam tests these trade-offs with short scenarios.
Picking a model
Amazon Bedrock offers models from Amazon (Nova, Titan), Anthropic, Meta, Mistral, Cohere, AI21 and others. Match the model to the task:
- Complex reasoning and agents: a large model.
- High-volume classification or simple extraction: a small, fast, cheap model.
- Image or video input: a multimodal model.
- Search embeddings: an embeddings model, not a text-generation model.
Use model evaluation to compare candidates with your own data before you commit.
Which API to call
| API | Use it when |
|---|---|
| InvokeModel | You need the model’s native request body. |
| InvokeModelWithResponseStream | You want tokens as they are produced. |
| Converse / ConverseStream | You want one message format for all models, with tool use and system prompts. |
| Batch inference job | Large offline workloads. |
Streaming improves perceived latency because users see text while the model is still generating.
Pricing and capacity modes
- On-demand: pay per input and output token, no commitment, subject to quotas.
- Provisioned Throughput: you buy model units for a 1-month, 6-month or no-commitment term. It gives predictable throughput and is required for custom (fine-tuned or continued pre-trained) models.
- Batch: asynchronous, discounted.
- Prompt caching: reuses a long repeated prompt prefix to cut cost and latency.
Cross-Region inference
An inference profile lets Bedrock route a request to another Region when the source Region is busy. Geographic profiles (for example US or EU) keep data inside a geography; global profiles route anywhere. Check this when a question mentions data residency.
Customizing a model
| Approach | Best for |
|---|---|
| Prompt engineering | Quick format and tone changes. |
| RAG | Fresh or private facts. |
| Fine-tuning | Consistent style or task behaviour with labelled examples. |
| Continued pre-training | Learning a domain vocabulary from unlabelled text. |
| Distillation | Moving a large model’s skill into a smaller, cheaper one. |
Try them in that order; each costs more than the one before.
Exam traps
- Fine-tuned models cannot be called on-demand; they need Provisioned Throughput.
- Larger is not always better; the cheapest model that meets quality wins.
- Cross-Region inference profiles are not the same as copying a model to another Region.
- Use RAG, not fine-tuning, when the problem is outdated or missing facts.
- Batch inference is for offline work; do not use it for interactive chat.
Practice questions
Choose an answer, then review the explanation to check your understanding.
1. A developer wants one request format that works across many Bedrock text models, including multi-turn chat and tool use. Which API should be used?
Show answer
Answer: B.The Converse API gives a consistent message format across models, so switching models needs little code change.
2. A steady, high-volume production workload needs guaranteed throughput for a fine-tuned custom model. What is required?
Show answer
Answer: B.Custom models can only be invoked through Provisioned Throughput, which reserves model units for guaranteed capacity.
3. Traffic spikes cause throttling in one Region. What is the simplest way to raise throughput without managing Regions yourself?
Show answer
Answer: A.A cross-Region inference profile routes requests to several Regions automatically for higher throughput and resilience.
4. A nightly job must summarize 500,000 documents and results are not needed immediately. How should it run to save cost?
Show answer
Answer: B.Batch inference processes prompts asynchronously from S3 at a lower price than on-demand.