Azure Vision and Content Understanding
Computer vision workloads appear on every AI-901 exam. The skill is matching a scenario to the right service, not remembering API details.
Azure Vision
Azure Vision in Foundry Tools (formerly Azure AI Vision) analyses images. Its main capabilities:
- Image analysis: captions, dense captions, tags, object detection, people detection and smart cropping (for thumbnails).
- OCR (Read): extracts printed and handwritten text from images and documents.
- Image classification and object detection with custom models: you train them with your own labeled images when prebuilt tags are too general, such as identifying specific product types.
Computer vision is the workload name. Classification answers “what is this image?”, object detection answers “what is in it and where?” (bounding boxes).
Document extraction
Plain OCR gives you text and layout, but not meaning. When you need “invoice total” or “vendor name” as fields, you need extraction that understands document structure. Document Intelligence (formerly Form Recognizer) does this with prebuilt models for invoices, receipts and IDs.
Azure Content Understanding
Azure Content Understanding in Foundry Tools turns unstructured content into structured output. It works on documents, images, audio and video.
The core idea is the analyzer. You define (or pick a prebuilt) analyzer that says which fields to extract and, optionally, a schema with nested values. You then submit content to it and read the result.
| Content | Typical output |
|---|---|
| Document | Markdown text plus fields (totals, dates, line items) |
| Audio | Transcription, speakers, summary fields |
| Video | Scenes, transcript, key frames, fields per segment |
| Image | Description and fields |
For audio, converting speech to text is called transcription. Content Understanding runs it for you as part of analysis.
Calling it from Python
Analysis can take seconds to minutes, so it is a long-running operation. The pattern is: create a client, call begin_analyze with the analyzer ID and your input, get a poller, then call poller.result() to read the extracted fields. If you see “begin” in an Azure SDK method, expect a poller.
Exam traps
- OCR vs structured extraction: OCR returns text. Fields with a schema need an analyzer (or Document Intelligence).
- Vision vs Language: pictures go to Vision, text goes to Language. Handwriting in a photo is Vision (OCR) first.
- Classification vs detection: only detection gives locations of objects.
- Synchronous vs long-running: Content Understanding uses a poller, so do not pick a plain
analyze()that returns instantly. - Old names: Computer Vision, Form Recognizer and Azure AI services are older names that may still appear.
Practice questions
Choose an answer, then review the explanation to check your understanding.
1. An app must read printed and handwritten text from photos of letters. Which capability is this?
Show answer
Answer: B.OCR (the Read capability in Azure Vision) extracts printed and handwritten text from images.
2. Invoices must be turned into structured fields such as vendor, total and nested line items using a defined schema. What do you create?
Show answer
Answer: B.An analyzer defines the fields to extract, including nested ones, and returns structured results.
3. In the Content Understanding Python SDK, document analysis can take time. How do you get the result?
Show answer
Answer: A.Analysis is a long-running operation, so you start it with begin_analyze and wait on the poller for the result.
4. Which task is best suited to Azure Vision?
Show answer
Answer: B.Azure Vision analyses images, with captions, tags, objects, people and OCR.