Skip to content

🎯 Azure AI Vision Image Analysis Study Guide

🔍 Problem

You need to analyze images to automatically:

  • 📝 Generate descriptive captions in natural language
  • 🏷️ Suggest tags representing objects, scenery, or actions
  • 🎯 Detect and locate objects or people in the image

You want to implement this solution using Azure services with appropriate architecture, components, and configurations.

☁️ Solution with Azure

Use Azure AI Vision (Computer Vision) service to analyze images and extract information. Provision an Azure AI Vision resource and connect to it through REST API or SDK (Python, .NET, etc.).

🔧 Key capabilities include:

  • 📝 CAPTION: Generate a natural language description of the image
  • 🏷️ TAGS: Identify objects, scenery, setting, and actions
  • 📦 OBJECTS: Locate objects with bounding boxes
  • 👥 PEOPLE: Locate people with bounding boxes
  • 📖 DENSE_CAPTIONS: Generate detailed captions for detected objects
  • ✂️ SMART_CROPS: Suggest crop regions for a specified aspect ratio
  • 🔤 READ: Extract readable text from images (OCR)

🧩 Components Required

  • 🌐 Azure AI Vision resource, provisioned in one of the following ways:

    • Azure AI Foundry project → AI Foundry hub → multi-service resource (includes AI Vision)
    • Azure AI services multi-service resource
    • Standalone Computer Vision resource (includes free tier for testing)

  • 📱 Client app (Python, .NET, etc.) using:

    • REST API
    • SDK (e.g., azure.ai.vision.imageanalysis for Python)

  • 🔐 Authentication:

    • Key-based (authorization key)
    • Microsoft Entra ID token
    • (Production) Managed identity or Azure Key Vault for securing credentials

🏗️ Architecture / Development

1. 🚀 Provision the resource:

  • Create AI Vision / multi-service resource
  • Obtain endpoint (e.g., https://<resource_name>.cognitiveservices.azure.com/)
  • Obtain key or set up Entra ID access

2. 🔌 Connect client app:

Example in Python (key-based):

from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.core.credentials import AzureKeyCredential

client = ImageAnalysisClient(
    endpoint="<YOUR_RESOURCE_ENDPOINT>",
    credential=AzureKeyCredential("<YOUR_AUTHORIZATION_KEY>")
)

result = client.analyze(
    image_data=<IMAGE_DATA_BYTES>,
    visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS],
    gender_neutral_caption=True,
)

3. 🖼️ Image requirements:

  • Format: JPEG, PNG, GIF, BMP
  • Size: < 4 MB
  • Dimensions: > 50 x 50 pixels

4. 📤 Submit image:

  • Upload image bytes
  • Or provide URL using analyze_from_url

5. 📥 Receive response (JSON with captions, tags, bounding boxes, confidence scores, etc.)

Example JSON excerpt:

{
    "denseCaptionsResult": {
        "values": [
            {
                "text": "a house in the woods",
                "confidence": 0.705,
                "boundingBox": { "x": 0, "y": 0, "w": 640, "h": 640 }
            }
        ]
    }
}

⭐ Best Practice / Considerations

  • 🔒 Use Microsoft Entra ID authentication + managed identity for production security
  • 🗝️ Secure keys in Azure Key Vault if key-based authentication is required
  • 🤝 For collaborative or multi-AI service solutions, prefer Azure AI Foundry projects
  • 📏 Ensure image size, format, and dimensions meet requirements to avoid API errors
  • ⚡ Specify only necessary visual features to reduce processing time and cost

📝 Simulated Exam Questions

1️⃣ You need to generate a natural language caption and identify tags for an image using Azure. Which visual features should you request?

  • ✅ CAPTION, TAGS
  • ❌ OBJECTS, PEOPLE
  • ❌ READ, SMART_CROPS

2️⃣ A client app sends images for analysis but fails due to large file size. What is the maximum allowed file size for image analysis in Azure AI Vision?

  • ✅ 4 MB
  • ❌ 10 MB
  • ❌ 50 MB
  • ✅ Microsoft Entra ID with managed identity
  • ❌ Hardcoded authorization key in app
  • ❌ Anonymous access

4️⃣ Which resource type should you choose if you want to experiment with Azure AI Vision at no cost?

  • ✅ Standalone Computer Vision resource (free tier)
  • ❌ AI Foundry project
  • ❌ Multi-service AI resource