# aitour26-LTG151-build-trustworthy-ai-with-systematic-evaluations-in-azure-ai-foundry

**Repository Path**: mirrors_microsoft/aitour26-LTG151-build-trustworthy-ai-with-systematic-evaluations-in-azure-ai-foundry

## Basic Information

- **Project Name**: aitour26-LTG151-build-trustworthy-ai-with-systematic-evaluations-in-azure-ai-foundry
- **Description**: Learn to build trustworthy AI with systematic evaluations in Azure AI Foundry. The session covers quality, safety, agent and custom evaluators
- **Primary Language**: Unknown
- **License**: CC-BY-SA-4.0
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-09-13
- **Last Updated**: 2025-09-20

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<p align="center">
<img src="img/Banner-MS-AI-Tour-26.png" alt="decorative banner" width="1200"/>
</p>

# [Microsoft AI Tour 2026](https://aitour.microsoft.com)

## LTG151: Build trustworthy AI with systematic evaluations in Azure AI Foundry

[![Microsoft Azure AI Foundry Discord](https://dcbadge.limes.pink/api/server/ByRwuEEgH4)](https://discord.com/invite/ByRwuEEgH4)
[![Azure AI Foundry Developer Forum](https://img.shields.io/badge/GitHub-Azure_AI_Foundry_Developer_Forum-blue?style=for-the-badge&logo=github&color=adff2f&logoColor=fff)](https://aka.ms/foundry/forum)

---

🎙️ | **Delivering This Session On The Tour?**

- 1️⃣ | Fork this repo to your personal profile
- 2️⃣ | Check out the [session-delivery-sources](./session-delivery-resources/) for next steps
- 3️⃣ | Submit an [issue](https://github.com/microsoft/aitour26-LTG151-build-trustworthy-ai-with-systematic-evaluations-in-azure-ai-foundry/issues/new) if you find bugs or have

---


### **Session Description**

Building generative AI apps starts with model selection—but earning user trust requires continuous evaluation. 

In this talk, learn how Azure AI Evaluations SDK helps assess models pre- and post-production, analyze results, and improve quality through Observability.



### **Learning Outcomes**

By the end of this session, learners will be able to:

- Describe what Observability is, and why it is critical for building trustworthy AI applications end-to-end.
- Describe the different types of Evaluators, and how they help developers assess quality, safety and performance in agentic AI.
- Run evaluations on their target model or application - then view and analyze relevant results in the Azure AI Foundry portal
- Understand how Azure AI Foundry delivers end-to-end observability support through a rich set of tools and features.


### **Technologies Used**

1. GitHub Codespaces
1. Visual Studio Code
1. Azure AI Evaluations (Python SDK)
1. Azure AI Foundry (Portal & SDK)

### **Session Resources**
| Resources          | Links                             | Description        |
|:-------------------|:----------------------------------|:-------------------|
| **Documentation** | [Observability in generative AI](https://aka.ms/models-for-beginners/observability/docs) | Azure AI Foundry documentation for all observability-related tools and features including evaluations, red teaming, tracing, and continuous monitoring. |
 **Samples** | [Azure AI Evaluation Samples](https://aka.ms/models-for-beginners/observability/samples)| Azure AI Evaluation SDK samples (Python) showcasing common scenarios for observability in Azure AI Foundry.| 
| **Breakout** | [AI and Agent Observability in Azure AI Foundry and Azure Monitor](https://build.microsoft.com/en-US/sessions/BRK168?source=sessions) | Learn how evaluation and continuous monitoring can help you iterate quickly and move from pilot to production faster in this hour-long breakout from Microsoft Build 2025 (which inspired this talk) |
| **Skilling**| [Models For Beginners](https://aka.ms/models-for-beginners) | Collection with links to an evolving set of resources from a new open-source curriculum focused on model development - with a focused track on observability|
| | | |


### **Continued Learning Resources**
| Resources          | Links                             | Description        |
|:-------------------|:----------------------------------|:-------------------|
| AI Tour 2026 Resource Center | [https://aka.ms/AITour26-Resource-Center](https://aka.ms/AITour26-Resource-Center?ocid=AITour26_resourcecenter_cnl) | Links to all repos for AI Tour 26 Sessions |
| Azure AI Foundry Community Discord | [![Microsoft Azure AI Foundry Discord](https://dcbadge.limes.pink/api/server/ByRwuEEgH4)](https://discord.com/invite/ByRwuEEgH4)| Connect with the Azure AI Foundry Community! |
| Learn at AI Tour | [https://aka.ms/LearnAtAITour](https://aka.ms/LearnAtAITour?ocid=AITour26_resourcecenter_cnl) | Continue learning on Microsoft Learn |

### **Multi-Language Support**

Additional Languages Coming Soon

## **Content Owners**

<!-- TODO: Add yourself as a content owner
1. Change the src in the image tag to {your github url}.png
2. Change INSERT NAME HERE to your name
3. Change the github url in the final href to your url. -->


<table>
<tr>
    <td align="center">
        <a href="http://github.com/nitya">
            <img src="https://github.com/nitya.png" width="100px;" alt="Nitya Narasimhan"/><br />
            <sub> <b> Nitya Narasimhan, PhD</b> </sub>
        </a>
        <br/>
        <a href="https://linkedin.com/in/nityan" title="talk">📢</a> 
    </td>
    <td align="center">
        <a href="http://github.com/slister1001">
            <img src="https://github.com/slister1001.png" width="100px;" alt="Sydney Lister"/><br />
            <sub> <b> Sydney Lister</b> </sub>
        </a>
        <br/>
        <a href="https://www.linkedin.com/in/sydney-lister/" title="talk">📢</a> 
    </td>
</tr></table>

<br/>

## **Responsible AI**

Microsoft is committed to helping our customers use our AI products responsibly, sharing our learnings, and building trust-based partnerships through tools like Transparency Notes and Impact Assessments. Many of these resources can be found at [https://aka.ms/RAI](https://aka.ms/RAI).
Microsoft’s approach to responsible AI is grounded in our AI principles of fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.

Large-scale natural language, image, and speech models - like the ones used in this sample - can potentially behave in ways that are unfair, unreliable, or offensive, in turn causing harms. Please consult the [Azure OpenAI service Transparency note](https://learn.microsoft.com/legal/cognitive-services/openai/transparency-note?tabs=text) to be informed about risks and limitations.

The recommended approach to mitigating these risks is to include a safety system in your architecture that can detect and prevent harmful behavior. [Azure AI Content Safety](https://learn.microsoft.com/azure/ai-services/content-safety/overview) provides an independent layer of protection, able to detect harmful user-generated and AI-generated content in applications and services. Azure AI Content Safety includes text and image APIs that allow you to detect material that is harmful. Within Azure AI Foundry portal, the Content Safety service allows you to view, explore and try out sample code for detecting harmful content across different modalities. The following [quickstart documentation](https://learn.microsoft.com/azure/ai-services/content-safety/quickstart-text?tabs=visual-studio%2Clinux&pivots=programming-language-rest) guides you through making requests to the service.

Another aspect to take into account is the overall application performance. With multi-modal and multi-models applications, we consider performance to mean that the system performs as you and your users expect, including not generating harmful outputs. It's important to assess the performance of your overall application using [Performance and Quality and Risk and Safety evaluators](https://learn.microsoft.com/azure/ai-studio/concepts/evaluation-metrics-built-in). You also have the ability to create and evaluate with [custom evaluators](https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk#custom-evaluators).

You can evaluate your AI application in your development environment using the [Azure AI Evaluation SDK](https://microsoft.github.io/promptflow/index.html). Given either a test dataset or a target, your generative AI application generations are quantitatively measured with built-in evaluators or custom evaluators of your choice. To get started with the azure ai evaluation sdk to evaluate your system, you can follow the [quickstart guide](https://learn.microsoft.com/azure/ai-studio/how-to/develop/flow-evaluate-sdk). Once you execute an evaluation run, you can [visualize the results in Azure AI Foundry portal ](https://learn.microsoft.com/azure/ai-studio/how-to/evaluate-flow-results).