Part 1: Understanding Azure AI Foundry - Hub, Projects & Connections

Sakaldeep Yadav

Dec 8, 2025 • 6 min read

Introduction

You're an enterprise architect at a mid-sized retail company with 200+ stores across multiple regions. Your organization has been using Azure for years - App Services, SQL Database, Data Lake, Synapse Analytics. Now, leadership wants to launch an internal employee support chatbot powered by AI. Before you write a single line of code, you need to answer a fundamental question: What exactly am I building?

This is where Azure AI Foundry comes in. Unlike generic cloud services, Azure AI Foundry introduces a three-tier architectural model that fundamentally changes how you think about AI systems: the Hub, Projects, and Connections. Understanding these three components is the foundation for everything that follows in this 13-part series.

What you'll learn in this post:

What Azure AI Foundry Hub is and why it matters
How Projects provide isolation and governance
How Connections enable secure access to external services
How these three components work together
How your retail chatbot pilot fits into this architecture

Prerequisites: None (this is Part 1)

Complexity Level: Medium

Azure AI Foundry: The Three-Tier Architecture

Azure AI Foundry is Microsoft's unified platform for building, deploying, and managing AI applications at enterprise scale. At its core, it introduces a three-tier architectural model that's fundamentally different from traditional cloud services:

The Hub: Your AI Control Plane

Think of the Hub as the central nervous system of your AI infrastructure. It's not just a workspace, it's a governance boundary, a security boundary, and a compliance boundary all rolled into one.

Key characteristics of a Hub:

Centralized workspace: All your AI projects, connections, and resources live here
Governance scope: Hub-level policies control who can create projects, what compliance rules apply, and how resources are managed
Multi-tenancy: A single Hub can serve multiple business units, teams, or even customers (in multi-tenant scenarios)
Audit trail: Every action in the Hub is logged and auditable
Managed identity: The Hub has its own Azure AD identity for secure service-to-service communication

In your retail scenario: Your Hub is the central workspace where your employee support chatbot project lives. The IT team (Hub Admin) controls who can create projects. The HR team (Project Owner) manages the chatbot project. All activity is logged for compliance.

Projects: Isolated Workspaces Within the Hub

Projects are isolated workspaces within your Hub. Each project is completely separate from other projectsâ€”different data, different compute, different team members, different costs.

Key characteristics of Projects:

Data isolation: Project A's data never mixes with Project B's data
Compute isolation: Project A's models run on separate compute from Project B
Team isolation: Project A's team members can't access Project B's resources (unless explicitly granted)
Cost isolation: Project A's costs are tracked separately from Project B
RBAC isolation: Project A has its own role-based access control

In your retail scenario: Your chatbot is one Project. If you later build an inventory optimization AI, that's a separate Project. Each project has its own data, its own team, its own costs. The HR team can't accidentally access inventory data, and the inventory team can't access HR data.

Connections: Secure Access to External Services

Connections are managed, secure connections to external servicesâ€”Azure OpenAI, data sources, vector databases, etc. They're managed separately from Projects because multiple Projects might use the same Connection.

Key characteristics of Connections:

Credential management: Secrets are stored in Key Vault, not in code
Access control: You control who can use which connections
Audit trail: Every connection usage is logged
Managed identity: Connections use managed identities for secure authentication
Credential rotation: Credentials can be rotated automatically

In your retail scenario: Your chatbot needs to connect to Azure OpenAI to generate responses. That's a Connection. It might also need to connect to your HR system to answer employee questions. That's another Connection. The Data team manages these connections and controls who can use them.

Environments: Deployment Targets

Environments are deployment targets for your models and applications. Typically, you have dev, staging, and production environments. Each environment is isolated and can have different policies.

In your retail scenario: Your chatbot is developed in the dev environment, tested in staging, and deployed to production. Each environment has different security policies, different compute resources, and different monitoring.

Why This Architecture Matters for Enterprise Architects

Governance at Scale

Traditional cloud governance is binary: you either have access to a resource or you don't. Azure AI Foundry governance is granular:

Hub level: Who can create Projects?
Project level: Who can access Project resources?
Connection level: Who can use which external services?

This granularity enables compliance at scale. Your GDPR officer can enforce data residency at the Hub level. Your security team can enforce encryption at the Project level. Your data team can enforce credential management at the Connection level.

Security by Design

Each tier has built-in security controls:

Hub security: Network isolation (VNet, Private Endpoints), identity (managed identities), encryption (CMK), audit (activity logs)
Project security: Data isolation, compute isolation, RBAC, audit
Connection security: Credential management (Key Vault), access control, audit

Cost Management

Because Projects are isolated, you can track costs separately:

Hub costs (shared infrastructure)
Project A costs (chatbot)
Project B costs (inventory optimization)
Connection costs (Azure OpenAI usage)

This enables chargeback to business units.

Operational Clarity

The three-tier model creates clear operational boundaries:

Hub Admin manages the Hub (IT responsibility)
Project Owner manages the Project (business unit responsibility)
Data Team manages Connections (data/security responsibility)

Everyone knows their role and responsibility.

Terraform Implementation Approach

To deploy Azure AI Foundry, you'll use Terraform to create these components. Here's the basic pattern:

# Create the Hub (Machine Learning Workspace)
resource "azurerm_machine_learning_workspace" "hub" {
  name                = "aif-hub-prod"
  location            = "eastus"
  resource_group_name = azurerm_resource_group.hub_rg.name
  
  # Hub identity for secure service-to-service communication
  identity {
    type = "SystemAssigned"
  }
  
  # Reference to Key Vault for encryption keys
  key_vault_id = azurerm_key_vault.hub.id
  
  # Reference to Storage Account for model artifacts
  storage_account_id = azurerm_storage_account.hub.id
}

# Create a Project within the Hub
resource "azurerm_machine_learning_compute" "project_compute" {
  name                          = "chatbot-compute"
  location                      = azurerm_machine_learning_workspace.hub.location
  machine_learning_workspace_id = azurerm_machine_learning_workspace.hub.id
  
  # Compute configuration
  vm_priority = "Dedicated"
  vm_size     = "Standard_D4s_v3"
}

What this does:

Creates a Hub (Machine Learning Workspace) in Azure
Assigns a system-managed identity to the Hub
References Key Vault for encryption keys
References Storage Account for model artifacts
Creates compute resources for the Project

For complete Terraform code with all parameters, networking, RBAC, and monitoring, see:

Compliance & Governance Implications

This three-tier architecture enables compliance at scale:

Hub level: Enforce data residency (EU data stays in EU)
Project level: Isolate personal data by project
Connection level: Control access to data sources
Audit: All data access is logged in Hub audit logs

PCI-DSS Compliance

Hub level: Enforce encryption for payment data
Project level: Isolate payment data from other projects
Connection level: Control access to payment systems
Audit: All payment data access is logged

SOC 2 Type II Compliance

Hub level: Enforce access controls and change management
Project level: Maintain audit trails for all activity
Connection level: Track credential access and rotation
Audit: Evidence of controls available in Hub audit logs

Operational Considerations

Monitoring Hub Health

Monitor Hub availability and latency
Alert on Hub errors or failures
Track Hub resource utilization
Monitor audit log ingestion

Cost Tracking

Hub costs are tracked separately from Project costs
Project costs can be allocated to business units
Connection costs can be tracked per connection
Chargeback can be done at Hub or Project level

Troubleshooting

If a user can't create a Project, check Hub Admin RBAC
If a user can't access a Project, check Project Owner RBAC
If a user can't use a Connection, check Connection Owner RBAC
Check Hub audit logs for all activity

Conclusion & Next Steps

You now understand the three-tier architecture of Azure AI Foundry:

Hub: The governance and security boundary
Projects: Isolated workspaces for specific AI initiatives
Connections: Secure access to external services

This architecture enables governance at scale, security by design, cost management, and operational clarity.

In Part 2, we'll dive deeper into security: how to secure your Hub with network isolation, identity controls, and encryption.

Next steps:

Review your organizational structure and identify Hub Admin, Project Owner, and Connection Owner roles
Plan your Hub deployment (region, compliance requirements)
Review the Terraform examples and understand the resource structure
Read Part 2 to understand security controls

Relevant Azure documentation:

Connect & Questions

Want to discuss Azure AI Foundry architecture, share feedback, or ask questions?

Reach out on X (Twitter) @sakaldeep

Or connect with me on LinkedIn: https://www.linkedin.com/in/sakaldeep/

I look forward to connecting with fellow cloud professionals and learners.

Published by: Azure User Group Nepal
Series: Enterprise AI Governance, Security & Infrastructure with Azure AI Foundry
Part: 1 of 13