Cross-cutting Experiments¶

Experiments that span multiple Azure PaaS services and test platform-level behaviors shared across App Service, Functions, and Container Apps.

Why Cross-cutting?¶

Some troubleshooting scenarios are not specific to a single compute service. Identity propagation, DNS resolution, and networking behaviors apply across App Service, Functions, and Container Apps. Testing these cross-cutting behaviors reveals whether the root cause is in the shared platform infrastructure or in service-specific implementation.

graph LR
    subgraph "Cross-cutting Concerns"
        MI[Managed Identity<br/>& RBAC] --> AS[App Service]
        MI --> FN[Functions]
        MI --> CA[Container Apps]

        DNS[Private DNS<br/>& Endpoints] --> AS
        DNS --> FN
        DNS --> CA

        VNET[VNet<br/>Integration] --> AS
        VNET --> FN
        VNET --> CA
    end

    subgraph "Shared Infrastructure"
        AAD[Microsoft Entra ID]
        PDNS[Azure Private DNS Zones]
        PE[Private Endpoints]
    end

    MI --> AAD
    DNS --> PDNS
    DNS --> PE

    style MI fill:#4a9eff,color:#fff
    style DNS fill:#ff9800,color:#fff
    style VNET fill:#9c27b0,color:#fff

Key Cross-cutting Patterns¶

Pattern	Description	Affected Services
RBAC Propagation Delay	Role assignment changes take time to propagate through Entra ID and local token caches	All services using managed identity
DNS Negative Caching	Failed DNS lookups are cached, extending outages beyond the original failure window	All services with VNet integration
VNet DNS Resolution	Custom DNS servers and Azure DNS Private Zones interact differently across services	App Service, Functions (Flex), Container Apps
Token Cache Behavior	Managed identity tokens are cached at multiple layers with different TTLs	All services using managed identity

Experiment Status¶

Experiment	Type	Status	Description
MI RBAC Propagation	Hybrid	Published	Role assignment delay and token cache interaction
PE DNS Negative Cache	Hybrid	Published	Extended outages from DNS negative cache during PE cutover
Ingress Idle Timeout	Hybrid	Draft	Ingress idle timeout vs application streaming
Config Change Restart	Hybrid	Draft	Container restart behavior on configuration changes

Planned Experiments¶

Managed Identity RBAC Propagation — Published¶

Role assignment delay and token cache interaction across services. Tested end-to-end propagation time from az role assignment create to successful API call across Key Vault (~10s), Storage (~60s), and Service Bus (~5 min). Discovered that Service Bus has a server-side RBAC authorization cache that persists through application restarts — the delay is NOT Azure Identity SDK token cache.

Private Endpoint DNS Negative Caching — Published¶

Extended outages from DNS negative cache during private endpoint cutover. Unexpected finding: DNS negative caching was NOT observed in App Service Linux — transitions were immediate. The experiment refutes the original hypothesis for this platform.

Draft Experiments¶

Ingress Idle Timeout — Draft¶

Ingress idle timeout vs application streaming behavior. Tests what happens when long-running requests (SSE, file downloads) exceed the ingress idle timeout, and whether the timeout applies to idle connections or total request duration.

Status: Draft - Awaiting Execution

Designed based on Oracle recommendations. High priority for streaming/SSE use cases.

Config Change Restart — Draft¶

Container restart behavior on configuration changes. Tests which configuration changes trigger container restarts across services, and whether in-flight requests are handled gracefully.