Lab: DNS Resolution Behavior for App Service Before and After VNet Integration¶

This Level 3 lab guide documents how DNS behaves in Azure App Service Linux when the app is not VNet-integrated, and how failures emerge after introducing VNet integration with misconfigured DNS components.

This lab intentionally includes a scientific outcome where the original “failure” does not occur in the captured artifact set. That is expected and valid. The artifacts prove that the non-VNet baseline resolves public hostnames correctly and provide a reusable diagnostic framework for the true failure mode.

Lab Metadata¶

1) Background¶

1.1 Why this lab matters¶

DNS problems in App Service are commonly misdiagnosed as generic network outages. In practice, name resolution, route reachability, and TLS validation are separate stages. If you skip that separation, you can “fix” routing while DNS still fails, or “fix” DNS while TLS still fails.

This lab exists to separate those stages with evidence.

1.2 Core platform model¶

For outbound dependency calls from App Service Linux, there are three independent questions:

1. Can the app resolve the dependency hostname to an IP address?
2. Can packets reach that IP over the selected network path?
3. Does TLS/HTTP succeed once TCP is established?

If question 1 fails, questions 2 and 3 are never reached.

1.3 DNS behavior without VNet integration¶

When App Service is not integrated with a VNet, DNS resolution follows Azure public resolver behavior. For typical public endpoints, this is straightforward and stable.

In this lab’s baseline evidence, DNS queries from inside the app process return valid records for:

• management.azure.com
• login.microsoftonline.com
• <storage-account>.blob.core.windows.net

1.4 DNS behavior with regional VNet integration¶

Regional VNet integration changes outbound network path and DNS dependency chain. After integration, resolution can depend on:

• VNet DNS server settings
• Custom DNS forwarders
• Azure DNS Private Resolver (if used)
• Private DNS zone links
• Zone record correctness

If any link in that chain is wrong, specific hostnames may fail while others continue to resolve.

Portal view: Virtual Network Integration blade (configuration entry point)¶

The Virtual Network Integration blade is the Portal entry point that switches the app from the non-VNet DNS path described in section 1.3 to the VNet-integrated DNS dependency chain described above - clicking Add virtual network integration selects a subnet and binds the app to it, after which DNS queries route through VNet DNS settings, custom forwarders, or an Azure DNS Private Resolver instead of Azure public resolvers. This blade is distinct from the unified Networking blade embedded later at section 3.8 in two ways: this one is the dedicated remediation surface where you actually perform the integration action, while the Networking blade is the diagnostic snapshot for confirming current state. The Disconnect and Troubleshoot toolbar buttons are correctly disabled in this capture because there is no existing integration to disconnect or troubleshoot - once a subnet is bound, both buttons activate and Troubleshoot opens the same VNet-integration detector reachable from the Diagnose and solve problems blade. When this blade transitions from the placeholder shown here to a populated subnet binding, expect the failure modes described in sections 1.5 (split-horizon DNS for Private Endpoints) and 1.8 (VNet-integrated failure mode) to become reachable, which is why the after-state is the focus of this lab's hypothesis.

1.5 Private Endpoints and split-horizon DNS¶

Private Endpoints rely on split-horizon DNS. The same logical service can return different answers depending on resolver context:

• Public resolver path: public IP answer
• Private resolver path (with zone links): private IP answer

If private zone linkage is missing, results may degrade to:

• NXDOMAIN
• wrong (public) target for private dependency intent
• stale/cached unexpected answer

1.6 Diagram: end-to-end resolution and connect path¶

flowchart TD
    A[App code calls requests.get or socket.getaddrinfo] --> B[Container resolver 127.0.0.11]
    B --> C{Resolver chain}
    C -->|No VNet integration| D[Azure public DNS path]
    C -->|VNet integration enabled| E[VNet DNS settings and forwarders]
    D --> F[Public DNS answer]
    E --> G{Private zone linked and records valid?}
    G -->|Yes| H[Private IP answer]
    G -->|No| I[NXDOMAIN or wrong answer]
    F --> J[TCP/TLS stage]
    H --> J
    I --> K[Dependency failure at resolution stage]

1.7 Diagram: non-VNet baseline behavior in this experiment¶

sequenceDiagram
    participant App as App Service app
    participant Resolver as Embedded resolver (127.0.0.11)
    participant DNS as Azure DNS path
    participant Storage as Azure Storage endpoint

    App->>Resolver: Resolve storage.blob.core.windows.net
    Resolver->>DNS: Upstream query
    DNS-->>Resolver: Public IP result
    Resolver-->>App: Address list
    App->>Storage: HTTPS GET /?comp=list
    Storage-->>App: HTTP 403 (expected unauthenticated)

1.8 Diagram: VNet-integrated failure mode (target scenario)¶

sequenceDiagram
    participant App as App Service app (VNet-integrated)
    participant Resolver as Embedded resolver (127.0.0.11)
    participant Custom as Custom DNS or Private Resolver
    participant Zone as Private DNS Zone

    App->>Resolver: Resolve storage.privatelink.blob.core.windows.net
    Resolver->>Custom: Forward query
    Custom->>Zone: Query zone record
    Zone-->>Custom: Missing link or record
    Custom-->>Resolver: NXDOMAIN / timeout / wrong target
    Resolver-->>App: Resolution error

1.9 What /etc/resolv.conf means in this lab¶

Captured diag-dns artifacts show:

nameserver 127.0.0.11
options ndots:0 timeout:15 attempts:2

Interpretation:

• The app sees an embedded DNS resolver inside the container context.
• Upstream DNS path is abstracted behind that listener.
• You must validate effective behavior with real lookups, not assumptions.

1.10 Observable signatures by layer¶

1.11 Why this lab uses both /resolve and /connect¶

/resolve isolates DNS.

/connect adds HTTP/TLS behavior after resolution.

This separation is required because a failed /connect alone is ambiguous. It could be DNS, routing, TLS, auth, or service-level rejection.

1.12 Ground truth from captured baseline artifacts¶

The baseline artifact set in this repository shows:

• DNS succeeded for tested public hostnames.
• App startup probe succeeded.
• HTTP status codes for lab endpoints were 200.
• No captured KQL DNS error signatures in the provided snapshot files.

This is consistent with a healthy non-VNet DNS path.

1.13 What this background section is not claiming¶

This guide does not claim that non-VNet is always better. It claims only that, in this controlled artifact set:

• public DNS worked,
• and the intended “broken private DNS after VNet integration” condition was not actually materialized in captured runtime results.

That distinction is critical for incident-quality documentation.

2) Hypothesis¶

2.1 Primary hypothesis for this lab¶

When an App Service is deployed without VNet integration, DNS resolution uses Azure public DNS behavior and resolves public hostnames correctly. When VNet integration is added with custom DNS or Private DNS zones, resolution may fail for certain hostnames if the DNS chain is misconfigured.

2.2 Causal chain¶

1) App Service without VNet integration
   -> DNS path resolves public hostnames normally
   -> dependency calls to public endpoints can proceed

2) Enable VNet integration + custom/private DNS path
   -> DNS responsibility shifts to VNet resolver chain
   -> missing forwarders or private zone links cause lookup errors

3) Lookup errors propagate upward
   -> /resolve returns errors
   -> /connect fails before successful dependency interaction
   -> application requests may show elevated latency or 5xx

2.3 Proof criteria¶

The hypothesis is considered supported if all of the following are observed:

1. Non-VNet baseline:
   • /resolve returns successful DNS answers for public hostnames.
   • diag-dns shows valid answers for management/login/storage public names.
2. VNet-integrated misconfigured run:
   • one or more hostnames fail resolution (ok: false or resolver error).
   • resolver/path evidence shows missing zone link or forwarding issue.
3. Correlated telemetry:
   • HTTP failures or increased latency align with DNS error window.

2.4 Disproof criteria¶

The hypothesis is considered not supported if any of the following occur:

1. Non-VNet baseline already fails public hostname resolution.
2. VNet-integrated scenario resolves correctly despite intended DNS misconfiguration.
3. Failures are explained by non-DNS causes only (for example TLS hostname mismatch with successful DNS).

2.5 Partial-support criteria¶

The hypothesis is considered partially supported if:

• baseline behavior is proven healthy,
• but the misconfigured VNet-integrated failure is not directly reproduced in captured artifacts,
• while architecture analysis still identifies the expected failure mechanism.

2.6 Scientific interpretation for this repository’s artifact set¶

This repository’s captured artifacts support only the first half of the hypothesis directly:

• Non-VNet baseline and trigger data show successful DNS for tested names.

The second half (actual failure under VNet + misconfigured DNS) is provided as:

• a validated troubleshooting framework,
• a runbook to reproduce,
• and explicit diagnostic checkpoints.

That is still valid lab output.

2.7 Confounders to control¶

Potential confounders:

• TLS certificate mismatch for privatelink hostnames
• temporary platform restart events unrelated to DNS
• stale DNS caches
• dependency-side authorization errors (for example HTTP 403)

Control approach:

• separate /resolve from /connect
• log timestamps in UTC
• correlate with KQL HTTP + platform + console data

2.8 Decision tree for investigators¶

flowchart TD
    A[/resolve success?] -->|No| B[DNS failure confirmed]
    A -->|Yes| C[/connect success?]
    C -->|No| D[Not a pure DNS failure; inspect TLS/auth/routing]
    C -->|Yes| E[No dependency-path failure in current run]
    B --> F[Check VNet DNS chain: resolver, forwarders, private zone links]

3) Runbook¶

This runbook is written as an incident-grade procedure with strict command formatting. All CLI examples use long flags only.

3.1 Prerequisites¶

3.2 Environment variables¶

export RG="rg-lab-dns"
export LOCATION="koreacentral"
export APP_NAME=""
export APP_URL=""

3.3 Deploy baseline environment¶

az group create \
    --name "$RG" \
    --location "$LOCATION"

az deployment group create \
    --resource-group "$RG" \
    --template-file "labs/dns-vnet-resolution/main.bicep" \
    --parameters "baseName=labdns"

3.4 Discover app name and URL¶

APP_NAME=$(az webapp list \
    --resource-group "$RG" \
    --query "[0].name" \
    --output tsv)

APP_URL="https://$(az webapp show \
    --resource-group "$RG" \
    --name "$APP_NAME" \
    --query "defaultHostName" \
    --output tsv)"

az webapp show \
    --resource-group "$RG" \
    --name "$APP_NAME" \
    --query "{name:name,state:state,defaultHostName:defaultHostName}" \
    --output table

3.5 Capture baseline endpoint evidence¶

curl --silent --show-error "$APP_URL/health"

Expected shape:

{"status":"healthy"}

curl --silent --show-error "$APP_URL/diag/stats"

Expected fields:

• pid
• process_start_time
• uptime_seconds
• request_count
• endpoint_counters

curl --silent --show-error "$APP_URL/diag/env"

Expected fields:

• PORT
• WEBSITES_PORT
• STORAGE_ACCOUNT_NAME

curl --silent --show-error "$APP_URL/diag/dns"

Check:

• resolv_conf
• resolutions array

3.6 Trigger DNS/connect checks¶

bash "labs/dns-vnet-resolution/trigger.sh" "$APP_URL"

The script calls in order:

1. /resolve
2. /connect

3.7 Pull app config for network context¶

az webapp config show \
    --resource-group "$RG" \
    --name "$APP_NAME" \
    --output json

Inspect:

• vnetName
• vnetRouteAllEnabled
• linuxFxVersion
• appCommandLine

3.8 Query HTTP logs in Log Analytics¶

Portal view: Networking blade (VNet integration context)¶

The Networking blade is the Portal anchor for DNS-resolution failures: Outbound DNS under Outbound traffic configuration reveals whether the app is using Default (Azure-provided) resolvers or a custom Private DNS Zone, and Virtual network integration shows whether outbound traffic is going through a VNet at all. In this capture, Virtual network integration: Not configured and Outbound DNS: Default (Azure-provided) together mean DNS queries are using the public Azure DNS resolver - which cannot resolve *.privatelink.<service>.azure.com hostnames. If your KQL queries below show ENOTFOUND or EAI_AGAIN on private hostnames, this blade tells you immediately that the cause is missing VNet integration, not application code. Click the Troubleshoot button in the toolbar for the Networking-specific detector that walks through the same diagnosis.

AppServiceHTTPLogs
| where TimeGenerated > ago(6h)
| where CsHost has "azurewebsites"
| project TimeGenerated, CsUriStem, ScStatus, TimeTaken, CsHost
| order by TimeGenerated desc

3.9 Query console logs for DNS-related signatures¶

AppServiceConsoleLogs
| where TimeGenerated > ago(6h)
| where ResultDescription has_any ("ENOTFOUND", "EAI_AGAIN", "Name or service not known", "getaddrinfo", "DNS")
| project TimeGenerated, ResultDescription
| order by TimeGenerated desc

3.10 Query platform lifecycle logs¶

AppServicePlatformLogs
| where TimeGenerated > ago(6h)
| project TimeGenerated, Level, Message
| order by TimeGenerated desc

3.11 Controlled transition to VNet-integrated failure simulation¶

If you want to reproduce the second half of the hypothesis (misconfigured DNS), run these steps:

1. Integrate app with delegated subnet.
2. Configure custom DNS or private DNS pattern.
3. Intentionally omit private zone link.
4. Repeat /resolve and /connect.

Example link check command:

az network private-dns link vnet list \
    --resource-group "$RG" \
    --zone-name "privatelink.blob.core.windows.net" \
    --output table

Example record check command:

az network private-dns record-set a list \
    --resource-group "$RG" \
    --zone-name "privatelink.blob.core.windows.net" \
    --output table

3.12 Apply remediation¶

az network private-dns link vnet create \
    --resource-group "$RG" \
    --zone-name "privatelink.blob.core.windows.net" \
    --name "link-to-app-vnet" \
    --virtual-network "<vnet-name>" \
    --registration-enabled false

Then rerun:

curl --silent --show-error "$APP_URL/resolve"
curl --silent --show-error "$APP_URL/connect"

3.13 Runbook acceptance checklist¶

3.14 Troubleshooting tips during execution¶

4) Experiment Log¶

This section is based on real artifacts in:

labs/dns-vnet-resolution/artifacts-sanitized/

4.1 Experiment metadata¶

Source files:

• baseline/app-config.json
• trigger/web-derived KQL snapshots

4.2 Baseline health and process state evidence¶

From baseline/health.json:

{"status":"healthy"}

From baseline/diag-stats.json:

{"endpoint_counters":{"<unknown>":1,"diag_stats":2,"index":1},"pid":1897,"process_start_time":"2026-04-04T05:06:04.352628+00:00","request_count":4,"uptime_seconds":1621.163}

Interpretation:

• App was healthy and serving requests.
• Process uptime exceeded 27 minutes at capture.
• No restart storm signal in baseline snapshot.

4.3 Baseline DNS evidence¶

From baseline/diag-dns.json:

• management.azure.com resolved
• login.microsoftonline.com resolved
• <storage-account>.blob.core.windows.net resolved
• resolver config includes nameserver 127.0.0.11

Selected payload excerpt:

{
  "resolutions": [
    {
      "hostname": "management.azure.com",
      "resolved_ips": ["2603:1030:a0b::10", "<ip-redacted>"]
    },
    {
      "hostname": "stlabdnsfbg6cycknd2gm.blob.core.windows.net",
      "resolved_ips": ["<ip-redacted>"]
    }
  ],
  "resolv_conf": "nameserver 127.0.0.11 ..."
}

4.4 Environment variables in baseline¶

From baseline/diag-env.json:

Interpretation:

• App binds to 8000.
• No explicit WEBSITES_PORT override in this snapshot.

4.5 Trigger endpoint results¶

From trigger/resolve-response-20260404T053457Z.json:

{
  "status": "ok",
  "results": [
    {
      "hostname": "stlabdnsfbg6cycknd2gm.blob.core.windows.net",
      "ok": true
    },
    {
      "hostname": "stlabdnsfbg6cycknd2gm.privatelink.blob.core.windows.net",
      "ok": true
    }
  ]
}

Important:

• Both hostnames resolved successfully in this capture.
• This includes the privatelink hostname.

4.6 Trigger connect results and interpretation¶

From trigger/connect-response-20260404T053457Z.json:

Key point:

The private hostname failure captured here is TLS hostname validation, not DNS resolution failure.

4.7 HTTP log evidence (KQL export)¶

From trigger/kql-http-20260404T060610Z.json:

Relevant rows:

Interpretation:

• Trigger endpoints executed successfully from HTTP perspective.
• /connect latency higher than /resolve, consistent with outbound dependency attempt.

4.8 Platform log evidence (KQL export)¶

From trigger/kql-platform-20260404T060610Z.json:

Observed messages include:

• Setting value of PORT variable to 8000
• Overriding PORT environment variable with pre-calculated port: 8000.
• Pinging warmup path to ensure container is ready to receive requests.
• Site startup probe succeeded after 36.0705838 seconds.

Interpretation:

• Startup flow was healthy during captured run.
• No platform evidence of DNS-related startup failure in this dataset.

4.9 Console log evidence (KQL export)¶

From trigger/kql-console-20260404T060610Z.json:

Observed lines include:

• Listening at: http://0.0.0.0:8000
• Booting worker with pid: ...
• Starting gunicorn ...

Interpretation:

• App process was healthy and correctly bound.

4.10 Empty snapshot files and why they matter¶

The following files are intentionally empty in artifacts:

• trigger/kql-http-20260404T060048Z.json
• trigger/kql-console-20260404T060048Z.json
• trigger/kql-platform-20260404T060048Z.json

This is useful evidence:

• query timing and ingestion windows can produce empty exports,
• so runbook should include retry/snapshot timing controls,
• and no conclusion should be drawn from empty files alone.

4.11 Experimental verdict¶

Hypothesis evaluation¶

Overall verdict: Partially supported (baseline confirmed, failure scenario framework documented).

4.12 Scientific finding¶

The captured dataset demonstrates:

1. Public hostname DNS resolution works without VNet integration.
2. Triggered checks return successful resolution even for the tested privatelink hostname.
3. The observed failure in /connect is TLS-hostname mismatch, not lookup failure.
4. The intended “DNS failure due to misconfigured VNet-integrated chain” remains a valid theoretical and reproducible scenario, but it is not present in this particular artifact run.

4.13 Reproduction guidance for true failure mode¶

To force the second-half failure and complete hypothesis proof, capture a run where:

• app is VNet-integrated,
• resolver path points to custom/private chain,
• private zone link to integration VNet is missing,
• /resolve returns explicit lookup error.

Evidence minimum for that run:

1. diag-dns showing resolver path and failing hostname.
2. resolve-response with ok=false and resolver exception.
3. kql-console entries with DNS error signatures.
4. zone link list proving misconfiguration before fix and correctness after fix.

4.14 Incident response value of this lab output¶

Even without the failing VNet-integrated artifact, this guide provides:

• a validated healthy baseline,
• clear layer separation (DNS vs TLS vs HTTP),
• a deterministic runbook for reproducing and proving the real failure,
• and a data model for post-incident evidence collection.

4.15 Artifact index used by this document¶

Baseline artifacts used:

• baseline/health.json
• baseline/diag-stats.json
• baseline/diag-env.json
• baseline/diag-dns.json
• baseline/app-config.json

Trigger artifacts used:

• trigger/resolve-response-20260404T053457Z.json
• trigger/connect-response-20260404T053457Z.json
• trigger/diag-dns-20260404T053457Z.json
• trigger/diag-stats-20260404T053457Z.json
• trigger/kql-http-20260404T060610Z.json
• trigger/kql-console-20260404T060610Z.json
• trigger/kql-platform-20260404T060610Z.json
• trigger/kql-http-20260404T060048Z.json (empty)
• trigger/kql-console-20260404T060048Z.json (empty)
• trigger/kql-platform-20260404T060048Z.json (empty)

4.16 Appendix: command catalog used in this investigation¶

az group create --name "$RG" --location "$LOCATION"
az deployment group create --resource-group "$RG" --template-file "labs/dns-vnet-resolution/main.bicep" --parameters "baseName=labdns"
az webapp list --resource-group "$RG" --query "[0].name" --output tsv
az webapp show --resource-group "$RG" --name "$APP_NAME" --query "defaultHostName" --output tsv
az webapp config show --resource-group "$RG" --name "$APP_NAME" --output json
az webapp vnet-integration list --resource-group "$RG" --name "$APP_NAME" --output json
az network private-dns link vnet list --resource-group "$RG" --zone-name "privatelink.blob.core.windows.net" --output table
az network private-dns record-set a list --resource-group "$RG" --zone-name "privatelink.blob.core.windows.net" --output table
az network private-dns link vnet create --resource-group "$RG" --zone-name "privatelink.blob.core.windows.net" --name "link-to-app-vnet" --virtual-network "<vnet-name>" --registration-enabled false
az group delete --name "$RG" --yes --no-wait

Expected Evidence¶

This section defines what you SHOULD observe at each phase of the lab. Use it to validate your investigation is on track.

Before Trigger (Baseline)¶

During Incident¶

After Recovery¶

Evidence Timeline¶

graph TD
    A[Baseline Capture] --> B[Trigger Fault]
    B --> C[During: Collect Evidence]
    C --> D[After: Compare to Baseline]
    D --> E[Verdict: Confirmed/Falsified]

Evidence Chain: Why This Proves the Hypothesis¶

Clean Up¶

az group delete --name "$RG" --yes --no-wait

Related Playbook¶

• DNS Resolution with VNet-Integrated App Service

See Also¶

• DNS Resolution with VNet-Integrated App Service
• Outbound network first-10-minutes checklist
• KQL HTTP: 5xx trend over time
• KQL Console: startup errors

Sources¶

• Integrate your app with an Azure virtual network
• Azure App Service networking features
• Name resolution for resources in Azure virtual networks
• Azure DNS private zones overview
• Azure private endpoint DNS configuration
• Enable diagnostic logging for apps in Azure App Service
• Azure Monitor Logs query language tutorial