Azure VNet Flow Log Analytics Without Traffic Analytics

Problem / Motivation

Azure VNet flow logs contain valuable network telemetry — every allowed and denied flow through an NSG or Azure Firewall — but Azure’s built-in Traffic Analytics adds ongoing per-workspace cost and does not always expose the specific query patterns a connectivity troubleshooting session needs. This project explores a lighter-weight, cost-conscious pipeline that ingests raw flow log records directly and makes them queryable without that dependency.

Approach

A small pipeline reads VNet flow log records from the storage account Network Watcher writes to, normalizes and enriches them with Python (resolving subnet, NSG rule, and resource context that the raw record alone does not carry), and loads the result into a queryable store so common connectivity questions — which rule denied this flow, which subnet generated the most traffic, where did a specific source/destination pair actually route — can be answered with KQL. Grafana provides exploratory visualization during development.

Technologies

• Azure Network Watcher (VNet flow logs)
• Python (parsing and enrichment)
• KQL (querying)
• Grafana (exploratory visualization)

Current status

This is a cost-conscious parsing, enrichment, querying, and visualization concept currently at the prototype stage. No production deployment exists, no measured cost comparison against Traffic Analytics has been published, and no public repository or demo is available yet. The status label above reflects that honestly.

Next steps

Validate query performance against a realistic flow-log volume, and evaluate whether the approach meaningfully reduces cost compared to Traffic Analytics before considering a wider rollout.