Building Azure VNet Flow Analytics Without Relying on Traffic Analytics

Azure VNet flow logs carry every allowed and denied flow through an NSG or Azure Firewall, but reaching for Traffic Analytics to make sense of them adds a recurring per-workspace cost that is not always justified for a smaller environment or a focused troubleshooting task. This is a preview of the approach explored in the companion Azure VNet Flow Log Analytics project: parsing and querying flow logs directly.

Preview

This article previews an approach that is still being built out (see the linked project’s Concept / Prototype status). The steps below describe the intended pipeline; they are not yet a completed, benchmarked implementation.

Problem / Context

A flow log record on its own is compact but not immediately answerable against: it identifies source/destination IP and port and an allow/deny decision, but not which NSG rule made that decision or which subnet or resource it belongs to without a join against the environment’s own inventory.

Architecture

The intended pipeline reads flow log JSON records from the storage account Network Watcher writes to, enriches each record with the NSG rule, subnet, and resource context needed to make it answerable, and loads the enriched result into a queryable store.

Implementation

Step 1 — Parse the raw flow log record

Flow log records arrive as JSON blobs containing tuples of (timestamp, source IP, destination IP, source port, destination port, protocol, direction, decision). A Python parser reads each blob and expands it into one row per tuple.

import json

def parse_flow_log_blob(blob_text: str) -> list[dict]:
    records = json.loads(blob_text)["records"]
    rows = []
    for record in records:
        for flow in record["properties"]["flows"]:
            for tuple_group in flow["flows"]:
                for raw_tuple in tuple_group["flowTuples"]:
                    rows.append(_expand_tuple(raw_tuple))
    return rows

Step 2 — Query enriched records

Once enriched records are loaded into a queryable store, a question like “which rule denied the most traffic in the last 24 hours” becomes a direct KQL query rather than a manual log search.

FlowLogRecords
| where TimeGenerated > ago(24h)
| where Decision == "D"
| summarize DeniedFlows = count() by RuleName
| top 10 by DeniedFlows desc

Validation

Warning — not yet benchmarked

No performance or cost comparison against Traffic Analytics has been measured yet. This section will be replaced with real numbers once that benchmark exists, per the project’s Concept / Prototype status.

Test	Expected Result	Result
Parse a sample flow log blob	Rows expand correctly	Pending
Query denied flows by rule	Correct top-N ranking	Pending

Challenges / Limitations

Flow log volume can be substantial in a busy environment, so the enrichment step’s cost and latency need real measurement before this approach can be recommended as a general replacement for Traffic Analytics rather than a targeted, cost-conscious alternative for a specific use case.

Lessons Learned

• A flow log record is only answerable once it carries rule and resource context, not just IPs and ports.
• KQL is a comfortable query layer once records are enriched and loaded.
• A meaningful cost comparison requires real traffic volume, not a synthetic sample.

Conclusion

This preview describes the intended shape of a cost-conscious flow log analytics pipeline. The next update to this article will replace the “pending” validation results above with real measurements from the companion project.