Log Analytics: the ingestion bill nobody forecast

Nobody approves a Log Analytics bill. Someone opens a resource, clicks Diagnostic settings, ticks allLogs because it's the safe-looking box, and points it at the workspace. A setting that adds 20 GB a day is 7,300 GB a year of billable ingestion. Multiply that by your region's per-GB price and "five-figure decision" stops sounding like an exaggeration. Nothing in that dialog shows a price.

This is the playbook for taking that decision back: find out what you're paying to ingest, then decide category by category what's worth collecting, which plan it lands in and how long it stays. Six steps, each with a check that tells you it's done.

Step 1: find out which tables you're paying for

Usage
| where TimeGenerated > ago(32d)
| where StartTime >= startofday(ago(31d)) and EndTime < startofday(now())
| where IsBillable == true
| summarize BillableDataGB = sum(Quantity) / 1000. by DataType
| sort by BillableDataGB desc

In most workspaces, three or four tables carry most of the bill. AzureDiagnostics, Perf, ContainerLog and the App* tables are the usual suspects. To see which resource is sending the data, the same page has a find query grouped by _ResourceId. Run it over a single day, because it scans every table. Done when each of the top five tables has a named owner who can say why it's collected.

Step 2: replace allLogs with a list you've read

Go through the settings that feed your top tables and tick the categories someone actually queries. The audit group is often the right smaller default. Untick AllMetrics unless you need to join metrics with logs in KQL: platform metrics are collected automatically and already available in metrics explorer, so sending them to the workspace means paying to ingest a copy. Done when no setting on a high-volume resource uses allLogs without a written reason.

Step 3: give every category a destination before you collect it

Two details decide whether this works. First, Basic and Auxiliary aren't available for every Azure table, and a table's plan can change only once a week. Moving a table to Auxiliary also stops any alerts that run on it. Second, a table can only have one plan, so resource logs need a table of their own. In Azure diagnostics mode every service writes into the shared AzureDiagnostics table. Microsoft says to use resource-specific mode for any new diagnostic setting, which gives each category its own table. When you switch an existing setting, the old rows stay in AzureDiagnostics until retention removes them. Done when every high-volume table is on the plan that matches how it's read.

Step 4: drop rows and columns at ingestion

There's a catch. On Analytics and Basic tables, if a transformation drops more than 50% of the incoming data, the part above 50% is billed as data processing: 20 GB in, 12 GB dropped, 2 GB billed. (Workspaces with Sentinel enabled are exempt.) If you find yourself dropping most of a category, you shouldn't be collecting that category at all. That's step 2. And check your schemas too: column values that don't match the destination table are billed even though they're never stored. Done when the top table's daily volume in Usage drops, and the alerts that read it still fire.

Step 5: set retention per table, and know what doesn't save money

• Going below 31 days. You can set analytics retention as low as four days, but 31 days are already in the ingestion price, so it doesn't reduce costs.
• Purge. Deleting data with the Purge feature doesn't affect retention costs. Only a shorter retention period does.
• Expecting it to apply today. When you shorten total retention, Azure Monitor waits 30 days before it removes the data, so you can undo a mistake.

Done when every table that keeps data longer than the workspace default has a reason written next to it.

Step 6: add a tripwire, then commit

Done when the alert exists, the cap is set, and the tier matches the baseline after cleanup.

The Usage table shows what's inside one workspace. It won't tell you which workspace, in which subscription, is the one that grew. GraphPaaS reads Azure spend for each tenant and breaks it down by resource, resource type and resource group, with a month-by-month drill-down. A workspace that jumped after someone ticked allLogs shows up as its own line, and if the tags reach the cost data, so does the team that owns it. For an MSP, that works across every client tenant from the same screen.