Reading sign-in logs: the anomalies worth waking up for

Every tenant that got compromised had the evidence in the sign-in logs first. Not in a SIEM, not in a Defender alert — in the log that ships with the licence, sitting there for anyone who opened the blade. Nobody opened it, because the blade shows the last hundred rows of a firehose and no obvious way to tell an anomaly from a Tuesday. Below are the numbers that came out of real tenants, and what each one turned out to actually mean.

The log you opened is one of four

The first surprise in any tenant is that "the sign-in log" doesn't exist. Microsoft Entra keeps four types of sign-in log: interactive user, non-interactive user, service principal, and managed identity. The legacy portal experience only ever showed the first one, and that habit stuck — admins look at interactive sign-ins, which is where a human typed a password, and never at the other three.

That's backwards for anomaly hunting. A stolen refresh token doesn't generate an interactive sign-in; it generates non-interactive ones. An over-permissioned app registration being used by someone who shouldn't have it shows up under service principal sign-ins, a log with no user column at all. The tab most people never click is the tab where the quiet attacks live.

One row is not one sign-in

The non-interactive log has a property no other log has: rows are aggregated. Microsoft collapses sign-ins that are identical except for their timestamp into a single row with a # sign-ins count, using a documented five-field match — application, user, IP address, status, resource ID — and a time aggregate of 1, 6 or 24 hours.

The practical consequence: row counts lie. A page showing 40 failed non-interactive sign-ins can represent several thousand attempts against one account from one IP. If you are eyeballing volume — and volume is how you spot a spray — you have to read the count column, not the row count. Confidential clients add a second trap on the same log: their IP address is the one from the original token issuance, not the machine actually refreshing the token.

"hidden" is not "none"

Here is the number that changes what you can do at all. The signIn resource exposes riskLevelDuringSignIn, riskLevelAggregated and riskDetail — and the reference carries the same note on all three: details for this property are only available for Microsoft Entra ID P2 customers; all other customers are returned hidden. The value hidden also means the sign-in wasn't enabled for Entra ID Protection.

Reading sign-in logs at all needs Entra ID P1 or P2 in the first place — the signIn reference states it plainly for the Graph API. So a tenant lands in one of three postures, and knowing which one you're in is step zero of any detection work.

The clock you're actually working against

Retention is where good intentions die. The Entra data retention reference is blunt: sign-ins and audit logs keep seven days on Free, 30 days on P1 and P2. Risky sign-ins are the exception that rewards the P2 licence — 7 days Free, 30 days P1, 90 days P2 — while risky users have no limit and aren't deleted until the risk is remediated. And the change isn't retroactive: upgrade from Free to P1 today and you still only get whatever is still inside the seven-day window.

Five anomalies, and the noise that looks like them

With that established, these are the patterns worth an alert. Each one is computable from properties on the signIn resource itself — no P2, no SIEM.

The first row is not hypothetical: legacy protocols are still authenticating in most tenants, and naming the accounts behind them is a short, finite exercise. The fourth row pairs with registration coverage — an account that no policy touches and that has no registered strong method is one password away from being someone else's.

If you do have P2, the riskDetection resource gives you the machine-learned version of the same list — unlikelyTravel, passwordSpray, anonymizedIPAddress, leakedCredentials, suspiciousInboxForwarding and the rest — with a detectionTimingType telling you whether it was caught in real time or offline. Worth knowing before you build detections Microsoft already ships.

Why nobody is reading them

Not laziness — arithmetic. The log is per tenant, capped at 30 days, paginated, aggregated in ways that hide volume, and silent about risk unless you hold the right SKU. Reviewing it properly means an export per tenant, a join against role assignments and registration state, and a comparison to what last week looked like — which is the part nobody keeps. GraphPaaS syncs the sign-in data per tenant on a schedule and keeps the history past the Microsoft retention window, so the question stops being "what happened in the last 30 days" and starts being "what changed since last month".