Security review in one prompt
26 Oct 2026
A Microsoft 365 security review in one prompt
Open the Conditional Access list in most tenants and every policy has a state next to it. "Report-only" sits there looking as switched on as "On". It isn't: a report-only policy is evaluated at every sign-in, written to the log, and enforces nothing. A security review that counts policies instead of reading them will call that tenant protected.
The GraphPaaS connector for Claude ships a Security posture review prompt in its tools menu. It starts from the tenant advisor, then reads nine domains in order: baseline settings, MFA registration, standing admin access, what Conditional Access actually enforces, risky OAuth apps, external forwarding, Entra recommendations, Secure Score and open risk detections. Below is that run against Fabrikam HQ, our demo tenant (synthetic people, real tool calls, data collected on 25 October at 05:45 UTC).
The advisor goes first, and says what it can't see
Run the security review on Fabrikam HQ.
tenant_advisor(tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e", area="identity") → identity: 7 fail · 6 pass · 0 unknown · 1 set aside · Secure Score 61.2%
The worst identity finding is critical: 2 of 7 admins have no MFA method registered, Ana Dias and Diego Yilmaz. Security defaults are off, which passes, because Conditional Access replaces them. "Users can create tenants" is still allowed, set aside by an admin until 14 November.
Two details in that answer are the prompt doing its job. "Unknown" is zero, so no check was skipped for a missing licence or permission; when it isn't, Claude reports the reason and never counts it as passing. And the snoozed finding is reported as set aside, with its date, not as fixed. Then the brief moves to the raw domains, because a check that counts policies cannot tell what they enforce.
The policy that reads "on" and enforces nothing
What do the Conditional Access policies enforce?
get_metrics(domain="auth_methods", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e")
→ 13 rows (8 methods, 5 policies)
get_metrics(domain="mfa_registration", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"mfa_registered": false}, group_by="user_type")
→ member 6 · guest 1
Three policies enforce: MFA for Global Administrators, a legacy authentication block, and a compliant device for Exchange. "Require MFA for all users" is report-only, and "Legacy pilot — retired" is off. Nothing requires MFA from ordinary users, and 7 accounts (6 members, 1 guest) have no method registered.
Graph gives a conditionalAccessPolicy
one of three states: enabled, disabled or
enabledForReportingButNotEnforced. The third one is report-only.
Microsoft's report-only
documentation is clear about what it does: the policy is evaluated, the
result lands in the Report-only tab of the sign-in log, and users are not
prompted for MFA or blocked. It is a test mode. Here it has been testing since
the policy was last modified on 25 September.
The demo also shows why this survives reviews. With security defaults off, Microsoft expects Conditional Access to carry MFA for every user. A baseline that sees "security defaults: disabled, CA policies present" passes. Only reading the state of each policy shows the gap.
Methods allowed, strength demanded
Can the admins actually satisfy the admin policy?
get_metrics(domain="mfa_registration", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"is_admin": true},
columns=["display_name", "mfa_registered", "passwordless_capable", "methods"])
→ 7 rows
The policy demands phishing-resistant MFA, but no admin has a passkey or certificate registered: their methods are Authenticator push, phone and software codes. Bruno Keller, the Global Administrator it covers, has push only. Ana Dias, the other one, is excluded as break glass and has no method at all.
The built-in Phishing-resistant MFA strength accepts Windows Hello for Business or platform credential, FIDO2 security keys and multifactor certificate-based authentication. In this tenant's methods policy, passkeys are enabled for the IT department group only and certificate-based auth is off. So the strongest policy in the tenant asks for something most admins can't present; check Bruno's sign-in log before assuming he gets in the way you think. The same rows show the opposite problem: SMS is enabled for all users and usable for sign-in, which Microsoft's SMS sign-in page says to enable only for frontline workers.
Baseline settings: warn, ok, and snoozed
Which tenant baseline settings are flagged?
get_metrics(domain="tenant_baseline", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"status": "warn"})
→ 8 rows
Users can create tenants and register applications; everyone, guests included, can invite guests; joining a device needs no MFA; Entra Connect last synced 9 days ago; a delegated admin partner can administer the tenant. LAPS is enabled and risky app consent is blocked.
The tenant creation row is the one people shrug at. Microsoft's default user permissions page spells it out: whoever creates a tenant becomes its Global Administrator, and the new tenant inherits none of your settings. Setting "Restrict non-admin users from creating tenants" to Yes leaves it to the Tenant Creator role. The admin who snoozed it here wrote "Planned for next month's clean-up"; the review keeps it visible until then.
One name on four lists
The last steps read Entra's own recommendations, third-party apps, forwarding rules and risk detections. Each list is short. The overlap is the finding.
Risky apps, mail leaving the company, open risk detections?
get_metrics(domain="oauth_apps", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"first_party": false})
→ 6 rows (2 with risky scopes)
get_metrics(domain="forwarding_rules", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"external": true})
→ 2 rows (1 enabled)
get_metrics(domain="risk_detections", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"risk_state": ["atRisk", "confirmedCompromised"]})
→ 4 rows (2 high)
get_metrics(domain="entra_recommendations", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e")
→ 6 rows (5 active)
get_metrics(domain="pim", tenant_id="ce48e256-019f-4f8c-b951-0c52d1d65b8e",
where={"permanent": true, "privileged": true}, group_by="principal_type")
→ User 6 · Service principal 3
Diego Yilmaz is an admin with no MFA, has a high leaked-credentials detection still at risk, forwards his mail to a personal Gmail address, and can reach Global Administrator through the HR Sync Connector app. He goes first.
The two risky apps are both unverified: Mail Sync Pro, admin-consented with
Mail.ReadWrite and Mail.Send and no publisher, and PDF
Converter Plus, with Files.ReadWrite.All from 9 user consents. Entra's
two high recommendations are the missing sign-in risk and user risk policies,
which matches the policy rows: none of the five has a risk condition.
| Risk | How many | Fix |
|---|---|---|
| Admin with leaked credentials, no MFA, external forwarding | 1 person | Reset and revoke sessions, remove the rule, register MFA |
| MFA for all users is report-only | 7 accounts with no method | Read the Report-only results, then switch it to On |
| Admin policy demands a strength no admin holds | 0 of 7 admins hold one | Enable passkeys for admins and register them |
| Standing privileged access | 9 (6 users, 3 service principals) | Make the users PIM-eligible; review the apps |
| Unverified apps with risky scopes | 2 apps | Revoke the grants or confirm the vendor |
| No sign-in or user risk policy | 2 recommendations | Create both, in report-only for a week, then On |
| SMS usable for sign-in, all users | 1 method | Untick "Use for sign-in" or scope it to frontline staff |
- Analyze Conditional Access policy impact — report-only policies are evaluated and logged; users aren't prompted for MFA or blocked.
- conditionalAccessPolicy resource type — the three states, including enabledForReportingButNotEnforced.
- Security defaults in Microsoft Entra ID — turning them off means Conditional Access has to carry the same protections.
- Conditional Access authentication strengths — which methods the built-in phishing-resistant strength accepts.
- SMS-based authentication — SMS as a first factor is meant for frontline workers only.
- Default user permissions in Microsoft Entra ID — a user who creates a tenant becomes its Global Administrator; nothing is inherited.
Run the security review on your own tenant: it is in the connector's tools menu.
Add the connector →