Build vs buy: your PowerShell scripts vs a reporting platform

Every demo of a Microsoft 365 reporting tool reaches the same moment. The admin on the call leans back and says: I could build that with PowerShell.

They are right. That is the part most vendors handle badly — arguing with a true statement makes you sound like you are selling something. You can absolutely build it. The interesting question was never whether the script works; it is what the script costs on its second birthday. So here are the objections in the order they actually come up, answered without the sales voice.

Can I build this with PowerShell?

Yes, and faster than you think. A licence-waste report against one tenant is maybe forty lines: connect, pull subscribedSku, pull users with their sign-in activity, join, export to CSV. It runs on your laptop, it produces a real number, and the first time you show a client that number you look excellent.

Nothing below contradicts that. Version one genuinely works. The problem is that version one is not what you end up maintaining.

So what actually breaks?

Not the logic. The logic you wrote is fine two years later. What breaks is every assumption the script makes about the platform it is standing on, and it breaks on a schedule that has nothing to do with your roadmap.

Month 14 is not hypothetical. Microsoft's migration guidance opens by stating that the Azure AD, Azure AD Preview and MSOnline PowerShell modules are deprecated, and it is blunt about the consequences: "Scripts written in Azure AD PowerShell won't automatically work with Microsoft Graph PowerShell." Not the cmdlet names alone — renamed modules, changed parameters, different filter syntax, and different output objects, which is the one that bites silently, because a property that quietly became null does not throw an error. It produces a report full of blanks that looks like good news.

What changes when it is twenty tenants instead of one?

Three things, and none of them are a loop.

Authentication stops being interactive. Unattended runs need certificate-based app-only auth, and for a multi-tenant app Microsoft is explicit that you generate an admin consent URL for each customer tenant and assign role permissions in each customer tenant. That is per-client onboarding work that exists forever, plus a certificate to guard — and the self-signed example in those same instructions is created with -NotAfter (Get-Date).AddYears(1), so the clock starts the day you set it up.

Then throttling arrives. Graph returns HTTP 429 with a Retry-After header, and the guidance contains the detail that catches home-grown clients: Microsoft "continues to log resource usage while a client is being throttled", so a naive retry loop makes things worse rather than recovering. Reach for $batch to be efficient and you meet the sharper trap — requests inside a batch are evaluated individually, so a batch can come back 200 OK with throttled requests inside it, and those are not retried automatically. Your script reports success. Your data has holes.

Third: the work becomes a service. Twenty tenants on a schedule means a job runner, a place for the output to live, alerting when a run fails, and somebody who is not you able to fix it at 3am. That is the point where the script stopped being a script, which is the same multi-tenant threshold that turns browser tabs into a real problem.

Why do the reports come back anonymised?

Because they are supposed to. In Microsoft 365 usage reports, user information such as usernames, groups and sites is concealed; actual values aren't displayed — hidden by default since 1 September 2021 for every tenant. Your join key is a hash. The script does not fail; it produces an adoption report about people it cannot name, and somebody has to notice.

Handling it properly means detecting the concealment, telling the reader that is what happened, and knowing the setting lives at organisation level and follows you everywhere the data goes. That is three behaviours nobody scopes at the start, for one checkbox in Org settings.

Can't I just query last quarter when the client asks?

No, and this is the failure that has no workaround. Entra keeps audit and sign-in logs for seven days on Free and 30 days on P1 or P2, and the same page is unambiguous that log retention changes aren't retroactive — buying P1 today does not resurrect last month. Usage data reaches 180 days and stops.

So a year-over-year comparison is not a query anyone can write. It is something you stored, or it does not exist. A script that emails a CSV and forgets is architecturally incapable of ever answering the question, no matter how well it is written, and you find that out on the call where it matters.

What does year two actually cost?

The last row is the one that decides most build-vs-buy arguments, and it never appears in the estimate. A reporting script written by one person, run from one laptop, understood by one brain, is a single point of failure that also happens to be on the payroll. It is the same arithmetic as the hours spent assembling reports by hand: the cost is real, recurring, and invisible because it is denominated in evenings rather than euros.

When is the script still the right answer?

Often. Genuinely. Write the script when the question is a one-off — an audit-day answer, a migration check, something you will run twice and delete. Write it when the thing you need is so specific to one client that no product will ever cover it. And write it when you have one or two tenants, because the per-tenant overheads that dominate this article simply are not there yet.

Buy when the answer has to arrive every month whether or not you are having a good month, across enough tenants that per-tenant setup is real work, with history nobody can query retroactively, and it needs to keep working while you are asleep. That is not a claim about code quality. It is a claim about who carries the maintenance, and it is the whole difference between the two columns above. GraphPaaS exists on the buy side of that line: the same Graph surfaces you would have called yourself, collected per tenant on a schedule, retained past the window Microsoft keeps them in — and when a module gets deprecated, that migration is somebody else's Tuesday.