FM360
FM360 is the platform a healthcare facilities team runs its maintenance operation on. We built it under contract for Avacend Labs, starting in 2024 with a set of reports and growing it into a web app, a mobile app for supervisors, a phone page for technicians, and the backend under all three. It has been in production since early 2025.
The brief
A healthcare facilities team runs its maintenance operation out of WorxHub, a third-party work-order system reachable only through its API. Supervisors needed to see what was overdue, what kept coming back, and whether preventive maintenance was keeping up — without exporting spreadsheets by hand. Over two years the answer grew from charts into a platform: a web app for supervisors and managers, a mobile app for supervisors on the floor, a phone page for technicians, one backend under all three, and a set of analysis tools beside it.
Where it started
It began in 2024 as a plotting script that turned monthly exports into charts. By the end of that year a small service was writing work-order data into ClickHouse every day. In February 2025 it became a FastAPI service on EC2, deployed from GitHub Actions: nightly extracts, alert emails, Box uploads and Tableau refreshes. In March 2026 we folded that service into the platform backend, where the pipeline lives now.
What it costs us. For thirteen months there were two Python services talking to the same ClickHouse, each with its own WorxHub and Tableau client, and a fix in one had to be remembered in the other. The migration moved the pipeline across module by module rather than in one cut-over, which kept both running until the last piece landed.
Three surfaces, one backend
The web app, at grady360.org, is where supervisors and managers work. It is a Next.js application scoped per organisation, with five dashboards — Lookahead, PM, Shop Scorecard, Repeat Category and Weekly Pulse — a gallery of Tableau reports rendered in place, and admin screens for users, apps, access, usage analytics and login alerts. An administrator can view the app as any other user, to see exactly what that user sees.
Supervisors also carry a mobile app, built in React Native with Expo in spring 2025. It opens on their own operation: work-order volume and SLA compliance, planned against unplanned work, what is overdue, in progress or at risk of breaching its SLA, and what was completed recently. Beside that sit a filtered work-order list and a per-worker view of completions and delays. Sign-in is the same magic link as the web: the email opens a small page that hands the token to the app through its own URL scheme, and the session is kept in the device’s secure storage.
Technicians get a phone-first page instead, currently in pilot. Pick your name, enter the work-order number, and record two to fifteen seconds about what you found. Whisper transcribes it, and it lands as a signal against that work order, next to how often that room has come up recently.
All three talk to a multi-tenant FastAPI backend. PostgreSQL holds people, organisations, apps and access. ClickHouse holds the work-order history. DragonflyDB caches the heavy analytical queries, invalidated when a new extraction lands. The backend runs on Railway.
What it costs us. Two databases with different jobs is the right split and a permanent tax: every feature has to decide which side it lives on, and every cached ClickHouse result has to survive serialisation — see the first entry under what we would do differently. In the web app, session state persists in cookies so the server render matches the client, which means sign-out has to tear that state down deliberately rather than waiting for it to expire.
The pipeline
Every hour, the backend pulls HVAC work orders from WorxHub, detects new and changed high-priority ones, and sends an alert email and a Teams message. Nightly, a full extract appends a daily snapshot to ClickHouse, uploads the file to Box, and refreshes the Tableau Cloud data sources directly. Each morning, every supervisor gets a personalised digest: priorities, SLA completion, status changes and what is overdue. Weekly, a snapshot goes to Google Drive as a spreadsheet.
The snapshots are append-only, keyed by extraction date. A work order’s
current state is argMax over its history, which means “what did this look
like last Tuesday” is a query rather than a restore.
What it costs us. The table only grows, and every “current state” read pays for the history. That is most of why the cache exists. And the whole run depends on one OAuth token from WorxHub: a single transient 500 on that endpoint once killed a nightly extract, so token requests now retry for minutes rather than once.
Proving the dashboard is current
A refresh job can report success while the dashboard still shows yesterday. So after every nightly run, a verifier renders the live Tableau dashboard to an image, reads its “Last Data Set Update” timestamp with OCR, converts it from New York time, and requires an exact match with the newest extraction in ClickHouse. It re-renders every 90 seconds until the two agree or fifteen minutes pass, then reports to Teams and email with the screenshot attached.
An unreadable timestamp counts as a failure, never a silent pass. And the verifier can never break the pipeline run it is checking.
What it costs us. Tesseract ships in the production image, and each night waits up to fifteen minutes after the data is already loaded. The first version also raised a false stale alert, because Tableau’s Bridge jobs report completion when they are merely assigned. The verifier now polls to the real finish code before it starts looking.
From clustering experiment to production
Repeat work — the same fault logged again and again in the same room — took the most attempts to get right.
The first was an experiment: 18,000 work-order descriptions embedded with a small sentence-transformers model, stored in Pinecone, clustered with UMAP and HDBSCAN, with a random-forest estimate of how long each kind of job takes. The second was a working analysis tool: around 33,000 HVAC corrective work orders, reduced with PCA and UMAP and clustered with HDBSCAN, explored in a Streamlit dashboard. Its refresh endpoint rebuilds the embeddings and clusters in the background — ten to fifteen minutes — and swaps the new snapshot in only once it is complete, so the dashboard never shows half a rebuild.
Production, the Repeat Category dashboard, kept the embedding model and
dropped the rest. Clustering there has to be incremental: each night, new
descriptions join the nearest existing cluster when cosine similarity is 0.85
or higher, or start a new one, without reclustering everything that came
before. It runs as the last step of the pipeline and writes to its own
ReplacingMergeTree table, so the dashboard reads precomputed cluster IDs. A
second mode, fuzzy, uses ClickHouse’s ngramSimHash; descriptions too short
for that are grouped by exact match.
What it costs us. HDBSCAN adapts to how dense the data is; a greedy threshold does not, and production gave that up to get nightly incremental updates. Greedy clustering also depends on the order descriptions arrive in, and one global threshold decides what counts as the same problem across every trade. The model brought native libraries into the Railway image that the default runtime did not have, which took a build change to fix.
A pulse for the Monday huddle
Weekly Pulse summarises the preventive-maintenance week that ended at Sunday midnight, ready for the Monday huddle. A deterministic layer computes the metrics and runs three detectors — repeat location, temporal pattern, and a single asset consuming repeated attention — and every finding must carry the work-order numbers behind it. Findings persist across weeks as signals with an audited status, so something raised last Monday is still visible this one.
Claude writes the prose, and only the prose. Its output is checked against a JSON schema, may only cite work orders that are in the facts it was given, and may only state numbers from a whitelist. If the API is missing or keeps failing, deterministic templates write the pulse instead. A week is never without one.
What it costs us. The guardrails reject output, and a rejected run falls back to the plainer template — some weeks read flatter than others. The pulse is also preventive maintenance only, by decision with the client, so corrective work is invisible to it for now.
Signing in
Magic links, email codes and SMS codes through Twilio, with an alert when an account signs in somewhere new. Tableau reports open inside the web app without a second login: the backend issues a signed pass for the individual viewer on every open, valid for about five minutes, and only if that user has been granted that report.
What it costs us. Every sign-in method is its own surface to get wrong. The first magic-link endpoint told a caller whether an account existed; it now answers every request the same way. And Tableau refuses the pass unless the user’s email matches a Tableau Cloud account exactly.
The stack
What we would do differently
Our cache handed back strings where the code expected dates.
The DragonflyDB cache serialises every value to JSON, so dates come back as ISO strings and tuples as lists, but only on a cache hit. The first request with a given filter worked; the second crashed the Repeat Category dashboard with a TypeError. A shared date helper at every read fixed it. The lesson outlived the bug: a cache is part of the data contract, not an optimisation added later. We now test the second request as well as the first, and make a cache return the types it was given.
We compared ISO 8601 timestamps as plain strings.
WorxHub sends timestamps with a T between the date and the time; the database stores them with a space. The hourly check compared the two as raw text, so every high-priority work order looked changed on every run, and supervisors got a false alert email every hour. Normalising timestamps, booleans and numbers at the integration boundary fixed it. On any third-party API integration, that normalisation layer is now the first thing we write, before the first comparison.
We believed Tableau when it said the refresh had worked.
Tableau Cloud can report a data-source refresh as successful while the dashboard still shows yesterday's numbers, and a Tableau Bridge job reports completion when it has only been assigned. We trusted the job status until a stale dashboard went unnoticed. The verifier now reads the timestamp off the rendered dashboard itself. For any reporting pipeline we build, we monitor what the reader sees, not the job that feeds it.
We put this section on every case study. If a write-up has no version of it, either the project was trivial or the write-up is an advertisement.