Data engineering in Perth
Getting data out of an industrial site without creating a way back in is the core data engineering problem in Perth. Historians hold years of sensor readings, maintenance systems hold work history, and planning lives in spreadsheets, but connecting any of it to analytics has to respect the boundary between the plant network and the business network.
We build pipelines that move data one way, from a replica or export on the business side, into a warehouse you own, with no path back towards the control systems. Sources land once, with checks that fail loudly when a feed stops or changes, and history kept so reports do not quietly move. On FM360 we run hourly syncs into ClickHouse and learned to normalise values at the boundary before comparing anything, which matters doubly with industrial timestamps and units.
Perth runs mines it cannot see. Rio Tinto controls trains, trucks and drills in the Pilbara from its operations centre in Perth, BHP runs much of its Western Australian iron ore logistics from a remote operations centre in the city, and Woodside, Fortescue and Chevron Australia keep their head offices here. Around them sits a large sector of engineering consultancies, contractors and mining equipment, technology and services firms. Most of the software work in the city exists to serve that industry, and it comes with that industry's habits: safety first, careful change control and very little tolerance for downtime.
The line that matters is between operational technology and everything else. Control systems, autonomous haulage and plant automation belong to specialists. The opportunity for outside engineers is on the business side of that line: maintenance and shutdown planning, historian data read out for analysis, contractor and workforce systems for a fly-in fly-out workforce, reporting, and the internal tools that stitch them together. Energy and utility businesses need the same kind of work, and beyond resources the state has its own buyers, from agencies and universities in the city to the wineries of Margaret River selling across the continent.
Hiring is shaped by the resources sector, which competes hard for experienced engineers and can pay for them. Indeed puts the average base salary for a software engineer in Perth at A$100,253, from 59 reported salaries, and a consultancy or scale-up bidding against a mining major for the same person often waits a long time. That is where we fit: senior engineers on a defined piece of work within days, with the code in your repositories from the first commit.
Our four hours of daily overlap land in your afternoon, moved to suit your team. Perth is the closest Australian capital to India's time zone, and neither Western Australia nor India uses daylight saving, so the window sits at the same point in your day all year. Your morning is for your own team and the site calls; the afternoon is for stand-ups, reviews and decisions with the engineer who writes your code.
We are the right fit for defined work on the business side of the operation: data, reporting, maintenance and workforce tools, and contractor systems. Senior engineers start within days, the first piece is a fixed-price two weeks, and we can carry the roadmap through the months before your own hire starts. Agencies can put the work under their own name.
Four ways this arrives.
Two definitions of the same metric, computed in two places. We pick one, write it as a tested model in version control, and point every report at it, so the argument moves from whose number is right to what the definition should be.
A job reports success while the report still shows yesterday. We give every table a written freshness guarantee, monitor what the reader actually sees rather than the job that feeds it, and alert a person when the guarantee is missed.
A CRM, an ERP or a work-order tool reachable only through its API. We build the sync with retries, rate limits and normalisation at the boundary, and keep append-only snapshots so last Tuesday is a query rather than a restore.
Analytical queries running against the application database at the worst possible moment. We move them to a columnar store, keep Postgres for transactions, and feed one from the other by change data capture or scheduled extracts, whichever the freshness requirement calls for.
A nightly work-order extract into append-only ClickHouse snapshots, with a verifier that reads the live Tableau dashboard to prove it shows the newest data.
Read the write-up →Asked by Perth teams.
How do you work with teams in Perth?+
We are in Bengaluru, and four hours of every working day overlap with your afternoon, moved to suit your team. Perth is the closest Australian capital to India's time, and neither side changes its clocks, so the window stays put all year. Stand-ups, code reviews and design calls happen live in it. We work in your tools, whether that is Slack, GitHub or Linear, and every call is with the engineer who writes the code.
Do you build for mining and energy companies?+
Yes, on the business side of the operation. We build pipelines that read historian data out for analysis, maintenance and shutdown planning tools, contractor management platforms, apps that put a FIFO worker's roster, flights and documents in one place, and dense planning views for remote operations teams. We integrate with the maintenance, HR and procurement systems you already run, and start with the problem that costs you most on site.
How does your rate compare to hiring in Perth?+
Indeed puts the average base salary for a software engineer in Perth at around A$100,000, and the resources sector often pays more to win experienced people. Our published rate is $35/hour with a $5,000 minimum, or $5,400 a month for one embedded engineer. There is no recruiting time and no employment overhead, and work can start within days. The first piece is a fixed-price two-week engagement at $2,800, so you judge us on what ships.
What do the first two weeks look like?+
You share the context, with an NDA signed before anything specific. Within two days you are on a 30-minute call with the engineer who would do the work, and within a week you have a written plan. Then comes a fixed-price two-week piece for $2,800: one real thing, such as a maintenance report, a contractor portal screen or a data feed, built in your repositories and reviewed with you in your afternoons.
Do we need a data warehouse yet?+
Maybe not. If your reporting runs comfortably against a Postgres read replica, a warehouse adds a second system to pay for and keep in step. The signs you need one: analytical queries slowing the product, data from several sources that has to be joined, or history you must keep that the application overwrites. We will tell you which side of that line you are on.
Why ClickHouse rather than Snowflake or BigQuery?+
Not always. For steady, high-volume analytical work, ClickHouse is fast and cheap to run, self-hosted or on its managed cloud, and we run it in production. Snowflake and BigQuery ask less of you operationally and suit ad hoc querying across a large team. If nobody on your side wants to operate a database, we will recommend the managed option even when the invoice is larger.
What does a freshness guarantee actually mean?+
A written statement per table or report, such as "never more than an hour behind" or "reflects yesterday's close by 07:00". Each one has a check that measures it and an alert that fires when it is missed. Without it, stale data looks exactly like correct data until somebody makes a decision on it.
Can you work with the tools we already have?+
Usually, yes. If you already run dbt, Airflow, Fivetran or a BI tool your team knows, we build inside it. We suggest replacing a tool only when it is the cause of the problem you hired us for, and we show you the evidence before asking you to fund a migration.
How do you handle personal data in the warehouse?+
We decide what should reach the warehouse at all before building the pipeline. Identifiers can be hashed or tokenised at ingestion, sensitive columns restricted by role, and retention enforced by the pipeline itself. Every field is documented from source to dashboard, so anyone can see where a number came from and who can read it.