Data engineering in Hyderabad
A common mandate for a Hyderabad capability centre is to become the parent company's data hub: pull together sales, operations and finance data from systems in several countries, and publish the numbers that leadership reads. The difficulty is less technical than it sounds. Each source defines things differently, some systems are decades old, and when a figure looks wrong, nobody can trace where it came from.
We build pipelines with lineage and freshness treated as requirements. Sources are pulled on a schedule into the warehouse your parent has standardised on, or into ClickHouse or Postgres where the choice is open. Transformations and definitions are versioned in dbt, large sources are summarised where they live rather than copied wholesale, and every published number can be traced back to its source rows. Freshness is monitored and shown next to the figure.
Hyderabad's technology economy grew around HITEC City, set up in 1998, and spread west through Gachibowli and the Financial District. It is where Microsoft runs one of its largest research and development centres outside Redmond and where Amazon opened what it described as its largest campus building, in 2019. JLL counts more than 355 global capability centres in the city, the in-house engineering and operations offices of foreign companies, employing over 200,000 people, which makes it the second-largest GCC hub in India. Those centres set the local price of an experienced engineer, and every other employer in the city, startups included, hires against them.
The second pillar is life sciences. Genome Valley, the pharmaceutical manufacturers on the city's edge and a fast-growing set of pharma GCCs make Hyderabad unusual: according to the same JLL research, nine of the world's top 18 pharma, biotech and medical device companies run capability centres here. Software in this world is about records people can trust. Quality teams need to see who changed a record, when and why. Approvals need a named person behind them. A batch, a deviation or a sample has to be traceable from start to finish. Much of that work still lives in shared drives and spreadsheets that have outgrown the job.
So the buyers here are usually one of three. GCC leaders with a mandate from headquarters and a hiring pipeline that moves slower than the roadmap. Quality, clinical and commercial teams in pharma who need a tool built properly and documented as it is built. And the city's own startups and established firms, who compete with those GCCs for every engineer and need senior people on a piece of work long before a hire can join.
We are in Bengaluru and share your time zone, so the working day is the same on both sides. Most of the work happens in your repository, your ticketing system and a shared chat channel, which is how most GCCs prefer to run anyway. You talk directly to the engineer who writes the code, and the runbooks and architecture notes are written as we go, so your permanent team inherits them.
We are the right fit for the proof of concept that wins your GCC its headcount, a funded build that needs senior engineers before hiring can staff it, or specialist work alongside your team. Start with a fixed-price two-week piece and judge us on what ships. Agencies and IT services firms can bring us in white-label under their name.
Four ways this arrives.
Two definitions of the same metric, computed in two places. We pick one, write it as a tested model in version control, and point every report at it, so the argument moves from whose number is right to what the definition should be.
A job reports success while the report still shows yesterday. We give every table a written freshness guarantee, monitor what the reader actually sees rather than the job that feeds it, and alert a person when the guarantee is missed.
A CRM, an ERP or a work-order tool reachable only through its API. We build the sync with retries, rate limits and normalisation at the boundary, and keep append-only snapshots so last Tuesday is a query rather than a restore.
Analytical queries running against the application database at the worst possible moment. We move them to a columnar store, keep Postgres for transactions, and feed one from the other by change data capture or scheduled extracts, whichever the freshness requirement calls for.
A nightly work-order extract into append-only ClickHouse snapshots, with a verifier that reads the live Tableau dashboard to prove it shows the newest data.
Read the write-up →Asked by Hyderabad teams.
How do you work with teams in Hyderabad?+
We are in Bengaluru and share your time zone, so calls, reviews and releases fit inside your normal working day. We work in your tools: Slack, GitHub and Linear, or whichever chat, repositories and ticketing your parent company has standardised on, with access through your own accounts. You talk directly to the engineer writing the code, and the decisions, runbooks and architecture notes are written down in your repository as we go.
Do you build for pharma and life sciences teams?+
Yes. We build the tools quality, clinical and commercial teams use every day: platforms where every change to a record shows who made it, when and why; approvals tied to a named person; field apps for medical representatives that work without signal; batch- and expiry-aware ordering for distributors; and document search that cites the exact source passage. The design documentation is written as we build, so your teams can see exactly what the system does.
How does your rate compare to hiring in Hyderabad?+
PayScale puts a mid-career software engineer in Hyderabad at about ₹14 lakh a year. Our published rate is $35 an hour, with a $5,000 minimum. That buys a senior engineer who writes the code, works in your Slack, GitHub and Linear, and starts within days, with no recruiting round, notice period or employment overhead. Begin with a fixed-price two-week piece at $2,800 and judge us on what ships.
Can you build the proof of concept our GCC needs to win headcount?+
Yes. We build one workflow as a small, real product inside your environment, usually in Next.js and Postgres, with usage you can report to headquarters. It starts with a fixed-price two-week piece, so the first working version arrives quickly and costs a known amount. The NDA comes before specifics, and the code sits in your repositories from the first commit, so when the headcount is approved your new team extends it.
Do we need a data warehouse yet?+
Maybe not. If your reporting runs comfortably against a Postgres read replica, a warehouse adds a second system to pay for and keep in step. The signs you need one: analytical queries slowing the product, data from several sources that has to be joined, or history you must keep that the application overwrites. We will tell you which side of that line you are on.
Why ClickHouse rather than Snowflake or BigQuery?+
Not always. For steady, high-volume analytical work, ClickHouse is fast and cheap to run, self-hosted or on its managed cloud, and we run it in production. Snowflake and BigQuery ask less of you operationally and suit ad hoc querying across a large team. If nobody on your side wants to operate a database, we will recommend the managed option even when the invoice is larger.
What does a freshness guarantee actually mean?+
A written statement per table or report, such as "never more than an hour behind" or "reflects yesterday's close by 07:00". Each one has a check that measures it and an alert that fires when it is missed. Without it, stale data looks exactly like correct data until somebody makes a decision on it.
Can you work with the tools we already have?+
Usually, yes. If you already run dbt, Airflow, Fivetran or a BI tool your team knows, we build inside it. We suggest replacing a tool only when it is the cause of the problem you hired us for, and we show you the evidence before asking you to fund a migration.
How do you handle personal data in the warehouse?+
We decide what should reach the warehouse at all before building the pipeline. Identifiers can be hashed or tokenised at ingestion, sensitive columns restricted by role, and retention enforced by the pipeline itself. Every field is documented from source to dashboard, so anyone can see where a number came from and who can read it.