Data Engineer Interview Questions (UK-Focused)

A strong data engineer interview answer should cover architecture, not just tools. I typically explain end-to-end flow: ingestion from source systems, transformation with dbt models, and serving into a warehouse or lakehouse. For orchestration, I describe how I’d structure Airflow DAGs with retries, timeouts, and dependency management so failures are contained and recoverable. I also bring in reliability patterns such as idempotent writes, partitioning strategy, and deterministic merge logic, because these directly impact incident frequency and operational load. Finally, I include operational targets—like a freshness SLA of under 5 minutes for incremental datasets and a failure rate below 0.2%—and show how observability is implemented using metrics and dashboards (e.g., Grafana with Prometheus).

In interviews, the difference between batch and streaming is rarely the tool—it’s the correctness model and the operational burden. I explain that batch pipelines are easier to reason about and are ideal when minutes-to-hours latency is acceptable for reporting, backfills, and model training. Streaming is chosen when product requirements demand immediate reaction—sub-second or low-second latency—such as fraud signals or live operational dashboards. I describe Kafka as the common event backbone, then detail how I handle duplicates and ordering using event keys, deduplication windows, and state management. Where needed, I mention Spark Structured Streaming or Flink for processing, and I tie this to the semantics of at-least-once vs exactly-once delivery in practice. A strong answer also notes hybrid architectures, where batch recomputation handles eventual consistency and streaming handles immediacy, balancing cost and complexity.

Data debugging is a core interview theme, and the best responses are structured and evidence-led. I explain how I start with a precise reproduction: which dataset, which feature/column, what time window, and what expected vs actual outcome looks like. Then I trace lineage across the system—back from the warehouse model through dbt transformations and into ingestion tasks in Airflow, ultimately to the source extract. I mention how I validate assumptions using row-count checks, null ratios, uniqueness constraints on business keys, and distribution comparisons for critical metrics. I also call out common real-world culprits like schema drift from upstream systems, timezone or timestamp conversion mistakes, late-arriving events in Kafka topics, or join cardinality changes. To prevent recurrence, I add targeted tests with dbt and Great Expectations, then I track KPIs such as time to detect and time to recover to show operational improvement.

Interviewers look for data engineers who treat quality as engineering discipline, not a manual step. I describe a layered testing approach: schema and type validations early, business rule checks in transformations, and end-to-end freshness checks at the pipeline boundary. In dbt, I build schema tests (not null, unique, accepted values) and relationship tests to catch broken joins; for richer assertions I use Great Expectations. I explain how I reduce runtime cost by running checks on incremental partitions, applying sampling for expensive validations, and using fail-fast vs warn strategies depending on business criticality. I also discuss contract testing between producers and consumers, so schema changes fail early instead of silently corrupting downstream dashboards. Finally, I mention how results feed monitoring and alerting—so engineers get actionable alerts with clear severity, and stakeholders trust that data failures are detected quickly and explained transparently.