Principal Data Engineer.
This mandate is run by SYMPHONI HR, a Mumbai-based executive search firm, est. 2003. Applications are email-verified and reach the search team running the role — confidentially, always.
About the organisation.
Our client is a leading global computer software organization known for delivering innovative data-driven solutions. They operate at a significant scale, supporting complex enterprise environments with a focus on cutting-edge technology and engineering excellence. The company fosters a collaborative culture that prioritizes technical growth and high-impact problem solving.
About the role.
We are seeking a highly experienced Principal Data Engineer to spearhead the technical direction of a next-generation data platform. In this hands-on leadership role, you will design sophisticated lakehouse architectures, drive the migration of legacy workloads, and establish engineering standards to ensure high-performance, secure, and scalable data delivery across the organization.
What you will do.
- Own the end-to-end design of the Databricks lakehouse, including ingestion, medallion architecture, and Unity Catalog implementation.
- Lead the hands-on migration of legacy Hadoop, Hive, and Spark workloads to Databricks while ensuring data parity.
- Define and enforce engineering standards for SQL, PySpark, and Delta Live Tables.
- Oversee CI/CD, automated testing, and observability frameworks to maintain high system reliability.
- Optimize performance and DBU spend through cluster right-sizing, liquid clustering, and advanced query tuning.
- Implement robust data security measures, including PII protection, encryption, and row/column-level security.
- Lead technical design reviews and author architecture decision records.
- Mentor senior and mid-level engineers to elevate overall engineering quality and technical strategy.
What you bring.
- 12–16 years of experience in data or software engineering, with at least 5 years in a technical lead, staff, or principal capacity.
- Expert-level proficiency in SQL with a focus on query tuning for massive datasets.
- Deep experience with Apache Spark, including both PySpark and Scala.
- Proven track record of managing Databricks in production, including Unity Catalog, Workflows, and cost management.
- Strong expertise in large-scale batch and streaming pipeline design using Structured Streaming, Delta Live Tables, and Kafka.
- Proficiency in data modeling techniques such as Dimensional, Medallion, and Data Vault.
- Hands-on experience with CI/CD tools including Git, Terraform, and Databricks Asset Bundles.
- Graduate degree in Computer Science, Data Science, or a related field.
Other open roles.
Senior Software Engineer — Pune · Hybrid
AI Developer — Pune · Hybrid
Senior ServiceNow Software Engineer — Pune · Hybrid