Senior Databricks Data Engineer.
This mandate is run by SYMPHONI HR, a Mumbai-based executive search firm, est. 2003. Applications are email-verified and reach the search team running the role — confidentially, always.
About the organisation.
Our client is a prominent global leader in the computer software industry, known for delivering innovative digital solutions and advanced analytics services. They foster a culture of continuous learning and technical excellence, supporting large-scale enterprise transformations for clients worldwide.
About the role.
We are looking for a highly skilled Senior Databricks Data Engineer to design, develop, and optimize scalable data platforms using the Databricks Lakehouse architecture. You will play a pivotal role in building robust ETL/ELT pipelines, integrating AI/ML workflows, and ensuring high-performance data processing for enterprise-level analytics initiatives.
What you will do.
- Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and cloud-native services.
- Build and optimize complex ETL/ELT workflows to handle large-scale structured and unstructured data.
- Develop sophisticated data models and implement rigorous data quality, validation, and governance frameworks.
- Optimize Spark jobs and Databricks workloads to ensure maximum performance, scalability, and cost efficiency.
- Implement robust security controls, access management, and data governance policies using Unity Catalog.
- Collaborate with cross-functional business, analytics, and AI/ML teams to deliver trusted data products.
- Monitor, troubleshoot, and resolve data pipeline issues to ensure high availability and reliability.
- Support CI/CD, DevOps, and infrastructure automation practices to streamline deployment cycles.
What you bring.
- 8–10 years of professional experience in Data Engineering and Data Warehousing.
- Expertise in Databricks, Apache Spark, PySpark, and Spark SQL.
- Strong proficiency in Python and SQL for data manipulation and pipeline development.
- Hands-on experience with Delta Lake, Unity Catalog, and modern ETL/ELT frameworks.
- Proven ability to integrate AI/ML pipelines and implement MLOps best practices.
- Experience with data governance tools, data lineage, and quality frameworks like Great Expectations or Deequ.
- Familiarity with Snowflake, Kafka, and event streaming architectures.
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Other open roles.
Delivery Manager — Hyderabad · Hybrid
Quantitative Analyst, Securitized Products — Mumbai · Hybrid
Adobe Cloud Enterprise Architect — Bengaluru · Hybrid