Job Title: Databricks - Sr. Consultant Role Overview: As a Senior Consultant, you will lead the design and optimization of high-performance data processing frameworks on Azure Databricks. You will be responsible for advanced Spark tuning, implementing robust data governance through Unity Catalog, and driving the transition toward a modern Medallion architecture. Experience: • 5–7 years of experience in data engineering, with deep expertise in Azure Databricks and Apache Spark. • Advanced experience with Delta Lake and Medallion Architecture (Bronze/Silver/Gold). • Proven track record in Spark Performance Tuning (handling data skew, shuffle tuning, etc.). • Strong experience with Unity Catalog for data governance and security. Key Responsibilities: Technical Skills: • Design and implement scalable Medallion Architectures using Delta Live Tables (DLT) and Databricks. • Lead the optimization of complex Spark workloads to improve execution speed and reduce cloud compute costs. • Implement enterprise-grade data governance and access control using Unity Catalog. • Architect automated CI/CD pipelines for Databricks using Databricks Asset Bundles (DABs) or Terraform. • Design sophisticated data integration patterns between Databricks, Azure Synapse, and various RDBMS. • Act as a technical mentor to junior engineers, conducting code reviews and enforcing best practices. Soft Skills: • Strong leadership and ability to drive technical delivery. • Advanced problem-solving skills for complex architectural challenges. • Excellent communication skills for interacting with stakeholders and technical leads. Preferred Qualifications: • Bachelor’s or Master’s degree in a technical field. • Databricks Certified Data Engineer Professional is highly desirable. • Experience with DataOps and Infrastructure as Code (Terraform). Job Details: • Location: Mumbai (Hybrid)