Tech Lead Data - Databricks

Accountabilities

• Provide technical leadership to Data Engineering teams, supporting solution design, code reviews, technical decisions, and the resolution of delivery blockers.

• Collaborate with the Data Architect to define the technical vision, coding standards, data modeling conventions, and best practices for the Lakehouse environment using Delta Lake and the Medallion architecture.

• Design, develop, and oversee data pipelines in Databricks using PySpark and Spark SQL across Bronze, Silver, and Gold layers, ensuring scalability, reliability, security, and governance.

• Establish and reinforce DataOps practices, including Git version control, CI/CD, automated unit and integration testing, data quality validation, and technical documentation.

• Optimize pipeline performance, including partitioning strategies, Spark tuning, cloud cost management, and operational reliability through monitoring, observability, and alerting.

• Ensure non-functional requirements are met, particularly regarding access control, security, and data policies across Azure and Databricks.

• Partner with business and analytics stakeholders to translate requirements into scalable technical solutions and realistic delivery plans.

• Mentor engineers, promote engineering excellence, and develop reusable templates, playbooks, and technical references to improve team consistency and productivity.

Requirements:

• Strong experience in Data Engineering projects, including previous experience providing technical and operational leadership as a Tech Lead or Senior Engineer.

• Extensive hands-on experience with Databricks, PySpark, and Spark SQL, particularly in designing and implementing batch data pipelines, with working knowledge of streaming when required.

• Deep knowledge of Delta Lake and practical experience implementing the Medallion architecture across Bronze, Silver, and Gold layers.

• Solid experience with Azure data services, including Azure Databricks, Azure Data Lake Storage (ADLS), and integration with identity and security services such as Azure Active Directory.

• Strong SQL and Python skills, along with a solid understanding of dimensional data modeling, consumption-oriented models, and schema and data contract best practices.

• Experience implementing DataOps practices, including Git, CI/CD for data pipelines, automated testing, and clear technical documentation.

• Proven ability to review architecture and code, troubleshoot performance and scalability issues, and guide engineering teams toward effective solutions.

• Advanced English proficiency for collaboration with stakeholders and the preparation of technical documentation.

• Nice to have:Experience with Unity Catalog, Databricks Workflows and Repos, Delta Live Tables, data quality frameworks such as Great Expectations or Soda, and dbt on the Lakehouse.

• Additional advantages:Familiarity with orchestration tools such as Airflow, observability practices involving metrics, logs, and alerts, and change data capture or streaming technologies such as Event Hubs and Kafka.

• Consulting experience, exposure to international clients, and Databricks or Azure certifications are also valued.

Benefits:

• Health and dental insurance.

• Meal and food allowances.

• Childcare assistance.

• Extended paternity leave.

• Access to gym memberships and health and wellness professionals through Wellhub (Gympass) and TotalPass partnerships.

• Profit Sharing and Results Participation (PLR).

• Life insurance.

• Access to a continuous learning platform and professional development resources.

• Discount club with partner offers.

• Free online resources dedicated to physical, mental, and overall well-being.

• Pregnancy and responsible parenting courses.

• Partnerships with online learning platforms.

• Access to a language-learning platform.

• Additional benefits and well-being initiatives.