Data, Engineer

Master works is seeking a Data Engineer to join our team and play a key role in developing, operating, and maintaining the organization’s data pipelines and data platform infrastructure. The role will be responsible for ensuring reliable data processing, platform availability, security, performance, and compliance with pipeline SLAs.

Key Responsibilities

• Design, develop, and maintain ELT/ETL pipelines to produce clean, structured, and reliable datasets.
• Build and maintain data orchestration workflows usingApache Airflow.
• Develop and test data transformations usingSQL, Python, and PySpark.
• Monitor pipeline execution, troubleshoot processing issues, and proactively address SLA risks.
• Administer and maintainlakehouse, object storage, Apache Spark, and Trinoenvironments.
• Build and maintain CI/CD pipelines usingBitbucket and Jenkins.
• Manage user access, roles, permissions, and resource quotas across data platforms.
• Monitor platform performance, respond to incidents, and perform system upgrades and patching.
• Optimize storage configurations, including partitioning, data formats, and data retention.
• Collaborate withData Integration, Data Engineering, BI, and Analytics Engineeringteams to support data initiatives.
• Maintain infrastructure documentation, operational procedures, and technical runbooks.
• Participate in capacity planning, platform optimization, and cost-management initiatives.
• Provide coverage and support for Data Integration activities during team absences or periods of increased workload.

Requirements

Qualifications

• Bachelor’s degree inComputer Science, Information Systems, Software Engineering, or a related field.
• Minimum5 years of relevant experiencein Data Engineering or a related field.
• Strong proficiency inSQL, Python, PySpark, Apache Spark, Trino, and Apache Airflow.
• Hands-on experience withlakehouse architecture, object storage, and ELT/ETL development.
• Practical experience withBitbucket, Jenkins, and CI/CD practices.
• Knowledge ofaccess management, platform monitoring, incident response, and performance optimization.
• Experience working with enterprise data platforms and supporting production data environments.

Key Skills & Experience

• ELT/ETL Pipeline Development
• Apache Airflow Orchestration
• SQL, Python & PySpark
• Apache Spark & Trino
• Lakehouse Architecture & Object Storage
• CI/CD using Bitbucket & Jenkins
• Data Platform Administration
• Access, Role, Permission & Quota Management
• Pipeline Monitoring & Incident Resolution
• Performance & Storage Optimization

Location: Client Site