Senior Data Engineer

Devsinc is looking for a highly skilledSenior Data Engineer with 4–6 years of professional experienceto design, build, and maintain scalable data pipelines and processing systems that support analytics, data products, and AI/ML capabilities.

The ideal candidate will have strong hands-on experience withETL/ELT pipelines, Python, SQL, Apache Airflow, Apache Spark, and Redis, along with the ability to develop reliable and maintainable workflows for large, complex, and continuously growing datasets. You will collaborate with Product, Business Intelligence, Data Science, and Engineering teams to deliver production-ready datasets and high-performance data solutions.

Responsibilities

• Design, develop, and maintain scalableETL/ELT pipelinesfor data ingestion, transformation, validation, and delivery.
• Build, schedule, and orchestrate production-grade data workflows usingApache Airflow.
• Develop distributed data-processing jobs usingApache Spark.
• Write efficient, reusable, and maintainablePythoncode for data processing, automation, and pipeline development.
• Develop and optimize complexSQL queriesfor data transformation, analysis, and quality validation.
• Design pipelines capable of processing large-scale structured, semi-structured, and unstructured datasets.
• ImplementRedisfor caching, high-performance data access, and data-intensive application requirements.
• Integrate data from APIs, relational databases, files, third-party providers, and other internal and external sources.
• Implement data validation, monitoring, logging, error handling, alerting, and pipeline observability.
• Optimize pipeline performance, data storage, processing time, and infrastructure costs.
• Develop reusable data-ingestion and transformation frameworks instead of one-off scripts.
• Troubleshoot pipeline failures, performance bottlenecks, and data-quality issues to ensure timely resolution.
• Collaborate with BI, Product, Data Science, and Engineering teams to deliver reliable, production-ready datasets.
• Establish and maintain data-engineering standards, technical documentation, and development best practices.

Requirements

• Bachelor’s degree inComputer Science, Software Engineering, Data Science, or a related field.
• 4–6 years of professional experiencein Data Engineering or a closely related role.
• Strong hands-on experience designing and implementing production-gradeETL/ELT pipelines.
• Strong proficiency inPythonfor data processing, automation, and data-engineering workflows.
• AdvancedSQLskills and a strong understanding of relational databases.
• Practical production experience withApache Airflowfor workflow orchestration.
• Hands-on experience withApache Sparkand distributed data processing.
• Strong understanding of data modelling, transformation patterns, and data-pipeline architecture.
• Experience withRedis, caching strategies, and high-performance data-access patterns.
• Experience processing large datasets and optimizing pipeline and query performance.
• Strong understanding of data quality, validation, monitoring, observability, and pipeline reliability.
• Familiarity withLinux, Git, Docker/containers, and modern software-engineering practices.
• Strong analytical, troubleshooting, communication, and cross-functional collaboration skills.

Preferred Qualification

• Experience with cloud data platforms and object storage services such asAWS, Azure, or GCP.
• Experience working withPostgreSQL, data warehouses, or analytical databases.
• Experience processing geospatial data or large-scale location-based datasets.
• Familiarity withDuckDB, Apache Sedona, Trino, Presto, or similar analytical technologies.
• Experience processing high-volume event, mobility, transactional, or geospatial data.
• Familiarity withCI/CD pipelinesand infrastructure-as-code practices.
• Experience supporting data products, analytics platforms, orAI/ML pipelines.