Staff Data Engineer

Accountabilities:

You will lead data architecture and platform design while developing robust data pipelines, improving engineering practices, and ensuring the reliability and performance of enterprise data systems.

• Lead enterprise-wide data architecture strategy and define long-term data platform design principles.
• Evaluate and approve major technical designs, establish engineering best practices, and advise leadership on data technology decisions and roadmap priorities.
• Champion data governance, data democratization, and scalable standards that improve data accessibility, consistency, and quality.
• Collaborate with Solutions Delivery, Business Intelligence, Predictive Analytics, Enterprise Services, and other cross-functional teams to resolve technical data challenges.
• Design, develop, and optimize ETL pipelines using Apache Airflow, dbt, Python, and Snowflake to ingest, transform, and load data into analytical warehouses.
• Build and manage large-scale data ingestion and transformation workflows for structured and unstructured datasets.
• Develop streaming pipelines using Confluent Kafka to support event-driven processing and real-time analytics.
• Use Spark and other big data technologies to support batch processing, streaming analytics, and large-volume data workloads.
• Build and maintain CI/CD pipelines and automate code and process deployments to improve engineering efficiency and release reliability.
• Establish enterprise-wide data quality frameworks, testing procedures, and validation processes to ensure data accuracy and consistency.
• Monitor pipeline health, freshness, availability, performance, and service-level agreement (SLA) compliance through dashboards and automated monitoring.
• Test, validate, and refine data models and analytical processes to improve accuracy and reliability.
• Analyze large datasets using statistical and programming tools to generate insights and support reporting requirements.
• Optimize cloud infrastructure and containerized workloads to improve scalability, operational reliability, and resource utilization.

Requirements:

The ideal candidate combines advanced academic training in a relevant technical discipline with extensive hands-on experience in data engineering, cloud infrastructure, streaming systems, and automated software delivery.

• A master's degree or foreign equivalent in Data Science, Applied Computer Science, Computer Science, or a related field.
• At least 4 years of experience in each of the following areas:
  • Developing and optimizing scalable data pipelines using Apache Airflow and Python.
  • Building streaming data pipelines using Kafka.
  • Tuning big data systems using Kafka and Hive.
  • Using Apache Spark to create jobs for batch processing and streaming analytics.
  • Writing complex SQL queries to retrieve data from warehouses for analysis and reporting.
  • Using AWS for data processing and large-volume workloads.
  • Automating code and process deployments using Python.
  • Working with cloud-based infrastructure.
  • Building Datadog dashboards to monitor data pipelines and system performance.
  • Using Docker and Kubernetes to improve and validate containerization and automation reliability.
  • Implementing Continuous Integration and Continuous Deployment (CI/CD) pipelines using Jenkins.
  • Using GitHub as a code repository and working with both Windows and Linux operating systems.
  • Strong understanding of enterprise data architecture, data modeling, data quality, and pipeline reliability.
  • Ability to evaluate technical designs, establish engineering standards, and make decisions based on scalability, performance, and maintainability.
  • Experience collaborating with cross-functional teams and communicating technical concepts to technical and non-technical stakeholders.
  • Strong analytical and problem-solving skills, with a focus on building reliable, maintainable, and efficient data systems.
  • Ability to work independently and provide technical guidance on complex, organization-wide engineering initiatives.

  Benefits:

  • Annual salary ranging from $175,000 to $200,000.
  • Fully remote work arrangement, with telecommuting available anywhere in the United States.
  • Paid parental leave.
  • 401(k) retirement plan with employer matching contributions.
  • Flexible vacation time.
  • Medical, dental, and vision coverage.
  • Commuter benefits.