Senior Observability Engineer

Accountabilities

• Enterprise Observability Architecture:Design, implement, and govern a unified observability framework covering metrics, logs, traces, and events using technologies such as IBM Instana, Grafana, OpenTelemetry, Telegraf, InfluxDB, and Prometheus.

• Technology Innovation and Implementation:Lead first-of-a-kind (FOAK) implementations by evaluating emerging observability technologies and transforming successful approaches into secure, scalable, repeatable, production-ready solutions.

• Standards and Governance:Establish enterprise-wide standards for telemetry pipelines, data retention, high-cardinality management, platform governance, and observability cost optimization.

• Site Reliability Engineering:Define and manage Service Level Indicators (SLIs), Service Level Objectives (SLOs), and error budgets to improve service availability, performance, and reliability.

• Major Incident Management:Act as a senior technical escalation point during critical P1/P2 incidents, lead incident response sessions, coordinate cross-functional troubleshooting, and conduct evidence-based root cause analyses.

• Monitoring Optimization:Reduce mean time to detect (MTTD), mean time to resolve (MTTR), and unnecessary alerts through event correlation, dynamic thresholds, dependency mapping, and proactive monitoring strategies.

• Automation and Platform Engineering:Drive Observability-as-Code initiatives and infrastructure automation using Ansible, Terraform, Python, Bash, and GitOps practices.

• Automated Operations:Develop automated deployment, configuration, monitoring agent management, telemetry collection, and self-healing workflows to improve operational consistency and efficiency.

• ITSM and DevOps Integration:Integrate observability platforms with ServiceNow, Netcool, and CI/CD pipelines to streamline incident management, service operations, and software delivery.

• Cloud-Native Observability:Design and implement deep observability for Docker, Kubernetes, OpenShift, microservices, and multi-cloud environments across AWS, Azure, and Google Cloud Platform.

• Secure Telemetry Management:Correlate application performance monitoring data with Kubernetes control planes, pods, nodes, and infrastructure dependencies while ensuring secure telemetry pipelines through role-based access control, encryption, secrets management, and image scanning.

• Technical Leadership and Transition Management:Lead knowledge transfer programs, vendor transitions, and operational readiness handovers for globally distributed, 24/7 operations teams.

• Mentoring and Strategic Collaboration:Mentor engineering teams, promote observability best practices, and influence enterprise technology roadmaps to support long-term reliability and modernization goals.

Requirements:

• Professional Experience:At least 12 years of overall IT experience, including a minimum of 5–7 years in a lead architect, site reliability engineering, or principal observability engineering role within a large-scale enterprise environment.

• Observability and APM Expertise:Extensive hands-on experience with enterprise monitoring and observability platforms, particularly IBM Instana, Grafana Enterprise, Grafana Alloy, Prometheus, OpenTelemetry, Telegraf, and InfluxDB.

• Legacy Monitoring Technologies:Strong knowledge of traditional monitoring and log analytics solutions such as SolarWinds, Netcool, Elastic, and Splunk, with a proven ability to modernize legacy monitoring environments.

• Enterprise Architecture:Demonstrated experience designing scalable, secure telemetry pipelines, managing time-series databases, and establishing observability standards across complex technology ecosystems.

• Platform Modernization:Proven success migrating organizations from traditional monitoring tools to proactive, automated observability platforms that improve operational efficiency and system reliability.

• Cloud and Containerization:Strong experience with Kubernetes, Docker, OpenShift, and cloud environments such as AWS, Azure, and Google Cloud Platform.

• Infrastructure Knowledge:Comprehensive understanding of Linux, particularly Red Hat Enterprise Linux (RHEL), Windows Server, VMware, Citrix Virtual Desktop Infrastructure (VDI), load balancers, and edge proxies.

• Automation and DevOps:Practical expertise in Ansible, Terraform, Python, Bash, webhooks, GitOps, and CI/CD tools such as GitHub Actions, GitLab, and Jenkins.

• ITSM and Service Management:Familiarity with ServiceNow, ITIL 4 principles, advanced major incident management, and enterprise operational processes.

• Leadership and Transition Delivery:Extensive experience leading first-of-a-kind technology deployments, complex vendor transitions, knowledge transfer initiatives, and operational handovers.

• Analytical and Problem-Solving Skills:Strong troubleshooting capabilities, a structured approach to root cause analysis, and the ability to resolve complex reliability and performance challenges.

• Communication and Collaboration:Excellent communication and stakeholder management skills, with the ability to work across technical teams, vendors, and global 24/7 operations.

• Preferred Certifications:Certified Kubernetes Administrator (CKA), cloud architect certifications for AWS or Azure, or relevant application performance monitoring and observability vendor certifications.

Benefits:

• Employment Arrangement:CLT employment contract with a standard 40-hour workweek.

• Remote Work:Flexible remote working arrangements, with the ability to work from home when client-site attendance is not required.

• Health Insurance:SulAmérica Prestige medical coverage, including eligible legal dependents.

• Dental Insurance:SulAmérica dental coverage, including eligible legal dependents.

• Life Insurance:Prudential life insurance coverage equivalent to 24 times salary.

• Private Pension:MetLife private pension plan, with company matching contributions of up to 6%, subject to the applicable plan terms.

• Meal and Internet Allowance:Flash benefit allowance of R$1,000 per month for meals and internet.

• Employee Assistance Program:Access to support resources designed to help employees manage personal and professional challenges.

• Wellness Program:Access to initiatives supporting employee health and well-being.

• Additional Compensation:Eligibility-based performance compensation and potential equity awards through an Associate Equity Appreciation Program.

• Professional Development:Opportunities to work with enterprise-scale technology environments, modern observability platforms, and cloud-native engineering practices.

• Inclusive Workplace:A commitment to equal employment opportunities, diversity, inclusion, and professional growth.