Senior Observability Engineer

We are looking for a smart, innovative, and passionate engineer for this role, someone who is interested in best practices around observability, incident management, and security. The ideal candidate welcomes the challenge of building in a dynamic and ever changing environment, and is interested in bringing a culture of operational excellence to a new team. They enjoy seeing their designs run at scale with automation, testing, and an excellent operational mindset. If you exemplify the ethics of, "know about a problem before your users," we want to hear from you!

• Build out observability program and process, recommending and implementing tooling and services
• Managed the security of critical Okta properties and manage security issues such as DDoS attacks and rate limiting
• Ensure our critical infrastructure is meeting uptime and availability standards, and is stable for our Okta customers
• Drive initiatives to evolve our observability platforms to increase efficiency in line with current standards and best practices, especially around incident management
• Build data pipelines into Splunk and use your expertise to build queries and dashboards for a variety of stakeholders
• Recommend, develop, implement, and manage appropriate policy, standards, process, and procedural updates
• Discover and execute on opportunities to automate and increase our automation

• Proficient with observability tools including Pingdom , New Relic, Cloudwatch and Prometheus/Grafana
• Proficient with logging and SIEM tools, especially Splunk
• Proficient with web security and web security tooling
• Experience working in SOC/NOC teams and handling outage escalations and remediation
• Experience with monitoring of hosted platforms , such as Adobe Experience Manager
• Experience with automating systems and infrastructure via Terraform
• Proficient with Git and building deployment pipeline using commercial tools, especially Gitlab
• Demonstrated ability to develop complex applications for cloud infrastructure at scale and deliver projects on schedule and within budget
• Experience with reliability engineering concepts and security best practices on public cloud platforms and web applications
• Experience with developing tooling and automation in Bash, Python, Go, etc.
• Familiar with Linux system administration skills
• Good communication skills, with the ability to influence others and communicate complex technical concepts to different audiences