A production engineer is a developer with who thinks deeply about systems and how they behave in the wild Whether it be networking or the Linux kernel or even a specific interest in scaling algorithms or distributed systems You are a systems engineer who aims to code themselves out of a job by automating all the things learning to leverage great development practices like Test Driven Development or continuous integration to start withResponsibilitiesInfrastructure ResponsibilitiesRadiate knowledge about the services infrastructure and reliability to the rest of the development teamIdentify parts of the system that do not scale provide immediate palliative measures and drive long term resolution of these incidentsPlan the growth of Benzingas infrastructureDevelopmentDeployment ResponsibilitiesDocument every action so your learnings turn into repeatable actions and then into automationImprove the deployment process to make it as boring as possibleManage our infrastructure with Kubernetes and TerraformSecurity ResponsibilitiesProactively identify and reduce security risksDevelop security training and guidance to internal development teamsAbility to discover and patch SQLi XSS CSRF SSRF authentication and authorization flaws and other web based security vulnerabilities OWASP Top and beyondKnowledge of common authentication technologies including JWT OAuth SAML CAs OTPTOTPProduction ResponsibilitiesDesign build and maintain core infrastructure pieces that allow Benzinga to scale to support hundred of thousands of concurrent usersBe on an on call rotation to respond to benzingacom availability incidents and provide support for service engineers with customer incidentsDebug production issues across services and levels of the stackMonitoring ResponsibilitiesMake monitoring and alerting alert on symptoms and not on outagesManage day to day maintenance and evolution of Benzingas Prometheus monitoring and alerting infrastructureBundle Prometheus monitoring as an out of the box monitoring solution for Benzinga productsConfigure dashboards and alerts using GrafanaBuild and maintain the benzingacom public monitoring gatewayImprove coverage of Benzinga performance monitoringCreate automated alerts to notify team members of regressionsRequirementsYou can reason about software algorithms and performance from a high levelYou have experience thinking about systems edge cases failure modes behaviors and specific implementationsYou have worked with distributed systems and have a solid understanding of how modern web stacks are built and whyYou know your way around Linux and the Unix ShellStrong communication skillsExperience with Docker Nginx Go Kubernetes Kong PrometheusExperience with managing large amounts of telemetryOddly enjoys updating services at am on a Sunday hanging out with other devs while everyone else is at the bar sleepingFIrst to jump in when fighting firesPrepares us to fight fires as little as possibleSelf motivated with strong organizational skills