Principal DevOps Engineer
WFA Digital Insight
The demand for skilled DevOps engineers has surged, with cloud-native experts in particularly high demand. As the tech industry continues to evolve, companies like NBC are at the forefront, seeking talent to drive their technical vision forward. With the rise of remote work, the opportunity to join a leading company like NBC from anywhere is a significant draw. Candidates should be aware that this role requires deep fluency in Kubernetes internals, cloud infrastructure, and Go systems programming, as well as a passion for mentoring engineers. The broadcast industry's unique demands, such as live production environments, also necessitate adaptability and innovative problem-solving.
Job Description
About the Role
The Principal DevOps Engineer will play a crucial role in architecting and evolving the platform that powers NBC's broadcast production environments. As a senior technical leader on the NBC Production Application Engineering team, this individual will design a Kubernetes-native platform that automates the provisioning, lifecycle management, and delivery of cloud infrastructure at enterprise scale. The team leverages infrastructure-as-code, custom Kubernetes controllers, and GitOps workflows to orchestrate resources across complex multi-account and hybrid cloud environments.The platform is built to support the complex demands of live broadcast production, spanning compute provisioning, dynamic software-defined networking, hybrid DNS models, remote Virtual Desktop Infrastructure (VDI), and robust configuration management via Puppet. The successful candidate will drive the technical vision of the platform, writing production Go services, architecting cloud-native solutions, and establishing the engineering standards the team relies on.
NBC operates at the cutting edge of the cloud-native ecosystem and is never shy about contributing upstream to the open-source projects it relies on to improve its solutions and push the industry forward. This role offers the chance to work with a talented team of engineers who are passionate about innovation and collaboration.
What You Will Do
- Architect a Kubernetes-native platform that models broadcast infrastructure as custom resources
- Lead the technical strategy leveraging Crossplane compositions and custom Go functions to automate provisioning across multi-account AWS environments and on-prem control rooms
- Design, build, and maintain production-grade Kubernetes operators, controllers, and internal platform APIs in Go
- Develop custom Crossplane providers to integrate external enterprise platforms into the control plane, managing resource lifecycles and approval workflows
- Lead the design of cloud networking, DNS strategies, and cross-account connectivity across hybrid environments, automating VPC topology and dynamic network routing
- Collaborate with broadcast systems engineers, system integrators, and external vendors to bridge the gap between broadcast hardware and automated infrastructure
- Lead efforts to 'Puppet-ize' bare-metal compute configurations and integrate proprietary vendor solutions into the configuration-as-code ecosystem
- Serve as a technical authority for the team, writing RFCs, driving architectural decisions, mentoring engineers, and establishing high-confidence CI/CD pipelines
- Own the platform's authorization model, designing hierarchical RBAC systems, resource identifier schemes, and identity integrations that enforce fine-grained access control
- Drive GitOps-based continuous delivery and manage configuration-as-code for compute fleets using Puppet
- Ensure deep operational visibility by designing comprehensive observability and alerting stacks
- Oversee the integration of remote desktop/VDI connectivity solutions, focusing on session authentication, credential management, and gateway routing
What We Are Looking For
- 10+ years of experience designing, building, and operating production infrastructure and cloud-native platforms at enterprise scale
- Strong proficiency in Go (systems-level programming, API servers) and deep experience building Kubernetes controllers/operators
- Expert-level knowledge of the Kubernetes ecosystem, including CRD/XRD generation, operators, informers, admission webhooks, and RBAC
- Experience with Crossplane and control plane architectures
- Deep understanding of cloud infrastructure, networking, and security principles
- Experience with GitOps, Flux, Kustomize, Helm, and other continuous delivery tools
- Strong leadership and mentoring skills, with the ability to drive technical decisions and collaborate with cross-functional teams
- Passion for innovation, open-source contributions, and industry leadership
Nice to Have
- Experience with Excel for data analysis and reporting
- Knowledge of broadcast systems and live production environments
- Familiarity with NBC's current technology stack and infrastructure
- Experience with other programming languages, such as Python or Java
- Certifications in cloud computing, DevOps, or related fields
Benefits and Perks
- Competitive salary and benefits package
- Opportunity to work with a leading media company
- Collaborative and dynamic work environment
- Professional development and growth opportunities
- Flexible remote work arrangements
- Access to cutting-edge technology and tools
- Comprehensive health and wellness programs
- Generous PTO and holiday policies
- Retirement savings plan and other financial benefits
How to Stand Out
- Develop a strong understanding of Kubernetes internals and cloud infrastructure to stand out in the application process.
- Showcase your experience with Go systems programming and ability to write production-grade services.
- Highlight any contributions to open-source projects, especially those related to cloud-native technologies.
- Prepare to discuss your approach to technical leadership, including mentoring engineers and driving architectural decisions.
- Be ready to provide specific examples of your experience with GitOps, Crossplane, and other continuous delivery tools.
- Ensure your portfolio includes examples of your work with cloud-native platforms, including any custom operators or controllers you've built.
- Research NBC's current technology stack and be prepared to discuss how your skills align with their infrastructure and goals.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.