The Senior Engineering Manager - Data Platform will lead and scale critical engineering teams responsible for streaming data infrastructure and big data platform capabilities. This role manages 10+ senior engineers across multiple teams, driving technical excellence and ensuring the delivery of robust, scalable data infrastructure that powers real-time and batch processing capabilities across the organization. You will be at the forefront of transforming traditional batch-oriented processes into real-time systems while building foundational data capabilities including storage infrastructure, streaming, data orchestration, big data processing, data warehousing, and real-time OLAP databases that enable data-driven decision-making company-wide.
Responsibilities
- Design and build data platforms that promote standardization, including data pipeline development, platform tooling, data lake formatting, and data democratization
- Oversee development of real-time streaming platforms and complex stream processing topologies using latest open-source technologies
- Lead architectural decisions for stream processing, batch processing, and data infrastructure at scale
- Lead and manage high-performing engineering teams including senior individual contributors and technical leads
- Provide mentorship, career development, and performance management for engineers across different experience levels
- Foster a culture of technical excellence, innovation, and continuous learning
- Recruit, hire, and onboard top engineering talent to scale the teams effectively
- Drive the technical roadmap and long-term vision for data platform capabilities, ensuring alignment with business objectives
- Drive initiatives to improve existing tools to deliver value to platform users
- Collaborate closely with Data Science, BI, Product, and Engineering teams to understand requirements and deliver solutions
- Build strong cross-functional partnerships across the organization, influencing teams to adopt best practices
- Ensure high availability, scalability, and performance of data platforms supporting critical business operations
- Establish metrics, monitoring, observability, and visualization practices for data platform reliability
Qualifications
- 12+ years of software engineering experience with 4+ years in engineering management
- 4+ years of experience in open-source big data and streaming technologies on PB scale
- Experience scaling engineering teams and establishing effective development processes
- Deep expertise in streaming technologies: Apache Kafka, Kafka Streams, KSQL, Spark, Spark Streaming
- Strong understanding of distributed systems architecture and integration of operational systems with analytical systems
- Experience with cloud-based data infrastructure including AWS S3, streaming, data lakes, data warehouses, and OLAP databases
- Knowledge of data modeling, schema design, and experience with traditional, NoSQL and columnar databases
- Experience building scalable infrastructure to support stream, batch and micro-batch data processing
- Experience driving technical roadmaps and making strategic technology decisions at scale
- Excellent communication and presentation skills with ability to influence technical and business stakeholders
- Strong project management skills with experience leading large, complex initiatives across multiple teams
- Experience in data processing, orchestration, and data engineering best practices
- Knowledge of data governance, platform standardization, and data democratization principles
The salary for this position is between $204,000 and $255,000 per year, in compliance with California's salary transparency requirements. This range reflects the expected compensation based on qualifications, experience, and location. In addition to the salary, we offer a comprehensive benefits package, including:
- Health, dental, and vision insurance
- 401(k) plan with company match
- Paid time off and holidays
- Professional development opportunities
- Flexible work arrangements
- For more, see www.benefitsatfanatics.com
We are committed to providing a supportive and inclusive work environment.