Senior Technical Program Manager, AI Evaluation and Experimentation
Microsoft
Senior Technical Program Manager, AI Evaluation and Experimentation
Multiple Locations, United States
Save
Overview
With more than 45,000 employees and partners worldwide, the Customer Experience and Success (CE&S) organization is on a mission to empower customers to accelerate business value through differentiated customer experiences that leverage Microsoft’s products and services, ignited by our people and culture. We drive cross-company alignment and execution, ensuring that we consistently exceed customers’ expectations in every interaction, whether in-product, digital, or human-centered. CE&S is responsible for all up services across the company, including consulting, customer success, and support across Microsoft’s portfolio of solutions and products. Join CE&S and help us accelerate AI transformation for our customers and the world.
The Strategy and Operations (S&O) team is dedicated to driving strategic vision and operational excellence across CE&S. We aim to enhance our company's edge by leveraging market insights, fostering innovation, and ensuring seamless execution of strategic initiatives.
The Technical Program Manager (TPM) will establish and lead the evaluation, testing, and experimentation function for AI solutions across Customer Experience & Success (CE&S). This role is responsible for defining the strategic vision, building technical blueprint and proving 0->1 motions, influencing technical roadmaps, and driving cross-functional execution to ensure AI programs deliver reliable, scalable, and responsible value.
The Senior Technical Program Manager-AI Evaluation and Experimentation will implement a unified framework for comprehensive evaluation, rigorous testing, and safe experimentation, working closely with engineering, data science, business, and governance teams.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Qualifications
Required:
- Bachelor's Degree AND 4+ years experience in engineering, product/technical program management, data analysis, or product development OR equivalent experience.
- 2+ years of experience managing cross-functional and/or cross-team projects.
- Bachelor's Degree AND 4+ years' experience in engineering, product/technical program management, data analysis, or product development OR equivalent experience.
- 2+ years of experience managing cross-functional and/or cross-team projects.
- Proven experience in AI/ML systems architecture and technical leadership.
- Strong knowledge of model evaluation frameworks (e.g., MLflow, TensorBoard) and automated QA tools.
- Expertise in Azure MLOps practices, CI/CD pipelines, and cloud-based AI platforms.
- Familiarity with responsible AI principles, compliance frameworks, and risk mitigation strategies.
Preferred:
- Hands-on experience with large-scale AI deployments and performance optimization.
- Background in data governance and ethical AI implementation.
- Excellent communication and stakeholder managemet
Technical Program Management IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft will accept applications for the role until November 8, 2025.
Responsibilities
Technical Architecture Leadership
- Collaborate with various technical team to validate that technical architecture standards for AI/ML systems across multiple projects to take into account the quality and resilience.
AI Evaluation & Quality Assurance
- Establish scalable quality assurance patterns for model deployment and integration
- Develop and maintain frameworks for model performance, fairness, and robustness evaluation.
- Implement quality gates and continuous testing pipelines for AI components.
Lifecycle Optimization
- Drive development velocity through streamlined processes and MLOps best practices.
- Monitor and optimize workflows to balance speed with compliance and risk mitigation.
Governance & Risk Management
- Translate responsible AI principles into actionable technical controls.
- Ensure adherence to security, privacy, and ethical standards throughout the lifecycle.
- Support various Agent-building teams with guidance on RAI standards.
Cross-Functional Collaboration
- Partner with solution designers and product teams to align technical execution with business goals.
- Provide technical guidance to cross-functional teams and stakeholders.
Other