Technical Program Manager, AI Data Quality
Microsoft
This job is no longer accepting applications
See open jobs at Microsoft.See open jobs similar to "Technical Program Manager, AI Data Quality" Matt Wallaert.Software Engineering, IT, Operations, Data Science, Quality Assurance
USD 137,600-267k / year
Posted on May 14, 2025
Technical Program Manager, AI Data Quality
Mountain View, California, United States
Date posted
May 14, 2025
Job number
MIC015629
Work site
Microsoft on-site only
Role type
Individual contributor
Profession
Program Management
Discipline
Technical Program Management
Employment type
Full-time
CLICK HERE TO APPLY
Job Description
Overview
As an AI Data Quality Technical Program Manager, you will work at the nexus of data creation, AI model development, and third-party partners. You will be responsible for ensuring that our research data needs are being met and that our data creation vendors are providing high-quality, high-value and compliant data to support our research priorities.
We are seeking a highly analytical and detail-oriented AI training data quality assurance (QA) specialist with a strong background in coding STEM, phenomenal attention to detail, and program management experience to ensure the quality and integrity of data creation and annotation work performed by Microsoft partners. This individual will play a key role in defining data collection requirements, reviewing and validating complex annotations, ensuring alignment with project guidelines, building instructions and guidelines to annotators, actively enhancing coordination with vendors, giving feedback to vendors, and supporting the continuous improvement of data quality processes that result in improved model training and evaluation. This role will involve deep interaction with senior AI scientists across Microsoft AI.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
By applying to this role based in U.S. Mountain View, CA. You are required to be local to the San Francisco area and be in office 3 days a week.
Responsibilities
- Help drive overall data creation and vendor management strategy.
- Perform quality assurance checks on vendor-provided coding, STEM, RLHF, and other data collection types.
- Collaborate with internal teams and vendors to define quality metrics and resolve discrepancies.
- Develop and maintain data annotation instructions, guidelines and templates.
- Provide detailed feedback and coaching to vendors to uphold high-quality standards.
- Identify systemic quality issues and drive root-cause analysis and resolution plans.
- Support program management of annotation workflows, timelines, and priorities.
- Monitor progress and help track key performance indicators (KPIs) related to QA.
- Participate in designing and improving annotation tooling and platform for QA features.
- Help ensure the safe, compliant, responsible, and agile use of data to support our research priorities.
- Help other Microsoft AI building teams access data for training as needed. Embody our Culture and Values.
Qualifications
Required Qualifications:
- Bachelor’s Degree in Computer Science, Mathematics, Engineering, or a related technical field AND at least 6 years' experience in STEM and Code data creation efforts that involve complex QA and program management
- OR Master’s Degree in Computer Science, Mathematics, Engineering, or a related technical field AND at least 4 years' experience in STEM and Code data creation efforts that involve complex QA and program management, or equivalent experience
- OR equivalent experience
- 4 years of experience in leading multi-disciplinary projects, defining requirements, developing project plans and working with technical teams to execute them
- Excellent attention to detail and critical thinking skills
- Solid understanding of programming concepts, data structures, and algorithms.
- Experience in managing and goal tracking data collection or other equivalent programs
- Experience in coding, or technical program management.
- Strong LLM practitioner, who frequently uses LLMs and understands how training data is used to improve them
- Solid communication and collaboration skills, with the ability to give clear, constructive feedback.
Preferred Qualifications
- PhD. in a relevant field AND at least 6 years' experience in STEM and Code data creation efforts that involve complex QA and program management, or equivalent experience.
- 6 years' experience managing cross-functional and/or cross-team projects.
- Familiarity with Python, SQL, JS or other programming languages used in data annotation.
- Background in educational content, coding assessments, or STEM domains.
- Experience working with data labeling platforms and quality monitoring tools.
Technical Program Management IC5 - The typical base pay range for this role across the U.S. is USD $137,600 - $267,000 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $180,400 - $294,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
Microsoft will accept applications and processes offers for these roles on an ongoing basis.
#MicrosoftAI #copilot
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.
Industry leading healthcare
Educational resources
Discounts on products and services
Savings and investments
Maternity and paternity leave
Generous time away
Giving programs
Opportunities to network and connect
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
This job is no longer accepting applications
See open jobs at Microsoft.See open jobs similar to "Technical Program Manager, AI Data Quality" Matt Wallaert.