Senior Product Manager, Copilot AI
Microsoft
Software Engineering, Product, Data Science
USD 119,800-234,700 / year
Posted on Jun 5, 2025
Senior Product Manager, Copilot AI
Mountain View, California, United States
Date posted
Jun 04, 2025
Job number
MIC016564
Work site
Microsoft on-site only
Role type
Individual contributor
Profession
Product Management
Discipline
Product Management
Employment type
Full-time
CLICK HERE TO APPLY
Job Description
Overview
Microsoft will accept applications and processes offers for these roles on an ongoing basis.
As Microsoft AI we are pushing the boundaries of technology.
We are creating unique, beautiful and powerful products that will change lives. A small, friendly, fast-moving team, we support each other to do the best work of our lives, always looking to break new ground, fast. We are proud of what we build, how we build it and that our products will define the AI era. We run lean, obsess about users, and always make our decisions based on the evidence. We ship regularly, so your work will have real and immediate impact. It’s a time of huge change in the AI landscape, and this role will put you right in the heart of it.
Being passionate and opinionated about human computer interaction, you will work at the nexus of product and research specifically focused on information retrieval for language models. Your products are the language models that power Microsoft Copilot. You will be responsible for balancing product needs with research priorities, ensuring that Copilot's messages are high quality, factual and safe. You will also be responsible for prioritizing new features and research, identifying gaps, building evaluations, defining metrics and working closely with Engineers, AI researchers and other dependency teams to build and execute product plans. We’re looking for someone with an abundance of positive energy, empathy, and kindness, in addition to being highly effective. The right candidate takes the initiative and enjoys building world-class consumer experiences and products in a fast-paced environment.
Microsoft AI (MAI) focuses on Copilot and other consumer AI products and research. We combine world-class AI research together with top notch design and product craft. This role requires a balance of technical skills and effective people management skills.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
By applying to this U.S. Mountain View, CA OR Redmond, WA position, you are required to be local to the San Francisco area OR Seattle area and in office 3 days a week.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Define the end-to-end evaluation strategy for LLM-based product, spanning defining metrics, building datasets, synthesize insights and translate them into actionable steps for product response quality improvement.
- Design and run human evaluations, write evaluation guidelines, and ensure evaluation quality.
- Design and write LLM-as-judge prompts, build and run machine evaluations.
- Collaborate with engineers, and product teams to drive eval implementations and share best practices across the board.
- Partner with data and insights teams to develop scalable pipelines and dashboards to track pre-launch eval performance across all dimensions.
- Own product requirements and roadmaps for internal tools used to run and analyze evaluations at scale.
- Translate evaluation insights into actionable product improvements and inform product quality improvement decisions.
- Monitor trends in product performance, user feedback, and competitive benchmarks to continuously refine evaluation approaches.
- Own key projects, proactively identifying risks and proposing solutions to ensure timely delivery.
- Coordinate Cross-Team Collaboration: Work closely with engineering, data science, and product stakeholders to align on goals, track deliverables, and communicate status updates effectively.
- Embody our Culture and Values.
Required Qualifications
- Bachelor's Degree AND 5 years experience in product/technical program management or software development
- OR equivalent experience.
- 3 years of experience leading multi-disciplinary projects, defining requirements, developing project plans and working with multi-disciplinary teams to execute them.
- Experience in building and evaluating ML-powered or LLM-powered products.
Preferred Qualifications
- Computer science background, experience in training/ evaluation of LLMs
- Technical depth in software development, data science and machine learning. While you are not expected to write code on the critical path, you’re able to define and execute on zero-to-one prototypes, reason over large amounts of data, and speak the language of the engineers and researchers you work with.
- Experience using and applying Large Language Models. LLMs will be your clay – you should know how to prompt engineer them, how to evaluate them, and understand how they are tuned.
The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.
Industry leading healthcare
Educational resources
Discounts on products and services
Savings and investments
Maternity and paternity leave
Generous time away
Giving programs
Opportunities to network and connect
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.