14663[JOB] Principal Development Manager, Experimentation Platform
- Jan 7, 2008"We have an unprecedented opportunity to run A/B tests with online
users and innovate more quickly based on actual user response.
Microsoft needs to shift the culture from planning the exact features
to planning a set of possible features, and letting customers guide
-- Ray Ozzie, Chief Software Architect, Microsoft
Do you want to transform the way people build features in websites
and online services? Do you want to have impact across multiple
products and services such as Live, MSN, and Microsoft.com? Do you
want to take an influential role in a strong senior team with huge
growth potential? Do you want to be part of the Software+Services
revolution? Then join the Microsoft Experimentation Platform team
The Experimentation Platform enables product teams to radically
improve their offerings using controlled experiments with live users.
The platform allows testing new ideas quickly using the best-known
scientific method for establishing causality between a feature and
its effects: randomized experimental design. The methodology is to
expose a percentage of users to a new variation of the website or
service, measure the effect on the business, and run statistical
tests to determine whether the difference in metrics were caused by
the change. Experimentation encourages more agile design and release
cycles and can transform the way websites and online services are
developed. We have been involved in multiple experiments where
radical ideas led the introduction of features worth tens to hundreds
of millions of dollars.
The platform is to initially enable experimentation everywhere
Microsoft has an online presence, whether website or services, then
to take experimentation to the developer ecosystem.
As the development manager, you will be responsible for leading and
managing the development of a highly-scalable, highly-available
platform, comprising of services to conduct experiments, and software
to analyze the data, yielding insight through statistical and data
mining techniques. Entrepreneurial spirit, ability to attract and
mentor top talent, and technical leadership skills are important.
You must have very high standards for quality, customer focus, and
trustworthy results because experimentation provides the safety net
for others to try ideas.
Strong technical leadership and proficiency in architecture
and design, preferably including experience in web services and
Experience in managing a senior dev team, and growing dev-
leads and SDEs
Strong track record of delivery
Strong interpersonal skills and the ability to successfully
collaborate with test, program management, and other groups with
At least 5-years of management experience
A CS degree or strong equivalent.
Energy and desire to deliver high-quality software quickly
through agile development (we use Scrum)
Additional valuable experience
Background in Statistics and/or Machine Learning/Data-Mining
Database and analytics experience (OLTP/ETL/OLAP)
For more information, see examples of controlled experiments at
To contact us, please e-mail rkjobs at microsoft dot you know what.
- Ronny Kohavi, General Manager, Experimentation Platform