Person, Flooring, Wood, Floor, Hardwood, Furniture, Plywood, Couch, Sphere


Human Computation Program Manager


Human Computation Program Manager

San Francisco | California | United States

We're looking for a talented Human Computation Program Manager to help us grow Pinterest internationally, focusing on human computation and content relevance. We need a dedicated person to run this program, which includes setting and running evaluations at scale for any team at Pinterest and managing our testing vendors. In order to meet the needs of engineers, PMs, data scientists, and designers who need data to be tested by our vendors, the human computation program manager will own the full process from inception to data delivery.  

What you’ll do: 

  • Meet with the human evaluation requester to define what needs to be tested and what the outcome of the testing will be (validation of an experiment, data labeling, or content quality assessment) 
  • Provide a complete testing interface with guidelines, a training guide, and label recommendations, to be validated by the requester
  • Select the best vendor to perform the judgments 
  • Launch tests and tracking progress
  • Provide results and analysis to the requester 
  • Localize the testing templates into several languages, leveraging our localization team 

What we’re looking for: 

  • 3-5+ years of experience with human judgment platforms, human computation, survey design, crowdsourcing, or experimental design
  • 2+ years of technical writing experience
  • Ability to translate and synthesize data and complex information into a compelling narrative
  • Excellent written and verbal communication skills
  • Experience with HTML, XML, CSS, JavaScript, and SQL or specialized database query languages a definite plus
  • Knowledge of concepts like precision/recall and f-scores
  • Active Pinterest user with a strong interest in our product 


Apply Now

Not You?

Thank you

Recommended Jobs

IT Ops Associate

Dublin Ireland Dublin, Ireland IT IT > IT
The IT Operations team at Pinterest solves problems in innovative and creative ways, and this role is an opportunity to have a big impact. You’ll join a team whose mission is to provide joy through frictionless support that is globally consistent....

Fullstack Engineer, Content Growth

San Francisco California United States San Francisco, California, United States Engineering Engineering > Eng
The growth team is dedicated to making Pinterest the best destination for billions of people around the world to discover and collect inspiration everyday.  We're looking for a strong candidate to work across all parts of our core product to ident...

Senior Localization Program Manager, Websites

San Francisco California United States San Francisco, California, United States International Sales and Partnerships > International
Localization at Pinterest is not just responsible for translation, but for cultural adaptation of everything related to the experience with Pinterest product, websites and marketing assets to provide Pinners with an experience that feels local. We...

Strategic Partner Manager, Nordics

Berlin Berlin Germany Berlin, Germany International Sales and Partnerships > International
We want to make sure that Pinners everytime, everywhere find content on Pinterest that helps them create a life they love. As a Strategic Partner Manager, Nordics, you will work directly with brands, publishers and creators in Sweden, Norway and D...