Note: The job is a remote job and is open to candidates in USA. Prolific is building the biggest pool of quality human data in the world. They are seeking Computer Science Specialists to help train and evaluate cutting-edge AI models by reviewing research papers, fact-checking AI outputs, and providing human feedback to improve AI performance.
Responsibilities
- AI Evaluation & Ranking: comparing multiple AI-generated responses to technical prompts and ranking them based on accuracy, logic, and safety
- Scientific Review: reviewing CS research papers alongside AI-generated summaries and graphical abstracts to ensure scientific integrity
- Fact-Checking: identifying inaccuracies where the AI has misinterpreted technical data, formulas, or research findings
- RLHF (Reinforcement Learning from Human Feedback): providing the human "ground truth" to help models align with professional standards in software engineering and data science
- Code & Logic Verification: auditing AI-generated code snippets or architectural diagrams for structural and functional correctness
Skills
- At minimum, a BSc (Bachelor of Science) in Computer Science or a closely related technical field
- Ability to interpret research papers, understand complex algorithms, and review code logic
- High level of cognitive competency with a sharp eye for technical hallucinations or logical flaws
- A valid LinkedIn profile to verify your degree and background during the screening process
- A PayPal account to receive payment from our clients
Benefits
- Competitive pay rates
- Flexible hours
- Ability to work from home
Company Overview
Building the most advanced global infrastructure for People Science. It was founded in 2014, and is headquartered in London, England, GBR, with a workforce of 51-200 employees. Its website is https://www.prolific.com/.