ai evaluation engineer

Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas...

Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas...

Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas...

Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas for...

Serve as the senior technical evaluator within the team, responsible for the design, calibration, and adjudication of evaluation suites and gate thresholds across agent archetypes. Lead gate reviews under the Team Lead’s authority, own the regression-pack methodology for AI/ML model changes, and act as the technical custodian of evaluation quality and...

Serve as the senior technical evaluator within the team, responsible for the design, calibration, and adjudication of evaluation suites and gate thresholds across agent archetypes. Lead gate reviews under the Team Lead’s authority, own the regression-pack methodology for AI/ML model changes, and act as the technical custodian of evaluation quality and...

SummaryWe are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas for...

AI Singapore (AISG) is Singapore's national programme in artificial intelligence, launched by the National Research Foundation (NRF) to anchor deep national capabilities in AI. Hosted at Nanyang Technological University (NTU), AI Singapore brings together Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies to perform...

AI Singapore (AISG) is Singapore's national programme in artificial intelligence, launched by the National Research Foundation (NRF) to anchor deep national capabilities in AI. Hosted at Nanyang Technological University (NTU), AI Singapore brings together Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies to perform...

AI Singapore (AISG) is Singapore's national programme in artificial intelligence, launched by the National Research Foundation (NRF) to anchor deep national capabilities in AI. Hosted at Nanyang Technological University (NTU), AI Singapore brings together Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies to perform...

Summary We are seeking an AI Evaluation Engineer to help understand, analyse, and improve the performance of advanced AI systems. This role combines data-driven analysis, evaluation design, and human-centered insights to assess AI behavior and drive continuous improvement. You will develop evaluation methodologies, analyse model outputs, identify areas for...

SME Careers, located in Singapore, is seeking a Biomedical Engineer to evaluate AI-generated biomedical content for accuracy and clarity. You will leverage your expertise to challenge advanced language models in realistic engineering scenarios, providing critical feedback to ensure responsible engineering practices benefit patient safety and clinical...

About Us Resaro was founded on the belief that AI will change the world in ways we cannot even imagine - but every new technology needs safeguards to advance. We are an independent, third-party AI assurance company: we build the software and run the evaluations that let enterprises and public-sector bodies deploy AI they can actually trust. Our work spans...

Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models, collaborating with a leading AI lab. You will assess AI-generated music across genres, rating it against detailed quality standards while working in Mandarin Chinese and English.You will label songs by genre, structure, instruments, vocals, and other...

Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models in collaboration with a leading AI lab. You will assess AI-generated music across genres and rate it against detailed quality standards, working in Malayalam and English.Key tasks include comparing musicality, creativity, and production quality; labeling...

Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models. You will assess AI-generated music across genres and rate it against detailed quality standards, working in Vietnamese and English.Responsibilities include head-to-head comparisons of musicality and production quality, labeling tracks by genre and...

Mercor is hiring experienced Musicians to evaluate generative musical AI models in partnership with a leading AI lab. You will assess model outputs across in different categories of music in your bilingual language.Ideal candidates have 3+ years as music producer/audio engineer, a degree in music, and native or near-native Malayalam and English. Start date...

Resaro is hiring a hands-on R&D engineering leader for its AI evaluation team. You will grow and manage a high-performing group of engineers and scientists while maintaining architectural depth to guide system design. You will partner with Product and drive engineering excellence across AI evaluation capabilities. You’ll lead 8–10 engineers and...

Resaro in Singapore seeks a senior engineer to own the agentic AI evaluation engine and drive the roadmap beyond the MVP. You’ll be the go-to expert for measuring tool-calling, memory, and multi-step control in agentic systems, collaborating with Product and Governance on testable evaluation logic. This hands-on role is operational, demanding end-to-end...

Polluxa, Inc. in Singapore seeks an ML Evaluation Engineer/Data Scientist to design statistical analyses for AI and LLM evaluation programs. You will ensure results are reliable, reproducible, and scientifically valid, driving rigorous measurement of model changes. You will also measure inter-annotator agreement between human evaluators and perform...