remote llm evaluator

UMELIFE (SINGAPORE) PTE. LTD. seeks an AI evaluation engineer to build and maintain an automated LLM evaluation pipeline covering general capabilities, agent capabilities, and persona/role-playing evaluations with one-click assessment and regression testing.You will execute benchmarks such as MMLU, C-Eval, HumanEval, GSM8K, MATH, and IFEval, manage results...

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale. You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes...

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale.You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes...

Job ResponsibilitiesBuild and maintain an automated LLM evaluation pipeline covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing. The pipeline should support one-click evaluation, historical result comparison, and regression testing. Conduct general capability evaluations using benchmarks such as MMLU,...

Job Responsibilities Build and maintain an automated LLM evaluation pipeline covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing. The pipeline should support one-click evaluation, historical result comparison, and regression testing. Conduct general capability evaluations using benchmarks such as MMLU,...

Kuailu Software is seeking an experienced evaluation engineer to build and maintain automated LLM evaluation pipelines. You will cover general, agent, and persona-based benchmarks, enabling one-click evaluation, historical comparisons, and regression testing.You will deploy and run benchmarks like MMLU, C-Eval, HumanEval, GSM8K, MATH, and IFEval, while...

About Bitdeer: Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud.Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes...

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale. You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes...

Job ResponsibilitiesBuild and maintain an automated LLM evaluation pipeline covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing. The pipeline should support one-click evaluation, historical result comparison, and regression testing. Conduct general capability evaluations using benchmarks such as MMLU,...

Shopee's Engineering and Technology team in Singapore seeks an experienced individual contributor to design and implement robust evaluation frameworks for AI models. You will conduct red-team tests, root-cause analyses of model outputs, and build automated tooling that accelerates product iterations. Role requires strong software engineering fundamentals,...

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale. You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes...

NTU Singapore’s AISG invites a capable researcher to join as an AI evaluation specialist with a focus on multilingual, multicultural, and multimodal LLM evaluation. You will build and maintain evaluation frameworks, collect and translate evaluation data, and collaborate with partners to ensure reliable AI solutions. You should hold a degree in CS/AI/data...

Responsibilities About the Team: We are looking for a sharp and organized Product Operations specialist to join TikTok's Tako team. You will play a key role in building and operationalizing the evaluation framework for Tako's in-app AI chatbot experience, bridging user insights, data analysis, and cross-functional execution to drive continuous product...

AI Singapore (AISG) is Singapore's national programme in artificial intelligence, launched by the National Research Foundation (NRF) to anchor deep national capabilities in AI. Hosted at Nanyang Technological University (NTU), AI Singapore brings together Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies to perform...

Shopee Singapore Private Limited is seeking an AI evaluation engineer to design robust evaluation frameworks and drive safety testing for AI models. You will collaborate with AI and engineering teams to diagnose failures, optimize prompts, and build automated tooling that accelerates product iteration.Strong Python, Linux, and shell scripting skills are...

About Bitdeer: Bitdeer is a world‑leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry‑leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This...

Shopee's Engineering and Technology team in Singapore seeks an experienced individual contributor to design and implement robust evaluation frameworks for AI models. You will conduct red-team tests, root-cause analyses of model outputs, and build automated tooling that accelerates product iterations.Role requires strong software engineering fundamentals,...

Job ResponsibilitiesBuild and maintain an automated LLM evaluation pipeline covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing. The pipeline should support one-click evaluation, historical result comparison, and regression testing. Conduct general capability evaluations using benchmarks such as MMLU,...

NTU Singapore’s AISG invites a capable researcher to join as an AI evaluation specialist with a focus on multilingual, multicultural, and multimodal LLM evaluation. You will build and maintain evaluation frameworks, collect and translate evaluation data, and collaborate with partners to ensure reliable AI solutions.You should hold a degree in CS/AI/data...

YO AI Labs is seeking Indonesian bilingual experts to support a large-scale linguistic and AI training project. You will evaluate AI-generated Indonesian speech for nativeness, fluency, and quality, and provide detailed feedback to improve language model performance.Strong Indonesian expertise, English communication, and meticulous attention to detail are...