LLM Evaluation Engineer: Benchmarking

Kuailu Software is seeking an experienced evaluation engineer to build and maintain automated LLM evaluation pipelines. You will cover general, agent, and persona-based benchmarks, enabling one-click evaluation, historical comparisons, and regression testing. You will deploy and run benchmarks like MMLU, C-Eval, HumanEval, GSM8K, MATH, and IFEval, while...

Shopee Singapore Private Limited is seeking an AI evaluation engineer to design robust evaluation frameworks and drive safety testing for AI models. You will collaborate with AI and engineering teams to diagnose failures, optimize prompts, and build automated tooling that accelerates product iteration.Strong Python, Linux, and shell scripting skills are...

Bitdeer is seeking an experienced ML infrastructure engineer in Singapore to build benchmarks, manage model evaluation, and optimize performance for AI/cloud workloads. You will design gate criteria for model launches, maintain realistic workloads, and compare deployment options to optimize latency, cost, and reliability.You will collaborate with runtime...

Allegis Group Singapore Pte Ltd is seeking an ML Evaluation Engineer/Data Scientist to bring statistical rigor and measurement excellence to AI and LLM evaluations. You will design analyses, ensure reliability and reproducibility of results, and partner with ML teams to answer critical questions about performance and trust. Responsibilities include...

Bitdeer is seeking an experienced ML infrastructure engineer to build and maintain benchmark pipelines, define model launch gates, and simulate realistic workloads at scale.You will evaluate model and runtime options, implement regression detection in CI/CD, and collaborate with SRE to turn benchmarks into actionable SLOs and alerts. The role emphasizes...

Job Responsibilities Build and maintain an automated LLM evaluation pipeline covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing. The pipeline should support one-click evaluation, historical result comparison, and regression testing. Conduct general capability evaluations using benchmarks such as...

About Bitdeer: Bitdeer is a world‑leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry‑leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This...

NTU Singapore’s AISG invites a capable researcher to join as an AI evaluation specialist with a focus on multilingual, multicultural, and multimodal LLM evaluation. You will build and maintain evaluation frameworks, collect and translate evaluation data, and collaborate with partners to ensure reliable AI solutions.You should hold a degree in CS/AI/data...

TRULYYY PTE. LTD. is seeking a data/model engineer to develop, curate, and validate high-quality training and fine-tuning datasets for real-world ML applications. You will translate complex reasoning and domain knowledge into clear, machine-trainable formats and run end-to-end training and optimization experiments. You will design evaluation criteria,...