LLM Evaluation Engineer: Benchmarking
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
KUAILU SOFTWARE (SINGAPORE) PTE. LTD. seeks a capable engineer to build and maintain an automated LLM evaluation pipeline across general, agent, and persona settings, enabling one-click evaluation and regression testing.
You will run benchmarks like MMLU, C-Eval, HumanEval, GSM8K, MATH, and IFEval; set up evaluation environments, track metrics, analyze results, and produce reports to guide pre-training progress.
-
LLM Evaluation Engineer – Benchmarking
1 day ago
singapore, singapore UMELIFE (SINGAPORE) PTE. LTD. Full-timeUMELIFE (SINGAPORE) PTE. LTD. seeks an AI evaluation engineer to build and maintain an automated LLM evaluation pipeline covering general capabilities, agent capabilities, and persona/role-playing evaluations with one-click assessment and regression testing. You will execute benchmarks such as MMLU, C-Eval, HumanEval, GSM8K, MATH, and IFEval, manage results...
-
LLM Evaluation Architect
6 hours ago
singapore, singapore KUAILU SOFTWARE (SINGAPORE) PTE. LTD. Full-timeKUAILU SOFTWARE (SINGAPORE) PTE. LTD. is seeking an experienced researcher/engineer to build and maintain automated LLM evaluation pipelines, covering general, agent, and persona evaluations. You will support one-click evaluation, historical comparisons, and regression tests to drive model improvements. Ideal candidates hold a degree in CS/AI, are fluent...
-
AI Evaluation Engineer: LLM Benchmarking
3 days ago
singapore, singapore SHOPEE SINGAPORE PRIVATE LIMITED Full-timeShopee Singapore Private Limited is seeking an AI evaluation engineer to design robust evaluation frameworks and drive safety testing for AI models. You will collaborate with AI and engineering teams to diagnose failures, optimize prompts, and build automated tooling that accelerates product iteration. Strong Python, Linux, and shell scripting skills are...
-
Applied AI Evaluations Intern: Benchmark LLMs
17 hours ago
Singapore, Singapore NUSX Full-time SGD 60,000 - SGD 100,000 ContractNUSX Labs is the technical arm of NUSX, turning research into inventions and licensed deals. As a hands-on team, you will build and benchmark datasets for multiple document types, and contribute to an LLM pipeline that reads patent documents into structured data and a query layer for analytics.You will supervise by an engineer, work on ground-truth datasets,...
-
LLM Evaluation Engineer
3 days ago
Singapore, Singapore KUAILU SOFTWARE (SINGAPORE) PTE. LTD. Full-time SGD 90,000 - SGD 150,000 ContractJob DescriptionBuild and maintain automated LLM evaluation pipelines covering multiple dimensions, including general capabilities, Agent capabilities, and persona/role-playing capabilities. Support one-click evaluation, historical result comparison, and regression testing.Conduct general capability evaluations using established benchmarks such as MMLU,...
-
AI Benchmark Engineer: Evaluation Infrastructure
21 hours ago
singapore, singapore Wwshemi Full-timeWwshemi in Singapore is seeking a software/ML engineer to design and build rigorous benchmarks for frontier AI agents. You will work with experts to translate workflows into evaluation tasks and run models at scale. The role emphasizes robust benchmarking infrastructure, metrics development, and clear documentation to ensure benchmarks reflect real-world...
-
Senior LLM Performance
17 hours ago
Singapore, Singapore Bitdeer Technologies Group Full-time SGD 120,000 - SGD 180,000 ContractAbout Bitdeer:Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud.Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes...
-
Singapore, Singapore NUS Enterprise Full-time SGD 60,000 - SGD 100,000 ContractNUSX Labs is the technical arm of NUSX, turning research into inventions, patents, and licensed deals. We are a hands-on team where every AI system shipped is graded against a ground-truth dataset.You will build, evaluate, and improve the data pipeline and benchmarking framework under direct supervision of an engineer. Join a small team with real...
-
Senior LLM Inference Performance
17 hours ago
Singapore, Singapore Bitdeer Full-time SGD 120,000 - SGD 180,000 ContractAbout Bitdeer: Bitdeer is a world‑leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry‑leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This...
-
Singapore, Singapore Nanyang Technological University Full-time SGD 90,000 - SGD 130,000 ContractNTU Singapore’s AISG invites a capable researcher to join as an AI evaluation specialist with a focus on multilingual, multicultural, and multimodal LLM evaluation. You will build and maintain evaluation frameworks, collect and translate evaluation data, and collaborate with partners to ensure reliable AI solutions.You should hold a degree in CS/AI/data...