Mid-Level Research Engineer, Benchmarks
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
By continuing, you agree to our Terms & Privacy Policy.
About the Role
Join a small, technical team at an early-stage AI evaluation infrastructure startup building benchmarks for frontier AI agents. You will own the design and implementation of rigorous evaluations for realistic, domain-specific workflows, helping technical teams understand how agent performance translates to practical settings.
What You'll Do
• Design, implement, and maintain benchmarks for evaluating frontier AI agents on domain-specific tasks.
• Collaborate with subject-matter experts to turn real workflows into realistic benchmark tasks and evaluation criteria.
• Build reliable infrastructure to run models and agents against evaluation tasks at scale.
• Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes.
• Validate how benchmark results relate to real-world evaluations and technical user needs.
• Write clear technical documentation and reports that communicate findings to research and engineering audiences. What We're Looking For
• Two or more years of experience in research engineering or machine learning engineering, with a focus on AI benchmarks, evaluation infrastructure, or agent environments.
• Experience designing, implementing, and operating benchmarks or evaluation environments for AI agents or large language models.
• Proficiency in Python, Docker, and Linux environments.
• Experience building infrastructure to run AI models or agents reliably, and collaborating with subject-matter experts to define tasks and criteria.
• Experience analyzing workflows across technical or business domains, and developing metrics or validation studies for benchmark quality.
• Strong technical writing, attention to detail, and ability to work independently in unstructured environments.
• Experience with reinforcement learning pipelines, data generation, published work, or public benchmarks is beneficial. Compensation & Benefits Annual salary range: $100,000 to $170,000 USD. Visa sponsorship is available. Location On-site in Singapore, Singapore.
-
Research Engineer, Synthetic Data
2 days ago
Singapore Clera Full-time US$100,000 - US$170,000 ContractAbout the Role Join an AI evaluation and reinforcement learning team building synthetic data pipelines for AI agents. You will turn real-world professional workflows into scalable training tasks and develop ways to measure whether those tasks improve model capabilities. What You'll Do • Build pipelines that transform domain-specific workflows into...
-
Mid-Level Forward Deployed Research Engineer
2 days ago
Singapore Clera Full-time US$100,000 - US$170,000 ContractAbout the Role Join a small engineering team at an early-stage AI evaluation infrastructure company. You will take ownership of urgent, ambiguous technical challenges for AI research organizations and data vendors, helping unblock deployments while turning recurring problems into reusable tools and processes. What You'll Do • Own technical deployment...
-
Mid-Level GTM Engineer
2 days ago
Singapore Clera Full-time US$100,000 - US$170,000 ContractAbout the Role An early-stage AI infrastructure startup is looking for a GTM Engineer to connect technical expertise with revenue growth. You will build and improve the outbound motion for products serving AI researchers, ML engineers, and engineering leaders, working closely with company founders to shape the go-to-market approach. What You'll Do • Own...
-
Mid-Level Research Engineer, QC Automation
5 days ago
Singapore Clera Full-time US$100,000 - US$170,000 ContractAbout the RoleJoin an engineering team building infrastructure for AI reinforcement learning and evaluation. In this role, you will own quality control automation for training data, developing systems that help maintain high standards as data needs grow.What You'll DoBuild automated quality control systems grounded in human judgment and a strong...
-
Mid-Level Research Engineer, QC Automation
3 days ago
singapore, singapore Wwshemi Full-timeAbout the Role Join an engineering team building infrastructure for AI reinforcement learning and evaluation. In this role, you will own quality control automation for training data, developing systems that help maintain high standards as data needs grow. What You'll Do Build automated quality control systems grounded in human judgment and a strong...
-
Software Engineer
2 days ago
singapore FUTUREX TECHNOLOGIES PTE. LTD. Full-timeWhat to expect:Design, develop, and maintain software applications: Write clean, scalable, and efficient code to create software solutions that meet user needs. Build scalable contact data pipelines: Develop robust data ingestion, deduplication, and segmentation logic for contact management at scale. Collaborate with cross-functional teams: Work closely with...
-
Mid Level Data Engineer
10 hours ago
singapore dci consultants private limited Full-timeRole Overview The Data Engineer works closely with statisticians, analytical teams, and business stakeholders to design, build, and maintain the data infrastructure needed to support rigorous statistical production processes. The role centres on building reliable and maintainable data pipelines, validation and frameworks that support statistical production...
-
IT Support Engineer
2 days ago
singapore Alphaeus Pte Ltd Full-time1 to 2 weeks training providedService Desk EngineerProvide first level off-site support to end-users, support to troubleshoot and solve technical problemsTo provide accessible, timely, effective, efficient, and professional quality service to resolve problemsPrompt response to queries via voicemail / email / faxAccurate assessment/ resolution of...
-
Mid-Level Automation
1 day ago
Singapore, Singapore TIANCHENG ELECTRICAL MARINE EQUIPMENT PTE. LTD. Full-time SGD 60,000 - SGD 90,000 ContractRole Overview:We are seeking a detail-oriented Engineer to design, code, and commission thecontrol systems for our marine electrical equipment. You will be responsiblefor integrating PLC/inverter drive logic with real-world mechanical equipmentto ensure safe and efficient marine operations.Key Responsibilities:Logic Programming: Develop andmaintain PLC and...
-
Mid-Level MxML
10 hours ago
singapore, singapore BOUNTEOUSXACCOLITE SINGAPORE PTE. LTD. Full-timeBounteous is seeking a mid-level Murex MxML Developer to design, build, and support interfaces between Murex and upstream/downstream applications for a leading financial institution. You will work on MxML Exchange, integration workflows, and messaging solutions within a fast-paced banking environment. The role requires hands-on experience with Murex MX.3,...