AI Task Assessment Specialist
5 hours ago
Join to apply for the AI Task Assessment Specialist role at Mercor Base pay range $100.00/hr - $120.00/hr About the Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: AI Task Evaluation & Statistical Analysis Specialist Type: Contract Compensation: $100–$120/hour Location: Remote Role Responsibilities Conduct comprehensive statistical failure analysis to identify patterns in AI agent failures across task components such as prompts, rubrics, and templates. Perform root cause analysis to determine if failures are due to task design, rubric clarity, file complexity, or agent limitations. Analyze performance variations across finance sub-domains, file types, and task categories to enhance understanding of AI model performance. Create dashboards and reports to highlight failure clusters, edge cases, and improvement opportunities. Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings. Present insights to data labeling experts and technical teams to foster collaboration and drive improvements. Qualifications Must-Have Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition. Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis. Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets. AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics. Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL. Preferred Experience with AI/ML model evaluation or quality assurance. Background in finance or willingness to learn finance domain concepts. Experience with multi-dimensional failure analysis. Familiarity with benchmark datasets and evaluation frameworks. 2-4 years of relevant experience. Application Process (Takes 20–30 minutes to complete)Upload resume AI interview based on your resume Submit form Resources & Support For details about the interview process and platform information, please check: For any help or support, reach out to: PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity. Seniority level Not Applicable Employment type Part-time Job function Analyst Industries Software Development Referrals increase your chances of interviewing at Mercor by 2x Get notified about new Assessment Specialist jobs in Singapore, Singapore . #J-18808-Ljbffr
-
AI Agent Engineer
5 hours ago
Singapore Manus AI Full timeResponsibilities Develop core modules of the Manus Agent, driving practical applications of generative AI in real‐world scenarios Optimize Agent performance through prompt engineering, tool design, RAG, and Agent system architecture optimization Validate new product feature ideas through rapid prototyping Build evaluation systems and automated evaluation...
-
Generative Ai Specialist
5 days ago
Singapore Jobline Resources Pte Ltd Full time**Responsibilities**: As a Generative AI Specialist, you will: - Evaluate Technologies: Assess and evaluate existing and emerging Generative AI technologies for suitability in meeting business objectives. - Collaborate Across Teams: Work closely with various departments to ensure seamless integration and effective deployment of AI solutions. - Leadership...
-
Research Officer
1 week ago
Singapore Singapore Examinations & Assessment Board Full time**Responsibilities**: Successful applicant will play an active role in the conduct of research on educational measurement and assessment issues and develop assessment services and products that are fit for purpose, educationally sound and positively transforming for stakeholders. - Conduct research studies which focus on harnessing technology to assess...
-
AI Data Specialist
1 week ago
Singapore RWS Full timeWe are hiring globally! We are looking for AI Data Specialists to support the improvement of AI-generated content in English . Job Type: Freelance Location: Remote Work Schedule: Part-time – 10+ hours per week. Flexible – work whenever you want. Start Date: Immediately Duration: Until the end of December 2025 (an extension is possible)Rate: 17/h USD Help...
-
Technical Support Specialist
6 days ago
Singapore Orca AI Full timeTechnical Support Specialist Operations Singapore Intermediate Full-Time Description At Orca AI, we're on a mission to shape the future of the shipping industry through cutting‐edge autonomous ship technology. Our solutions empower teams both on the ship and ashore to make informed decisions that dramatically enhance safety, reduce fuel consumption,...
-
AI Deployment Strategist
22 hours ago
Singapore Mistral AI Full time $120,000 - $180,000 per yearAbout Mistral At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life. We democratize AI through high-performance, optimized, open-source and cutting-edge models, products and solutions. Our comprehensive AI platform is...
-
Tier 2 Technical Support Specialist
2 days ago
Singapore Orca AI Full timeJoin to apply for the Technical Support Specialist role at Orca AI . Get AI-powered advice on this job and more exclusive features. At Orca AI, we're on a mission to shape the future of the shipping industry through cutting‐edge autonomous ship technology. Our solutions empower teams both on the ship and ashore to make informed decisions that dramatically...
-
Ai Product Specialist
2 days ago
Singapore S&P Global Full time**About the Role**: **Grade Level (for internal use)**: 03 Kensho is pioneering AI-powered solutions that reshape industries and unlock new business opportunities. As an AI Specialist, you will be responsible for helping to define and execute strategies that drive market expansion, revenue growth, and adoption of AI products. This role sits at the...
-
Ai Specialist
3 days ago
Singapore ONYX SG PTE. LTD. Full time**JOB DESCRIPTION** Our client, CMS licensed FX exchange platform company is currently looking for an AI Specialist to join their expanding team. Reporting to the Director, the AI Specialist will be responsible to design and optimize machine learning (ML) and deep learning algorithms to process large scale datasets and develop AI model to solve real-world...
-
Insurance Data Analyst and AI Specialist
5 hours ago
Singapore Mercor Full timeInsurance Data Analyst and AI Specialist Position: AI Insurance Data Specialist Location: Palo Alto, CA (in-office, 5 days per week) or Remote Type: Full-time Compensation: $90,000–$200,000/year Commitment: 9:00am–5:30pm PST during training, then local timezone thereafter Base Pay Range $90,000.00/yr – $200,000.00/yr Role Responsibilities Use...