*GOV* AI Quality Engineer | LLM | NLP

5 days ago

Singapore SCIENTEC CONSULTING PTE. LTD. Full time

AI Quality Engineer (LLM/NLP)Working Hours: Mon-Fri Location: Central Remuneration: Up to $9,000 + AWS Job Summary We are seeking an AI Quality Engineer to evaluate and ensure the accuracy, reliability, and performance of Large Language Models (LLMs) used in GenAI applications such as chatbots, classification tools, and RAG systems. The role focuses on identifying hallucinations, validating model behaviour, and supporting improvements through structured testing and collaboration. Key Responsibilities Design and execute test cases to assess LLM accuracy, relevance, and contextual correctness. Detect and analyse hallucinations or fabricated outputs, and document them clearly. Develop automated test scripts (Python, PyTest or similar) to streamline LLM regression testing. Conduct functional and non-functional testing, including performance and stress tests for LLM-based systems. Evaluate model output quality using NLP metrics and business-specific correctness rules. Collaborate with AI engineers, data scientists, and product teams to improve model behaviour based on test findings. Perform regression testing after fine-tuning, retraining, or system updates to ensure no degradation in accuracy. Maintain structured documentation: test plans, test cases, test logs, and issue reports. Use issue tracking tools (e.g., Jira) to report and track LLM-related bugs and inconsistencies. Apply knowledge of LLMs, NLP concepts, and cloud-based AI environments (AWS/GCP/Azure preferred) to support comprehensive QA coverage. Requirements Experience testing LLMs (e.g., GPT, BERT) for chatbots and conversational AI. Proficiency in test automation (PyTest, custom AI frameworks) to detect inaccuracies and hallucinations. Familiarity with accuracy evaluation methods for high-stakes NLP applications. Understanding of AI/NLP testing methodologies, including hallucination and relevance testing. Strong Python skills for writing test scripts and analysing model. Ability to document and track issues using tools like Jira. Strong problem-solving skills to propose improvements and reduce hallucinations. By submitting your resume, you consent to the collection, use, and disclosure of your personal information per ScienTec's Privacy Policy (scientecconsulting.com/privacy-policy). Aloysius Tan Sheng Rong - R ScienTec Consulting Pte Ltd - 11C5781#J-18808-Ljbffr

  • LLM/NLP QA Engineer — Up to $9,000 + AWS

    5 days ago

    Singapore SCIENTEC CONSULTING PTE. LTD. Full time

    A consulting firm in Singapore is seeking an AI Quality Engineer to ensure the accuracy and performance of Large Language Models (LLMs) in GenAI applications. The role involves identifying inaccuracies, developing automated test scripts, and collaborating with AI teams to enhance model behaviour. Candidates must possess strong experience in LLM testing,...

  • Ai Research Scientist

    5 days ago

    Singapore GK CONSULTING PTE. LTD. Full time

    We're seeking a talented AI Research Scientist to drive innovation in artificial intelligence, natural language processing, and deep learning. Collaborate with top academics and industry experts to develop cutting-edge AI-enabled products and services. Key Responsibilities: - Lead pioneering research in NLP, deep learning, and large language models -...

  • Nlp Engineer

    1 week ago

    Singapore Hays Full time

    **Your new company** This technologically cutting-edge company specialises on innovative software and hardware solutions, contributing to highly dynamic and progressive projects with AI: Computer Vison, Machine Learning, Robotics and Video Analytics. **Your new role** The AI focused team is looking for an **NLP Engineer** which requires the expertise of...

  • Senior Nlp Engineer

    1 week ago

    Singapore PATSNAP PTE. LTD Full time

    **About PatSnap** Patsnap empowers IP and R&D teams by providing better answers, so they can make faster decisions with more confidence. Founded in 2007, Patsnap is the global leader in AI-powered IP and R&D intelligence. Our domain-specific LLM, trained on our extensive proprietary innovation data, coupled with Hiro, our AI assistant, delivers actionable...

  • Senior ML Engineer

    5 days ago

    Singapore Refine Group Full time

    Job Description Our client is building a global network of elite AI professionals. They partner with fast‑scaling SaaS, fintech, and creative tech companies to deliver world‑class AI solutions. As part of their network, you’ll work on challenging projects, collaborate with top‑tier engineers, and shape the future of AI innovation. The Job As a Senior...

  • LLM Engineer

    1 week ago

    Singapore Input Output Hong Kong Full time

    Who are we? IOHK, is a technology company focused on Blockchain research and development. We are renowned for our scientific approach to blockchain development, emphasizing peer-reviewed research and formal methods to ensure security, scalability, and sustainability. Our projects include decentralized finance (DeFi), governance, and identity management,...

  • Principal/lead Engineer

    6 days ago

    Singapore Menyala Full time

    **Splore** is an AI company, building in stealth mode, to solve a new set of problems in the new internet. Focused on Gaming, Web3 and Virtual Worlds, we are re-imagining experiences, content and engagement from the fundamentals as we recognise that user behaviours and needs often go beyond the confines of indexed Web2.0 experiences. We are currently...

  • Gen AI Quality Engineer

    5 days ago

    Singapore ScienTec Consulting Full time

    We are looking for a Gen AI Quality Engineer experienced in testing Large Language Models (LLMs). Gain exposure to cutting-edge Gen AI applications, including RAG chatbots and AI-driven classification systems. Work on next-generation AI quality initiatives with cross-functional product and AI research teams. Dynamic and fast-paced working environment....

  • Ai Engineer

    3 days ago

    Singapore Office of the Deputy President(Res&Tech) Full time

    **Posting Start Date**:01/10/2024 AI Singapore (AISG) is a national AI programme launched by the National Research Foundation (NRF) to anchor deep national capabilities in Artificial Intelligence (AI). The programme office is hosted by the National University of Singapore (NUS) and brings together all Singapore-based research institutions and the vibrant...

  • Sales - Ai Architect, Llm & Gen Ai

    1 week ago

    Singapore Apple Full time

    Imagine what you could do here. At Apple, great ideas have a way of becoming phenomenal products, services, and customer experiences very quickly. Bring passion and dedication to your job and there’s no telling what you could accomplish! Why Apple? At Apple, we believe our products begin with our people. By hiring a diverse team we drive creative thought....