Gen AI Quality Test Engineer

1 week ago

Singapore SCIENTEC CONSULTING PTE. LTD. Full time

Working Hours: Monday – Thursday (8.30am – 6pm), Friday (8.30am – 5.30pm) (Hybrid working arrangement)Working Location: Central Salary Package: Basic + AWS We are looking for a skilled and detail-oriented Gen AI Quality Engineer to join our team. In this role, you will be responsible for testing Large Language Models (LLMs) integrated into Gen AI applications like a RAG chatbot and film classification system. Key Responsibilities Design and execute comprehensive test cases to evaluate the accuracy, reliability, and performance of LLMs integrated into Gen AI applications. This includes verifying that the model responses are relevant, contextually appropriate, and factually correct. Focus on detecting hallucinations where the model produces false or fabricated information, ensuring these are promptly reported and addressed by the development team. You will help refine the models to reduce these occurrences. Assess the accuracy of model outputs, especially in high-precision contexts like chatbot conversations or film classification, ensuring that LLMs produce responses that are both relevant and correct according to predefined business logic. Implement automated testing for common use cases, edge cases, and regression tests, especially focusing on cases that tend to trigger hallucinations or inaccuracies in the model's responses. Functional & Non-Functional Testing: Evaluate the LLM's functionality in different scenarios to check if it meets the functional requirements. Also, perform non-functional testing like performance, load, and stress tests to assess the scalability of LLMs when handling high loads or multiple queries. Identify and document bugs related to hallucinations, inaccurate outputs, or unexpected model behaviors. Work closely with data scientists and developers to resolve issues, refine models, and ensure quality. Collaborate with AI researchers, engineers, and product teams to understand the nuances of model training, improve the models based on feedback, and suggest improvements based on test findings. Ensure that model updates, fine-tuning, or new training data do not introduce regressions or increase hallucinations and inaccuracies. Perform retesting of fixed issues and reassess model accuracy after updates. Maintain thorough test documentation, including test plans, test cases, test logs, and issue reports focused on LLM-specific concerns like hallucinations and inaccuracies. Requirements Bachelor's degree in Computer Science, Information Systems, or related field. Minimum 4 years of QA experience, with hands‐on exposure to LLM and Gen AI testing. #J-18808-Ljbffr

  • Gen AI Quality Engineer

    5 days ago

    Singapore ScienTec Consulting Full time

    We are looking for a Gen AI Quality Engineer experienced in testing Large Language Models (LLMs). Gain exposure to cutting-edge Gen AI applications, including RAG chatbots and AI-driven classification systems. Work on next-generation AI quality initiatives with cross-functional product and AI research teams. Dynamic and fast-paced working environment....

  • Gen AI QA Engineer — LLM Testing

    1 week ago

    Singapore SCIENTEC CONSULTING PTE. LTD. Full time

    A leading technology consulting firm in Singapore is seeking a skilled Gen AI Quality Engineer to test and validate Large Language Models (LLMs) integrated into applications. This role requires at least 4 years of QA experience, with a strong emphasis on detecting and resolving model inaccuracies. Collaborate with developers and AI researchers while...

  • Gen AI LLM QA

    5 days ago

    Singapore ScienTec Consulting Full time

    A leading consulting firm in Singapore seeks a skilled Gen AI Quality Engineer to ensure the accuracy and reliability of LLMs integrated into AI applications. This role involves testing models for hallucinations, executing automated tests, and collaborating with AI teams. Ideal candidates will have strong skills in Python, test automation, and extensive...

  • Gen AI QA Engineer: LLM Testing

    5 days ago

    Singapore ScienTec Consulting Full time

    A technology consulting firm in Singapore is seeking a Gen AI Quality Engineer experienced in testing Large Language Models. The role involves designing test cases, developing automated scripts, and collaborating with cross-functional teams in a fast-paced environment. Ideal candidates have strong experience with AI testing methodologies and proficiency in...

  • Gen AI QA Engineer for LLM Testing

    5 days ago

    Singapore SCIENTEC CONSULTING PTE. LTD. Full time

    A technology consulting firm is seeking a Gen AI Quality Engineer in Singapore. The role involves testing Large Language Models (LLMs) for accuracy and reliability, working with cross-functional teams to enhance AI applications. Applicants should have experience with testing frameworks and Python, along with a strong foundation in AI/ML methodologies....

  • Test Engineer

    1 week ago

    Singapore Manus AI Full time

    Join to apply for the Test Engineer role at Manus AI Get AI-powered advice on this job and more exclusive features. Key Responsibilities Design, develop, and maintain automated test scripts covering core product features Build and optimize automation testing frameworks to enhance testing efficiency and coverage Work closely with development teams to analyze...

  • Gen AI Engineer

    1 week ago

    Singapore INCOME INSURANCE LIMITED Full time

    Analytics Center of Excellence (COE) at Income Insurance is the enterprise team that champions and develops AI/ML solutions. We focus on in‐house solution development as well as partnerships. To ensure the successful execution of Income's enterprise strategy to build Analytics as a product through impactful AI/ML solutions, driving innovation, and...

  • Developer - Generative AI (Gen AI)

    5 days ago

    Singapore ADOMITA TECHNOLOGIES PTE. LTD. Full time

    Key Responsibilities Design, develop, and deploy Generative AI models and applications. Collaborate with cross-functional teams including data scientists, ML engineers, and product managers. Implement and fine-tune large language models (LLMs) and diffusion models for various use cases. Evaluate and optimize model performance, scalability, and reliability....

  • Developer - Generative AI (Gen AI)

    4 hours ago

    Singapore ADOMITA TECHNOLOGIES PTE. LTD. Full time

    Key Responsibilities • Design, develop, and deploy Generative AI models and applications.• Collaborate with cross-functional teams including data scientists, ML engineers, and product managers.• Implement and fine-tune large language models (LLMs) and diffusion models for various use cases.• Evaluate and optimize model performance, scalability, and...

  • Gen Ai Lead

    2 weeks ago

    Singapore Capgemini Full time

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of...