LLM Training Operation Math

About the team
Seed Global Data is a team focused on producing international data for LLMs. For the training of large models, data is the lifeline of model quality - and the Global Data team is working closely with technical, product, and operations teams to ensure effective data production strategies and execution management.
As a key member of our LLM Global Data Team, the LLM Training Operations Analyst will play a pivotal role in managing the intricate processes involved in training large language models (LLMs) with diverse coding datasets. This role focuses on overseeing and improving operational workflows, primarily for code-related projects, ensuring they are delivered with high quality and efficiency.
Your Role Will Involve
- Project Management: Lead and manage multiple math-focused LLM training projects, ensuring timelines, quality standards, and objectives are met. Track project progress, identify risks, and implement corrective actions as necessary to keep projects on course. Build and maintain strong relationships with product managers, researchers, data annotators, and other cross-functional team members. Communicate project updates, address concerns, and align expectations to ensure successful project outcomes. Coordinate meetings and discussions with global teams to ensure seamless project execution and work with external vendors and trainers per project demands.
- Workflow Design and Management: Design, manage, and optimize workflows for math-focused LLM training projects, including training design, QA processes, and performance tracking to meet project needs. Collaborate closely with product managers, project leaders and cross-functional teams to ensure alignment on quality metrics and project expectations.
- Operational Improvement: Conduct quality and productivity improvement experiments to enhance operational processes for math-related training data. Lead and support general annotation operation improvement initiatives across various data domains. Develop and maintain technical guidelines and casebooks to support consistent, high-quality data production.
- Data Checking and Analysis: Design and implement robust data analysis strategies to evaluate training and evaluation datasets for LLM projects in the math domain. Ensure the mathematical accuracy and statistical validity of all project data. This includes designing and implementing robust data checking protocols, performing deep-dive analysis to identify trends and anomalies, and translating quantitative findings into actionable insights for model improvement. You will collaborate with data annotators, researchers, and product managers to define quality benchmarks and ensure data-driven decision-making throughout the project lifecycle.
- Team Leadership and Collaboration: Provide mentorship and guidance to team members, helping to develop their skills and ensuring the delivery of high-quality outputs. Foster a collaborative environment where team members can share knowledge and best practices to improve overall performance.
Qualifications
Minimum Qualifications
- Hold a Bachelor's degree or higher, ideally in Mathematics, Statistics, or a related quantitative discipline.
- Strong Professional communication skills in Chinese mandarin, to engage effectively with internal and external stakeholders that are based in Mandarin-speaking markets.
- Strong project management skills, with the ability to design, manage and optimize complex workflows.
- Possess robust communication and problem-solving skills, capable of clearly explaining and interpreting mathematical concepts.
- Exhibit sound independent judgment while thriving in collaborative, deadline-driven project settings.
- Show genuine enthusiasm for large language models (LLMs) and computational thinking, with the adaptability to excel in a dynamic, fast-paced work environment.
- Demonstrate exceptional English proficiency, with advanced writing and analytical evaluation abilities.
Preferred Qualifications
- Experience in competitive mathematics such as Mathematical Olympiad at regional or international level.
- Professional working proficiency in Mandarin Chinese (both written and spoken) for seamless collaboration with Chinese-speaking teams.
- Experience in RLHF annotation and working with leading AI/LLM companies on technical projects.
- Passionate about LLM technologies, human behavior and user experience with a keen interest in analyzing diverse case studies.
- Self-motivated, intellectually curious professionals who thrive in mentoring junior team members while maintaining rigorous analytical standards.