Big Data Engineer
2 months ago
About ByteDance
Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.
Why Join Us
Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible.
Together, we inspire creativity and enrich life - a mission we aim towards achieving every day.
To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always.
At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve.
Join us.
About The Team
The batch processing team is responsible for the company's offline data processing and distributed training, supporting various business scenarios such as offline ETL and machine learning within the company. The components involved include the offline computing engine Spark, the in-house distributed training framework Primus, feature storage solutions like Iceberg and Hudi, as well as Ray, a next-generation distributed application framework. Faced with massive-scale scenarios, extensive functional and performance optimizations have been carried out in Spark, Primus, Feature Store, and support for the adoption of the new-generation distributed application framework Ray in relevant company scenarios.
What you will be doing:
- Responsible for the development and performance optimisation of the in-house Feature Store functionality based on Iceberg;
- Participant in optimisation of the integration of Iceberg with various upper-level computing engines;
- Involve in platform-related infrastructure development.
Qualifications
Minimum Qualifications
- Bachelor's Degree or above, majoring in Computer Science, or related fields, with 4+ years of relevant development experience in the field with a strong programming ability, and proficiency in Java, Python, C++, with the ability to develop and optimize large-scale distributed systems.
- In-depth research and relevant experience in one or more data lake formats such as Delta, Hudi, or Iceberg.
Preferred Qualifications
- In-depth research or practical experience in open-source big data computing frameworks and scenarios like Hadoop, Spark, Flink, Presto, and more.
ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.
Tell employers what skills you have
Machine Learning
Scala
Big Data
Pipelines
Hadoop
Data Management
ETL
Data Engineering
SQL
Distributed Systems
Python
Java
Data Analytics
Databases
Visual Analytics
C++
-
Senior Big Data Engineer
3 weeks ago
Singapore NTT Data Singapore Full timeRequirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...
-
Senior Big Data Engineer
3 weeks ago
Singapore NTT Data Singapore Full timeJob Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...
-
Big Data Engineer
24 hours ago
Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full timeRoles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 6 - 8 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...
-
Big Data Framework Engineer
2 weeks ago
Singapore TANGSPAC CONSULTING PTE LTD Full timeRoles & ResponsibilitiesAs a Data Framework Engineer at TANGSPAC CONSULTING PTE LTD, you will be responsible for creating frameworks and technical features that enable faster operationalisation of data models, analytical models, and user-generated contents. This includes partnering with data scientists to enable faster adoption of AL/ML model-based systems,...
-
Big Data Developer
2 weeks ago
Singapore PRIMESTAFF MANAGEMENT SERVICES PTE LTD Full timeRoles & ResponsibilitiesOur client is looking for an experienced and highly skilled Senior Big Data Developer to join the dynamic team. The ideal candidate will have over 10 years of extensive working knowledge and hands-on expertise in big data technologies, especially in Spark on Scala or PySpark, and Hive. This role will involve the design, development,...
-
Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full timeJob DescriptionWe are seeking an experienced Senior Big Data Engineer to join our team at RAPSYS TECHNOLOGIES PTE. LTD.Responsibilities:Design and develop data applications using Python and other programming languages, ensuring efficient data processing on big data systems.Create APIs using Python to handle structured and unstructured data, with a strong...
-
Singapore SPRINGFIELD CONSULTING PTE. LTD. Full timeJob Title: Big Data EngineerWe are seeking an experienced Big Data Engineer to join our team at Springfield Consulting PTE. LTD.The estimated salary for this role is around SGD 120,000 - 180,000 per annum, depending on experience and qualifications.About the Job:Company Overview:Springfield Consulting PTE. LTD. is a reputable consulting firm that specializes...
-
Big Data Architect Specialist
14 hours ago
Singapore USER EXPERIENCE RESEARCHERS PTE. LTD. Full timeAbout the JobWe are seeking a highly skilled and experienced Big Data Engineer to join our team at USER EXPERIENCE RESEARCHERS PTE. LTD.With a minimum of 4 years of experience managing data engineering jobs in big data environments such as Cloudera Data Platform, the ideal candidate will be responsible for designing, developing, and maintaining the data...
-
Data Scientist
1 month ago
Singapore NTT Data Singapore Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore.Key Responsibilities:Design and develop large-scale data processing systems using Hadoop and Spark.Implement data pipelines using Spring Boot and integrate data from multiple sources.Develop and maintain data warehouses using NoSQL databases such as HBase and...
-
Big Data Engineer with AI Expertise
2 weeks ago
Singapore CORPORATE IT SOLUTIONS PTE. LTD. Full timeUnlock the Power of Big Data with Corporate IT Solutions PTE. LTD.We are seeking a highly skilled Senior Cloud Software Developer in Analytics and AI to join our dynamic and focused team. As an AI/ML Engineer, you will implement core software components and contribute to the scalable design of our cloud software architecture. This is an excellent opportunity...
-
Big Data Lead Architect
24 hours ago
Singapore COMBUILDER PTE LTD Full timeRoles & ResponsibilitiesSummary:We are seeking a highly skilled and experienced Big Data Lead Architect to oversee the design, development, and deployment of big data solutions. The ideal candidate will have strong expertise in managing and architecting large-scale data platforms using advanced technologies such as Cloudera, Azure Cloud, and Kubernetes,...
-
Senior Big Data Developer
1 week ago
Singapore Helius Full timeTalend Big Data Developer RoleAt Helius, we are seeking a skilled Big Data Developer to join our team. The ideal candidate will have experience with Talend Big Data version 7+ and Hadoop, with a strong understanding of data models and data flows.Key Responsibilities:Design and develop production-grade Talend Big Data jobs with consideration for IT...
-
Data Scientist
2 weeks ago
Singapore NTT Data Singapore Full timeRequirementsFor this role at NTT Data Singapore, we are looking for a highly skilled professional with a Bachelor's degree in Computer Science or a related field. The ideal candidate will have at least 7 years of IT experience, with a minimum of 5 years in a similar role, preferably in Big Data tools.Key ResponsibilitiesOur ideal candidate will have a strong...
-
Senior Big Data Solution Architect
3 weeks ago
Singapore Tencent Singapore Full timeResponsibilities: Cloud & Smart Industries Group (CSIG) is responsible for promoting the company's cloud and industry Internet strategy. CSIG explores the interactions between users and industries to create innovative solutions for smart industries via technological advancements such as cloud, AI, and network security. While driving the digitalization of...
-
Senior Big Data Architect
14 hours ago
Singapore WEB SYNERGIES (S) PTE LTD Full timeAbout WEB SYNERGIES (S) PTE LTDWe are a dynamic company seeking an experienced Big Data Architect to join our team.Job Summary:As a seasoned professional, you will be responsible for designing and implementing large-scale data architectures using Big Data technologies such as Hadoop and Spark. You will work closely with cross-functional teams to gather,...
-
Data Engineer
2 weeks ago
Singapore QUESS SELECTION & SERVICES PTE. LTD. Full timeRoles & ResponsibilitiesFull time roleAttractive Annual leaves + Insurance coverageResponsibilities:· Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives· Collaborate with cross-functional teams to understand data requirements and translate them into effective data...
-
Senior Big Data Solution Architect
3 weeks ago
Singapore Tencent Singapore Full timeJob Responsibilities:As a Senior Big Data Solution Architect at Tencent Singapore, you will be responsible for designing and implementing Big Data solutions tailored to customer business scenarios and technical requirements. You will need to deeply understand customers' needs and pain points regarding Big Data products and solutions.Recommend the core...
-
Data Engineer
24 hours ago
Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full timeRoles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 4 - 6 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...
-
Bigdata engineer
2 weeks ago
Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full timeRoles & ResponsibilitiesTo be responsible for managing technology in projects and providing technical guidance / solutions for work completion (1.) To be responsible for providing technical guidance / solutions ;define, advocate, and implement best practices and coding standards for the team. (2.) To develop and guide the team members in enhancing their...
-
Singapore THEE BIRAPANJAA SYSTEMS (PTE. LTD.) Full timeJob OverviewThe role of a Data Engineer is a pivotal position within THEE BIRAPANJAA SYSTEMS (PTE. LTD.) that involves designing, developing, and maintaining large-scale data systems. As an expert in ETL (Extract, Transform, Load) processes, the successful candidate will have extensive knowledge of data extraction, transformation, and loading. This includes...