Big Data Engineer

2 months ago


Singapore BYTEDANCE PTE. LTD. Full time
Roles & Responsibilities

About ByteDance

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join Us

Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible.

Together, we inspire creativity and enrich life - a mission we aim towards achieving every day.

To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always.

At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve.

Join us.

About The Team

The batch processing team is responsible for the company's offline data processing and distributed training, supporting various business scenarios such as offline ETL and machine learning within the company. The components involved include the offline computing engine Spark, the in-house distributed training framework Primus, feature storage solutions like Iceberg and Hudi, as well as Ray, a next-generation distributed application framework. Faced with massive-scale scenarios, extensive functional and performance optimizations have been carried out in Spark, Primus, Feature Store, and support for the adoption of the new-generation distributed application framework Ray in relevant company scenarios.

What you will be doing:

- Responsible for the development and performance optimisation of the in-house Feature Store functionality based on Iceberg;

- Participant in optimisation of the integration of Iceberg with various upper-level computing engines;

- Involve in platform-related infrastructure development.

Qualifications

Minimum Qualifications

- Bachelor's Degree or above, majoring in Computer Science, or related fields, with 4+ years of relevant development experience in the field with a strong programming ability, and proficiency in Java, Python, C++, with the ability to develop and optimize large-scale distributed systems.

- In-depth research and relevant experience in one or more data lake formats such as Delta, Hudi, or Iceberg.

Preferred Qualifications

- In-depth research or practical experience in open-source big data computing frameworks and scenarios like Hadoop, Spark, Flink, Presto, and more.


ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.



Tell employers what skills you have

Machine Learning
Scala
Big Data
Pipelines
Hadoop
Data Management
ETL
Data Engineering
SQL
Distributed Systems
Python
Java
Data Analytics
Databases
Visual Analytics
C++

  • Singapore NTT Data Singapore Full time

    Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...


  • Singapore NTT Data Singapore Full time

    Job Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...

  • Big Data Engineer

    24 hours ago


    Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 6 - 8 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...


  • Singapore TANGSPAC CONSULTING PTE LTD Full time

    Roles & ResponsibilitiesAs a Data Framework Engineer at TANGSPAC CONSULTING PTE LTD, you will be responsible for creating frameworks and technical features that enable faster operationalisation of data models, analytical models, and user-generated contents. This includes partnering with data scientists to enable faster adoption of AL/ML model-based systems,...

  • Big Data Developer

    2 weeks ago


    Singapore PRIMESTAFF MANAGEMENT SERVICES PTE LTD Full time

    Roles & ResponsibilitiesOur client is looking for an experienced and highly skilled Senior Big Data Developer to join the dynamic team. The ideal candidate will have over 10 years of extensive working knowledge and hands-on expertise in big data technologies, especially in Spark on Scala or PySpark, and Hive. This role will involve the design, development,...


  • Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Job DescriptionWe are seeking an experienced Senior Big Data Engineer to join our team at RAPSYS TECHNOLOGIES PTE. LTD.Responsibilities:Design and develop data applications using Python and other programming languages, ensuring efficient data processing on big data systems.Create APIs using Python to handle structured and unstructured data, with a strong...


  • Singapore SPRINGFIELD CONSULTING PTE. LTD. Full time

    Job Title: Big Data EngineerWe are seeking an experienced Big Data Engineer to join our team at Springfield Consulting PTE. LTD.The estimated salary for this role is around SGD 120,000 - 180,000 per annum, depending on experience and qualifications.About the Job:Company Overview:Springfield Consulting PTE. LTD. is a reputable consulting firm that specializes...


  • Singapore USER EXPERIENCE RESEARCHERS PTE. LTD. Full time

    About the JobWe are seeking a highly skilled and experienced Big Data Engineer to join our team at USER EXPERIENCE RESEARCHERS PTE. LTD.With a minimum of 4 years of experience managing data engineering jobs in big data environments such as Cloudera Data Platform, the ideal candidate will be responsible for designing, developing, and maintaining the data...

  • Data Scientist

    1 month ago


    Singapore NTT Data Singapore Full time

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore.Key Responsibilities:Design and develop large-scale data processing systems using Hadoop and Spark.Implement data pipelines using Spring Boot and integrate data from multiple sources.Develop and maintain data warehouses using NoSQL databases such as HBase and...


  • Singapore CORPORATE IT SOLUTIONS PTE. LTD. Full time

    Unlock the Power of Big Data with Corporate IT Solutions PTE. LTD.We are seeking a highly skilled Senior Cloud Software Developer in Analytics and AI to join our dynamic and focused team. As an AI/ML Engineer, you will implement core software components and contribute to the scalable design of our cloud software architecture. This is an excellent opportunity...


  • Singapore COMBUILDER PTE LTD Full time

    Roles & ResponsibilitiesSummary:We are seeking a highly skilled and experienced Big Data Lead Architect to oversee the design, development, and deployment of big data solutions. The ideal candidate will have strong expertise in managing and architecting large-scale data platforms using advanced technologies such as Cloudera, Azure Cloud, and Kubernetes,...


  • Singapore Helius Full time

    Talend Big Data Developer RoleAt Helius, we are seeking a skilled Big Data Developer to join our team. The ideal candidate will have experience with Talend Big Data version 7+ and Hadoop, with a strong understanding of data models and data flows.Key Responsibilities:Design and develop production-grade Talend Big Data jobs with consideration for IT...

  • Data Scientist

    2 weeks ago


    Singapore NTT Data Singapore Full time

    RequirementsFor this role at NTT Data Singapore, we are looking for a highly skilled professional with a Bachelor's degree in Computer Science or a related field. The ideal candidate will have at least 7 years of IT experience, with a minimum of 5 years in a similar role, preferably in Big Data tools.Key ResponsibilitiesOur ideal candidate will have a strong...


  • Singapore Tencent Singapore Full time

    Responsibilities: Cloud & Smart Industries Group (CSIG) is responsible for promoting the company's cloud and industry Internet strategy. CSIG explores the interactions between users and industries to create innovative solutions for smart industries via technological advancements such as cloud, AI, and network security. While driving the digitalization of...


  • Singapore WEB SYNERGIES (S) PTE LTD Full time

    About WEB SYNERGIES (S) PTE LTDWe are a dynamic company seeking an experienced Big Data Architect to join our team.Job Summary:As a seasoned professional, you will be responsible for designing and implementing large-scale data architectures using Big Data technologies such as Hadoop and Spark. You will work closely with cross-functional teams to gather,...

  • Data Engineer

    2 weeks ago


    Singapore QUESS SELECTION & SERVICES PTE. LTD. Full time

    Roles & ResponsibilitiesFull time roleAttractive Annual leaves + Insurance coverageResponsibilities:· Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives· Collaborate with cross-functional teams to understand data requirements and translate them into effective data...


  • Singapore Tencent Singapore Full time

    Job Responsibilities:As a Senior Big Data Solution Architect at Tencent Singapore, you will be responsible for designing and implementing Big Data solutions tailored to customer business scenarios and technical requirements. You will need to deeply understand customers' needs and pain points regarding Big Data products and solutions.Recommend the core...

  • Data Engineer

    24 hours ago


    Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 4 - 6 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...

  • Bigdata engineer

    2 weeks ago


    Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesTo be responsible for managing technology in projects and providing technical guidance / solutions for work completion (1.) To be responsible for providing technical guidance / solutions ;define, advocate, and implement best practices and coding standards for the team. (2.) To develop and guide the team members in enhancing their...


  • Singapore THEE BIRAPANJAA SYSTEMS (PTE. LTD.) Full time

    Job OverviewThe role of a Data Engineer is a pivotal position within THEE BIRAPANJAA SYSTEMS (PTE. LTD.) that involves designing, developing, and maintaining large-scale data systems. As an expert in ETL (Extract, Transform, Load) processes, the successful candidate will have extensive knowledge of data extraction, transformation, and loading. This includes...