Big Data Engineer

2 weeks ago

Singapore BYTEDANCE PTE. LTD. Full time
Roles & Responsibilities

About ByteDance

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join Us

Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible.

Together, we inspire creativity and enrich life - a mission we aim towards achieving every day.

To us, every challenge, no matter how ambiguous, is an opportunity; to learn, to innovate, and to grow as one team. Status quo? Never. Courage? Always.

At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve.

Join us.

About The Team

The batch processing team is responsible for the company's offline data processing and distributed training, supporting various business scenarios such as offline ETL and machine learning within the company. The components involved include the offline computing engine Spark, the in-house distributed training framework Primus, feature storage solutions like Iceberg and Hudi, as well as Ray, a next-generation distributed application framework. Faced with massive-scale scenarios, extensive functional and performance optimizations have been carried out in Spark, Primus, Feature Store, and support for the adoption of the new-generation distributed application framework Ray in relevant company scenarios.

What you will be doing:

- Responsible for the development and performance optimisation of the in-house Feature Store functionality based on Iceberg;

- Participant in optimisation of the integration of Iceberg with various upper-level computing engines;

- Involve in platform-related infrastructure development.

Qualifications

Minimum Qualifications

- Bachelor's Degree or above, majoring in Computer Science, or related fields, with 4+ years of relevant development experience in the field with a strong programming ability, and proficiency in Java, Python, C++, with the ability to develop and optimize large-scale distributed systems.

- In-depth research and relevant experience in one or more data lake formats such as Delta, Hudi, or Iceberg.

Preferred Qualifications

- In-depth research or practical experience in open-source big data computing frameworks and scenarios like Hadoop, Spark, Flink, Presto, and more.

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Tell employers what skills you have

Machine Learning
Scala
Big Data
Pipelines
Hadoop
Data Management
ETL
Data Engineering
SQL
Distributed Systems
Python
Java
Data Analytics
Databases
Visual Analytics
C++

  • Big Data Engineer

    2 weeks ago

    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesAre you passionate about big data and love working with cutting-edge technologies? Join our team as a Big Data Engineer and help us build scalable data solutions What You'll Do: Design, develop, and optimize big data pipelines Work with Apache Spark, Scala, Hadoop, Hive, Kafka, and more Manage and process large datasets efficiently...

  • Big Data Engineer

    2 weeks ago

    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesAre you passionate about big data and love working with cutting-edge technologies? Join our team as a Big Data Engineer and help us build scalable data solutions! What You’ll Do:✔ Design, develop, and optimize big data pipelines✔ Work with Apache Spark, Scala, Hadoop, Hive, Kafka, and more✔ Manage and process large datasets...

  • Big Data Engineer

    2 weeks ago

    Singapore INFINITE COMPUTER SOLUTIONS PTE LTD Full time

    Roles & ResponsibilitiesWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 9 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, building, maintaining and supporting big...

  • Big Data Engineer

    4 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor's or master's degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    2 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor's or master's degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    2 weeks ago

    Singapore INFINITE COMPUTER SOLUTIONS PTE LTD Full time

    Roles & ResponsibilitiesWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 9 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, building, maintaining and supporting big...

  • Big Data Engineer

    2 weeks ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesMy ClientIt is a global cryptocurrency exchange dedicated to providing secure, efficient, and innovative digital asset trading services. The platform offers a wide range of services, including spot trading, derivatives trading, and wealth management. They are looking for experienced and motivated Mobile app developer to join the...

  • Big Data Engineer

    2 weeks ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesMy ClientIt is a global cryptocurrency exchange dedicated to providing secure, efficient, and innovative digital asset trading services. The platform offers a wide range of services, including spot trading, derivatives trading, and wealth management. They are looking for experienced and motivated Mobile app developer to join the...

  • Big Data Engineer

    4 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor’s or master’s degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    4 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor’s or master’s degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    4 weeks ago

    Singapore MICHAEL PAGE INTERNATIONAL PTE LTD Full time

    Roles & Responsibilities Organisation is a global leader in the technology and telecoms industry. Fast-paced environment, rapidly growing organisation.About Our ClientThe organisation is a global leader in the technology and telecoms industry. They are well-known for their innovative approach to tackling complex technical challenges and their commitment to...

  • Big Data Engineer

    2 weeks ago

    Singapore MICHAEL PAGE INTERNATIONAL PTE LTD Full time

    Roles & ResponsibilitiesOrganisation is a global leader in the technology and telecoms industry. Fast-paced environment, rapidly growing organisation.About Our ClientThe organisation is a global leader in the technology and telecoms industry. They are well-known for their innovative approach to tackling complex technical challenges and their commitment to...

  • Big Data

    1 week ago

    Singapore EVO OUTSOURCING SOLUTIONS PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities:Collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient big data platform solutions. Design and architect robust big data platforms, ensuring high performance, reliability, and scalability. Implement end-to-end data processing workflows, from data...

  • Senior Big Data Engineer

    2 days ago

    Singapore TANGSPAC CONSULTING PTE LTD Full time

    Roles & ResponsibilitiesJob Description & RequirementsWorking in a large Big data and datawarehouse team of a bank, you will be responsible for carrying out the followingUnderstand functional requirements Carrying out data analysis, data mapping development of data pipeline, back end services performing data management and data transformation Ensuring...

  • Big Data Engineer

    4 weeks ago

    Singapore MINDMAP TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities· Perform the role of a Data engineer on the Enterprise Data Architecture and Governance (EDAG) Programs, to ensure the business requirements and deliverables are aligned to business priorities· Design develop data pipelines on EDAG platform to support Wholesale bank data model.· Perform assessments and analysis on...

  • Big Data Engineer

    2 weeks ago

    Singapore MINDMAP TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities· Perform the role of a Data engineer on the Enterprise Data Architecture and Governance (EDAG) Programs, to ensure the business requirements and deliverables are aligned to business priorities· Design develop data pipelines on EDAG platform to support Wholesale bank data model.· Perform assessments and analysis on...

  • Cloudera & Big Data Engineer

    1 week ago

    Singapore MANPOWER STAFFING SERVICES (SINGAPORE) PTE LTD Full time

    Roles & ResponsibilitiesJob scopes· Design, build, deployment, of ETL/ELT data pipelines within big data ecosystems ( Impala, Hue, Spark, Talend and Cloudera)· Develop technical design, and ETL specification documents· Manage, monitor and fine tune Hadoop cluster architecture (HDFS) jobs for performance, security and resource management.· Create detailed...

  • Big Data Engineer

    1 week ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesKey Responsibilities: Ensure seamless deployment, configuration, and operation of applications on cloud servers and container platforms. Collaborate with development teams to design scalable and highly available system architectures. Contribute to the development and maintenance of internal platforms and tools. Perform daily...

  • Big Data Engineer with Kubernetes Expertise

    2 days ago

    Singapore Tangspac Full time

    Company Overview:Tangspac is a leading provider of innovative solutions for the financial industry.We are seeking an experienced Big Data Engineer to join our team and contribute to the development of our cutting-edge big data applications.

  • Senior Big Data Engineer

    2 weeks ago

    Singapore PALO IT SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesWHO WE ARE:Build. Scale. Sustain.PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World...