Large Scale Data Storage Engineer

6 days ago

Singapore beBeeData Full time $80,000 - $120,000
Software Engineering Opportunity">

This role is responsible for crafting and implementing a storage solution for offline data in the recommendation system, catering to over a billion users.

The primary objectives are to guarantee system reliability, uninterrupted service, and seamless performance. The team aims to create a storage and computing infrastructure that can adapt to various data sources within the recommendation system, accommodating diverse storage needs.

The ultimate goal is to deliver efficient, affordable data storage with easy-to-use data management tools for the recommendation, search, and advertising functions.

Key Responsibilities:
  • Design and implement an offline/real-time data architecture for large-scale recommendation systems.
  • Design and implement a flexible, scalable, stable, and high-performance storage system and computation model.
  • Troubleshoot production systems, and design and implement necessary mechanisms and tools to ensure the overall stability of production systems.
  • Build industry-leading distributed systems such as offline and online storage, batch, and stream processing frameworks, providing reliable infrastructure for massive data and large-scale business systems.
Requirements:
  • Bachelor's Degree or above, majoring in Computer Science, or related fields, with 1+ years of experience building scalable systems;
  • Proficiency in common big data processing systems like Spark/Flink at the source code level is required, with a preference for experience in customizing or extending these systems;
  • A deep understanding of the source code of at least one data lake technology, such as Hudi, Iceberg, or DeltaLake, is highly valuable and should be prominently showcased in your resume, especially if you have practical implementation or customisation experience;
  • Knowledge of HDFS principles is expected, and familiarity with columnar storage formats like Parquet/ORC is an additional advantage;
  • Prior experience in data warehousing modeling;
  • Proficiency in programming languages such as Java, C++, and Scala is essential, along with strong coding skills and the ability to troubleshoot effectively;
  • Experience with other big data systems/frameworks like Hive, HBase, or Kudu is a plus;
  • A willingness to tackle challenging problems without clear solutions, a strong enthusiasm for learning new technologies, and prior experience in managing large-scale data (in the petabyte range) are all advantageous qualities.
Benefits:

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace.

We are passionate about this and hope you are too.

  • Large Scale Data Storage Engineer

    4 days ago

    Singapore beBeeData Full time

    Software Engineering Opportunity "> This role is responsible for crafting and implementing a storage solution for offline data in the recommendation system, catering to over a billion users. The primary objectives are to guarantee system reliability, uninterrupted service, and seamless performance. The team aims to create a storage and computing...

  • Large Scale Data Storage Engineer

    11 hours ago

    Singapore beBeeData Full time

    Software Engineering Opportunity "> This role is responsible for crafting and implementing a storage solution for offline data in the recommendation system, catering to over a billion users. The primary objectives are to guarantee system reliability, uninterrupted service, and seamless performance. The team aims to create a storage and computing...

  • Large Scale Data Architect

    1 day ago

    Singapore beBeeData Full time $120,000 - $180,000

    We are seeking a skilled Big Data Engineer to join our E-commerce Recommendation Infrastructure team.Job DescriptionThis role involves designing and implementing large-scale recommendation systems, ensuring high-performance storage and computing systems, and troubleshooting production issues. You will work closely with applied machine learning engineers to...

  • Large Scale Data Engineering Professional

    2 weeks ago

    Singapore beBeeDataEngineer Full time

    As a senior data engineer, you will be responsible for designing and developing large-scale data pipelines that meet the organization's needs. This involves collaborating with cross-functional teams to understand business requirements and ensuring the maintainability of these systems through rigorous testing. Key Skills and Qualifications 8+ years of...

  • Large Scale Data Engineering Professional

    3 days ago

    Singapore beBeeDataEngineering Full time $120,000 - $180,000

    Large Scale Data Engineering ExpertWe are seeking an experienced Large Scale Data Engineering Expert to join our team. As a key member of our team, you will play a vital role in designing and maintaining large-scale distributed systems.The ideal candidate will have hands-on experience with Spark (Core, SQL, Streaming), a good understanding of Hadoop (HDFS,...

  • Large Scale Data Engineering Professional

    21 hours ago

    Singapore beBeeDataEngineering Full time

    Large Scale Data Engineering Expert We are seeking an experienced Large Scale Data Engineering Expert to join our team. As a key member of our team, you will play a vital role in designing and maintaining large-scale distributed systems. The ideal candidate will have hands-on experience with Spark (Core, SQL, Streaming), a good understanding of Hadoop...

  • Large Scale Data Engineering Specialist

    5 days ago

    Singapore beBeeDataEngineer Full time $90,000 - $120,000

    Are you an accomplished data professional looking for a new challenge?Our ideal candidate will be responsible for architecting and developing large-scale data pipelines that meet our business needs.Design, develop, and maintain complex data systems using cutting-edge technologies.Collaborate with cross-functional teams to understand requirements and deliver...

  • Data Engineer

    3 days ago

    Singapore beBeeDataEngineer Full time $80,000 - $120,000

    Job DescriptionOur team is responsible for designing and implementing large-scale recommendation systems. We focus on optimizing the architecture to provide a stable and high-quality experience for users.We are looking for talented individuals to join our team. As a graduate, you will have opportunities to pursue bold ideas, tackle complex challenges, and...

  • Large Scale Data Infrastructure Specialist

    2 weeks ago

    Singapore beBeeDataInfrastructure Full time $80,000 - $120,000

    **System Architect for Large-Scale Recommendation Systems** Craft a robust data infrastructure to power offline recommendation systems serving over a billion users. Design an efficient architecture for real-time and offline data processing.Implement scalable storage solutions and computation models. Key Responsibilities:Develop large-scale distributed...

  • Data Architect for Large-Scale Recommendation Systems

    2 days ago

    Singapore beBeeRecommendation Full time $80,000 - $120,000

    About the RoleWe are seeking a seasoned Data Architect to lead the development of our recommendation system. This is an exceptional opportunity to leverage your expertise and passion for building scalable, stable, and high-performance recommendation systems.Key ResponsibilitiesDesign and implement offline data architectures for large-scale recommendation...