Big Data Engineer

4 weeks ago


Singapur, Singapore Capgemini Full time

Roles and Responsibilities:

Develop Spark Scala and PySpark jobs for data transformation and aggregation. 

Create unit tests for Spark transformations and helper methods. 

Utilize Spark and Spark SQL to read parquet data and create tables in Hive using the Scala API. 

Collaborate closely with the Business Analysts team to review test results and obtain sign-off. 

Prepare design and operations documentation for future reference. 

Conduct peer code quality reviews and act as a gatekeeper for quality checks. 

Engage in hands-on coding, often in a pair programming environment. 

Work within highly collaborative teams to build high-quality code. 

Qualifications and Requirements:

4-10 years of experience as a Hadoop data engineer. 

Strong expertise in Hadoop, Spark, Scala, PySpark, Python, Hive, and Impala. 

Familiarity with Oracle, Spark streaming, Kafka, and machine learning (ML). 

Proficiency in Agile methodologies, CI/CD, Git, Jenkins, and Cloudera Distribution. 

Solid understanding of data structures, data manipulation, distributed processing, application development, and automation. 

Experience in Core Banking or Finance domains is desirable. 

Cloud experience with AWS is a plus. 


  • Big Data Engineer

    4 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer - 111170 Description Job Summary We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer - 123101 Description Responsibilities Good understanding and completion of projects using waterfall/Agile methodology. • Good understanding of analytics and data warehouse implementations. • Ability to troubleshoot complex issues ranging from system resource to application stack traces. • Strong SQL, data modelling and data...

  • Big Data Engineer

    2 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer Description Responsibilities Good understanding and completion of projects using waterfall/Agile methodology. Good understanding of analytics and data warehouse implementations. Ability to troubleshoot complex issues ranging from system resource to application stack traces. Strong SQL, data modelling and data analysis skills are a must....

  • Big Data Architect

    2 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. Optimize data pipelines...

  • Big Data Architect

    4 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. Optimize data pipelines...

  • Big Data Engineer

    4 weeks ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...

  • Software Engineer

    4 weeks ago


    Singapur, Singapore Epam Full time

    Description We are seeking a highly skilled and experienced Big Data Engineer to join our team. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. The candidate will also be integrating data sets to provide seamless data access to users. What...


  • Singapur, Singapore TikTok Full time

    About the team This role is a member of our Data Platform team focusing on Data products and solutions to enterprise services, leveraging TikTok's cutting-edge data infrastructure and data application. You will be helping enterprise customers to build their next generation data warehouse and data analytics, enabling them to do things that were not possible...


  • Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...


  • Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...

  • Big Data Engineer

    4 weeks ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...


  • Singapur, Singapore TIKTOK PTE. LTD. Full time

    ResponsibilitiesAbout TikTokTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul, and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore TIKTOK PTE. LTD. Full time

    ResponsibilitiesAbout TikTokTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul, and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help...

  • Big Data Engineer

    2 weeks ago


    Singapur, Singapore TikTok Full time

    Team Introduction The mission of the Data Platform Singapore Business Partnering (DPSG BP) team is to empower the TikTok Business with data. Our goal is to build a Data Warehouse that can cater to batch and streaming data, Data Products that provide useful information to build efficient data metrics & dashboards which will be used to make smarter business...


  • Singapur, Singapore TIKTOK PTE. LTD. Full time

    TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help imaginations thrive. This is doubly...


  • Singapur, Singapore DBS Bank Full time

    Business Function Group Technology and Operations (T&O) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality & control, technology, people capability and innovation. In Group T&O, we manage the majority of the Bank's operational processes and inspire to delight our...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore Lynx Analytics Full time

    Apply Now Data Engineer at Lynx Analytics (View all jobs) Singapore COMPANY OVERVIEW Founded in 2010, Lynx Analytics is a predictive analytics company run by world-class quantitative marketing scientists and industry-experienced data scientists. Our focus is to become a leading analytics solution provider in our chosen fields of expertise (telecom,...


  • Singapur, Singapore United Overseas Bank Full time

    VP, Data Scientist, Big Data Analytics Centre, Data Management Office Posting Date: 18-May-2023 Location: Singapore (City Area), Singapore, 048624 Company: United Overseas Bank Ltd About UOB United Overseas Bank Limited (UOB) is a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and...

  • Data Engineer

    2 hours ago


    Singapur, Singapore Amazon Asia-Pacific Holdings Private Limited - D43 Full time

    Do you want to contribute to a team working on cutting edge technology, solve new problems that didn’t exist before, and have the ability to see the impact of your successes?Amazon is shaping the future of digital video entertainment. We seek Data Engineers who can help us store, process and analyze vast amounts of data as we build products in the video...

  • Data Engineer

    2 weeks ago


    Singapur, Singapore CommScope Full time

    In our 'always on' world, we believe it's essential to have a genuine connection with the work you do. RUCKUS Networks is hiring a Data Engineer who will work closely with our team in Singapore How You'll Help Us Connect the World You will join a dynamic and focused team to develop innovative big data applications in Analytics and Artificial...