Big Data Engineer

1 month ago


Singapur, Singapore Virtusa Full time
Big Data Engineer - 111170 Description

Job Summary

We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. Candidate will also be integrating data sets to provide seamless data access to users.

Key Responsibilities

• Analyze the Authority’s data needs and document the requirements.

• Refine data collection/consumption by migrating data collection to more efficient channels.

• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.

• Develop test plan and scripts for system testing, support user acceptance testing.

• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.

• Ensure the smooth operations and service level of IT solutions.

• Support production issues - Track record in implementing systems with high availability, high performance, high security

What we are looking for

• Good understanding and completion of projects using waterfall/Agile methodology.

• Good understanding of analytics and data warehouse implementations.

• Ability to troubleshoot complex issues ranging from system resource to application stack traces.

• Strong SQL, data modelling and data analysis skills are a must.

• Hands-on experience in big data engineering jobs using Python, PySpark, Linux, and ETL tools like Informatica.

• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.

• Hands-on experience in DevOps deployment and data virtualization tools like Denodo will be preferred.

• Understanding/ Hands on experience of reporting or visualization tool like SAP BO and Tableau will be beneficial but not required.

• Passion for automation, standardization, and best practices

• Good written and verbal communication and interpersonal skills, ability to understand the business requirement, communicate confidently with stakeholders.

Primary Location Moulmein-Kallang, Central Singapore, Singapore Job Type Experienced Years of Experience 5 Qualification

Job Summary

We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. Candidate will also be integrating data sets to provide seamless data access to users.

Key Responsibilities

• Analyze the Authority’s data needs and document the requirements.

• Refine data collection/consumption by migrating data collection to more efficient channels.

• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.

• Develop test plan and scripts for system testing, support user acceptance testing.

• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.

• Ensure the smooth operations and service level of IT solutions.

• Support production issues - Track record in implementing systems with high availability, high performance, high security

What we are looking for

• Good understanding and completion of projects using waterfall/Agile methodology.

• Good understanding of analytics and data warehouse implementations.

• Ability to troubleshoot complex issues ranging from system resource to application stack traces.

• Strong SQL, data modelling and data analysis skills are a must.

• Hands-on experience in big data engineering jobs using Python, PySpark, Linux, and ETL tools like Informatica.

• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.

• Hands-on experience in DevOps deployment and data virtualization tools like Denodo will be preferred.

• Understanding/ Hands on experience of reporting or visualization tool like SAP BO and Tableau will be beneficial but not required.

• Passion for automation, standardization, and best practices

• Good written and verbal communication and interpersonal skills, ability to understand the business requirement, communicate confidently with stakeholders.

Travel No
  • Big Data Engineer

    2 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer Description Responsibilities Good understanding and completion of projects using waterfall/Agile methodology. Good understanding of analytics and data warehouse implementations. Ability to troubleshoot complex issues ranging from system resource to application stack traces. Strong SQL, data modelling and data analysis skills are a must....

  • Big Data Architect

    2 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. Optimize data pipelines...

  • Big Data Architect

    1 month ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. Optimize data pipelines...

  • Big Data Engineer

    4 weeks ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...


  • Singapur, Singapore TikTok Full time

    About the team This role is a member of our Data Platform team focusing on Data products and solutions to enterprise services, leveraging TikTok's cutting-edge data infrastructure and data application. You will be helping enterprise customers to build their next generation data warehouse and data analytics, enabling them to do things that were not possible...


  • Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...


  • Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...

  • Software Engineer

    4 weeks ago


    Singapur, Singapore Epam Full time

    Description We are seeking a highly skilled and experienced Big Data Engineer to join our team. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. The candidate will also be integrating data sets to provide seamless data access to users. What...

  • Big Data Engineer

    1 month ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...


  • Singapur, Singapore TIKTOK PTE. LTD. Full time

    ResponsibilitiesAbout TikTokTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul, and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore TIKTOK PTE. LTD. Full time

    ResponsibilitiesAbout TikTokTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul, and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help...

  • Big Data Engineer

    2 weeks ago


    Singapur, Singapore TikTok Full time

    Team Introduction The mission of the Data Platform Singapore Business Partnering (DPSG BP) team is to empower the TikTok Business with data. Our goal is to build a Data Warehouse that can cater to batch and streaming data, Data Products that provide useful information to build efficient data metrics & dashboards which will be used to make smarter business...


  • Singapur, Singapore TIKTOK PTE. LTD. Full time

    TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo.Why Join UsCreation is the core of TikTok's purpose. Our platform is built to help imaginations thrive. This is doubly...


  • Singapur, Singapore DBS Bank Full time

    Business Function Group Technology and Operations (T&O) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality & control, technology, people capability and innovation. In Group T&O, we manage the majority of the Bank's operational processes and inspire to delight our...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore Lynx Analytics Full time

    Apply Now Data Engineer at Lynx Analytics (View all jobs) Singapore COMPANY OVERVIEW Founded in 2010, Lynx Analytics is a predictive analytics company run by world-class quantitative marketing scientists and industry-experienced data scientists. Our focus is to become a leading analytics solution provider in our chosen fields of expertise (telecom,...


  • Singapur, Singapore United Overseas Bank Full time

    VP, Data Scientist, Big Data Analytics Centre, Data Management Office Posting Date: 18-May-2023 Location: Singapore (City Area), Singapore, 048624 Company: United Overseas Bank Ltd About UOB United Overseas Bank Limited (UOB) is a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and...

  • Data Engineer

    2 days ago


    Singapur, Singapore Amazon Asia-Pacific Holdings Private Limited - D43 Full time

    Do you want to contribute to a team working on cutting edge technology, solve new problems that didn’t exist before, and have the ability to see the impact of your successes?Amazon is shaping the future of digital video entertainment. We seek Data Engineers who can help us store, process and analyze vast amounts of data as we build products in the video...

  • Data Engineer

    2 weeks ago


    Singapur, Singapore CommScope Full time

    In our 'always on' world, we believe it's essential to have a genuine connection with the work you do. RUCKUS Networks is hiring a Data Engineer who will work closely with our team in Singapore How You'll Help Us Connect the World You will join a dynamic and focused team to develop innovative big data applications in Analytics and Artificial...


  • Singapur, Singapore DBS Bank Full time

    Business Function The Transformation Group reports to the Group Head of Strategy, Transformation, Analytics & Research (GSTAR) who supports the CEO and senior management team on the implementation of the Group's strategic agenda, including working with the businesses and functions to drive strategic initiatives across geographies. The mandate of...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore CommScope Full time

    In our 'always on' world, we believe it's essential to have a genuine connection with the work you do. RUCKUS Networks is hiring a Data Engineer who will work closely with our team in Singapore! How You'll Help Us Connect the World You will join a dynamic and focused team to develop innovative big data applications in Analytics and Artificial...