Big Data Engineer

Found in: Talent SG C2 - 1 week ago


Singapur, Singapore Virtusa Full time
Big Data Engineer - 111170 Description

Job Summary

We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. Candidate will also be integrating data sets to provide seamless data access to users.

Key Responsibilities

• Analyze the Authority’s data needs and document the requirements.

• Refine data collection/consumption by migrating data collection to more efficient channels.

• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.

• Develop test plan and scripts for system testing, support user acceptance testing.

• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.

• Ensure the smooth operations and service level of IT solutions.

• Support production issues - Track record in implementing systems with high availability, high performance, high security

What we are looking for

• Good understanding and completion of projects using waterfall/Agile methodology.

• Good understanding of analytics and data warehouse implementations.

• Ability to troubleshoot complex issues ranging from system resource to application stack traces.

• Strong SQL, data modelling and data analysis skills are a must.

• Hands-on experience in big data engineering jobs using Python, PySpark, Linux, and ETL tools like Informatica.

• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.

• Hands-on experience in DevOps deployment and data virtualization tools like Denodo will be preferred.

• Understanding/ Hands on experience of reporting or visualization tool like SAP BO and Tableau will be beneficial but not required.

• Passion for automation, standardization, and best practices

• Good written and verbal communication and interpersonal skills, ability to understand the business requirement, communicate confidently with stakeholders.

Primary Location Moulmein-Kallang, Central Singapore, Singapore Job Type Experienced Years of Experience 5 Qualification

Job Summary

We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as per business requirements. Candidate will also be integrating data sets to provide seamless data access to users.

Key Responsibilities

• Analyze the Authority’s data needs and document the requirements.

• Refine data collection/consumption by migrating data collection to more efficient channels.

• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.

• Develop test plan and scripts for system testing, support user acceptance testing.

• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.

• Ensure the smooth operations and service level of IT solutions.

• Support production issues - Track record in implementing systems with high availability, high performance, high security

What we are looking for

• Good understanding and completion of projects using waterfall/Agile methodology.

• Good understanding of analytics and data warehouse implementations.

• Ability to troubleshoot complex issues ranging from system resource to application stack traces.

• Strong SQL, data modelling and data analysis skills are a must.

• Hands-on experience in big data engineering jobs using Python, PySpark, Linux, and ETL tools like Informatica.

• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.

• Hands-on experience in DevOps deployment and data virtualization tools like Denodo will be preferred.

• Understanding/ Hands on experience of reporting or visualization tool like SAP BO and Tableau will be beneficial but not required.

• Passion for automation, standardization, and best practices

• Good written and verbal communication and interpersonal skills, ability to understand the business requirement, communicate confidently with stakeholders.

Travel No
  • Big Data Architect

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design and architect Big Data solutions to meet business requirements. Collaborate with stakeholders to understand their data and analytics needs. Create and maintain data models, data flow diagrams, and data integration strategies. Develop and implement data ingestion, processing, and storage solutions using Big Data technologies. Optimize data pipelines...

  • Site Reliability Engineer Lead

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore Shopee Full time

    Site Reliability Engineer Lead (Big Data) - Data Infrastructure DepartmentEngineering and TechnologyLevelExperienced (Team Lead)LocationSingapore The Engineering and Technology team is at the core of the Shopee platform development. The team is made up of a group of passionate engineers from all over the world, striving to build the best systems with the...

  • Big Data Development Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...

  • Big Data Development Engineer

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore TikTok Full time

    About the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...

  • Big Data Software Engineer, Ad Data

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore TikTok Full time

    Team Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...

  • Big Data Software Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore TikTok Full time

    Team Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...


  • Singapur, Singapore United Overseas Bank Full time

    VP, Data Scientist, Big Data Analytics Centre, Data Management Office Posting Date: 18-May-2023 Location: Singapore (City Area), Singapore, 048624 Company: United Overseas Bank Ltd About UOB United Overseas Bank Limited (UOB) is a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and...

  • AVP/Senior Assoc, Big Data Analyst

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore DBS Bank Full time

    Business Function The Transformation Group reports to the Group Head of Strategy, Transformation, Analytics & Research (GSTAR) who supports the CEO and senior management team on the implementation of the Group's strategic agenda, including working with the businesses and functions to drive strategic initiatives across geographies. The mandate of...

  • Data Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Build and optimizing ‘big data’ data pipelines, architectures and data sets. Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. To support and develop Data modeling, Implementation activities. Strong analytic skills related to working with...

  • Data Quality Assurance Engineer, Data Platform

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore TikTok Full time

    About the team The mission of the Data Business Partner (BP) Quality Assurance Team is to build a highly mature (standardized/professional/efficient) data quality system, ensuring the better delivery of data. Our goal is to safeguard the quality of data throughout the entire lifecycle in terms of timeliness, accuracy, and stability, empowering the...


  • Singapur, Singapore DBS Bank Full time

    Business Function Group Technology and Operations (T&O) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality & control, technology, people capability and innovation. In Group T&O, we manage the majority of the Bank's operational processes and inspire to delight our...

  • Data Engineer(SAS/Python)

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using SAS, Spark, Python, and PySpark. Ensure the smooth flow of data from various sources to destination systems. Utilize SAS, Spark, Python, and PySpark for efficient data processing. Design and implement data models to support business requirements. Work closely with data architects to...

  • Data Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...

  • Engineer Data Management

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore MISC Berhad Full time

    Description KEY ACCOUNTABILITIES Manage, plan and maintain the facilitation of support for data analysis across Eaglestar’s growing global fleet . Project-manage and deliver on data-related implementations ensuring that deliverables are met within agreed scope and timelines Manage, develop, and provide Eaglestar’s internal/external stakeholders...

  • Manager (Data Engineer)

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore IHiS Full time

    The Data Engineer supports the implementation of data structure and architecture, master/meta-data management approach and data quality programme to facilitate access to data and information. He/She support the design, implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time...

  • Data Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore IHiS Full time

    Data EngineerJob Role DescriptionThe Data Engineer supports the implementation of data structure and architecture, master/meta-data management approach and data quality programme to facilitate access to data and information. He/She support the design, implementation and maintenance of data flow channels and data processing systems that support the...

  • Data Program Lead

    Found in: Talent SG C2 - 2 weeks ago


    Singapur, Singapore Infosys Singapore & Australia Full time

    We are looking for a highly experienced Subject Matter Expert (SME) in Data Management (FS) to join our team. The ideal candidate will have 12-15 years of experience in data governance, financial service data solutions, and data engineering. They will also have strong solution and delivery skills, as well as a view on business growth and managing...

  • Data Engineer

    Found in: Talent SG C2 - 1 week ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using Spark, Python, and PySpark.Ensure the smooth flow of data from various sources to destination systems.Utilize SAS, Spark, Python, and PySpark for efficient data processing.Design and implement data models to support business requirements.Work closely with data architects to ensure...


  • Singapur, Singapore DBS Bank Full time

    Business Function Group Technology and Operations (T&O) enables and empowers the bank with an efficient, nimble and resilient infrastructure through a strategic focus on productivity, quality & control, technology, people capability and innovation. In Group T&O, we manage the majority of the Bank's operational processes and inspire to delight our...

  • Data Engineer(AI/ML)

    Found in: Talent SG 2 C2 - 7 days ago


    Singapur, Singapore MATRIX PROCESS AUTOMATION PTE. LTD. Full time

    Roles & ResponsibilitiesSupport MLOps projects. The person will be working on Machine Learning Servers and Data Robot along with the projects that require AI model deploymentRequirements:Software development experience in making Artificial Intelligence(AI) /Machine Learning(ML) modelsGood understanding of programming language such as Python, R and...