Lead Data Engineer

2 weeks ago


Singapore HAYS SPECIALIST RECRUITMENT PTE. LTD. Full time
Roles & Responsibilities

About the job

A Software solution company is looking for a highly skilled and experienced Lead Data Engineer to join their dynamic team. The team is passionate about leveraging advanced analytics and data-driven strategies to deliver exceptional solutions to the clients.

Your new company

As a Lead Data Engineer, you will play a pivotal role in designing, building, and optimizing our data infrastructure to support the advanced analytics needs. You will lead a team of talented data engineers and collaborate closely with data analysts, the DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solutions.

Your new role

As the Lead Data Engineer, you will be responsible for:

  • Design, develop, and implement Spark Scala applications and data processing pipelines to handle large volumes of structured and unstructured data.
  • Integrate Elasticsearch with Spark to enable efficient data indexing, querying, and retrieval.
  • Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
  • Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.
  • Design and deploy data engineering solutions on the OpenShift Container Platform (OCP) using containerization and orchestration techniques.
  • Optimize data engineering workflows for containerized deployment and efficient resource utilization.
  • Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.
  • Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.
  • Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.
  • Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.
  • Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.
  • Provide technical leadership, mentorship, and guidance to junior team members, fostering a culture of continuous learning and innovation to enhance the bank's analytics capabilities.

What you need

  • Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field
  • At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments
  • Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc.)
  • Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes
  • Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java
  • Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
  • Experience with Grafana, Prometheus, and Splunk is a plus
  • Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus
  • Develop and validate hypotheses for threat hunting
  • Generate insights and provide supporting evidence through Big Data Analytics for informed decision-making
  • Assemble and disseminate intelligence to peers for swift intervention actions
  • Data Management to ensure that data is stored securely and efficiently, following best practices and compliance regulations
  • Continuously improving data processes and methodologies to enhance the quality and efficiency of data analysis

What you'll get in return

In return, you'll be part of an organization that values its employees. You'll be rewarded with:

  • structured career growth and plenty of developmental opportunities
  • Hybrid and stable working environment

What you need to do now

If you're interested in this role, click 'apply now' or for more information and a confidential discussion on this role or to find out about more opportunities in Technology contact Yuki Cheung or email yuki.cheung@hays.com.sg. Referrals are welcome.

At Hays, we value diversity and are passionate about placing people in a role where they can flourish and succeed. We actively encourage people from diverse background to apply.

EA Reg Number: R22110258 | EA License Number: 07C3924 | Company Registration No: 200609504D

Tell employers what skills you have

Apache Spark
Scalability
Scala
Data Analysis
Azure
Pipelines
Hadoop
Automation Tools
Big Data Analytics
Information Technology
Data Engineering
Spark
Logging
Python
Containerization
Java
Orchestration
  • Data Engineer

    2 days ago


    Singapore NTT DATA SINGAPORE PTE. LTD. Full time

    Roles & Responsibilities EMAIL ID : Interested candidates may also send their resume via email to mike.ramos@nttdata.comOnly shortlisted candidates would be contacted for interview.Role: Data Engineer - 12 months Renewable contractExperience: Minimum of 3 yearsLocation : Changi Business ParkRequirements:Min 3-4 years experience At least one year of...

  • Lead Data Engineer

    3 weeks ago


    Singapore VANGUARD SOFTWARE PTE. LTD. Full time

    Roles & ResponsibilitiesJOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data...


  • Singapore INFOGAIN SOLUTIONS PTE. LIMITED Full time

    Roles & ResponsibilitiesRole: Quantexa Lead Data EngineerJob Requisition Number: IS42654Job Level: More than 6 years of experience (L3/L4)Location: SingaporeKey Responsibilities:As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the...

  • Lead Data Engineer

    6 days ago


    Singapore AURUMJIN CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesDesign, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data. Optimize and tune Spark jobs for performance and scalability, ensuring efficient data...

  • Data Engineer

    3 weeks ago


    Singapore NTT DATA BUSINESS SOLUTIONS SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesYour career at NTT DATA Business SolutionsEmpowering You – to feel our passion for SAP / Analytic Solutions / ServiceNowAs NTT DATA Business Solutions, we are more than just a company. We are a team of innovative and passionate people who specialize in creating value-added SAP and Analytic solutions and guiding our customers on...

  • Lead Data Scientist

    11 hours ago


    Singapore OCBC Full time

    About the RoleThe Data Science Lead will play a crucial role in driving data-driven decision making across the OCBC Group. You will work closely with business stakeholders to understand their problem statements and transform them into ML and AI challenges when needed. Your expertise will enable the development and deployment of AI and Gen AI applications for...


  • Singapore RANDSTAD PTE. LIMITED Full time

    Roles & Responsibilitiesabout the companyA Singapore-based game publisher dedicated to bringing high-quality casual games to a global audience. Seeking a Senior Data Engineer to design and manage data pipelines and infrastructure for our advanced user growth platform. This role will ensure smooth data integration and enable real-time analytics...

  • Data Science Lead

    16 hours ago


    Singapore OCBC Full time

    OVERALL ROLE PURPOSE: • Working within the Group Data Science team, you will leverage huge volumes of Structured and Unstructured data to solve real business problems across the OCBC Group. • You will work closely with business to understand their problem statements and transform them into ML and AI challenges when needed. • You will work on AI and...

  • Data Engineer

    2 weeks ago


    Singapore ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. Full time

    Roles & ResponsibilitiesJob Title: Data Engineer - Mid-level/SeniorLocation: Singapore (CBD)Job Type: Full-Time PermanentKey Responsibilities:Design and maintain scalable ETL pipelines using Informatica. Optimize SQL queries and stored procedures for data manipulation. Lead data migration projects ensuring data integrity and quality. Collaborate with...

  • Data Science Lead

    3 days ago


    Singapore OCBC Full time

    OVERALL ROLE PURPOSE: • Working within the Group Data Science team, you will leverage huge volumes of Structured and Unstructured data to solve real business problems across the OCBC Group. • You will work closely with business to understand their problem statements and transform them into ML and AI challenges when needed. • You will work on AI and...

  • Lead Data Scientist

    2 weeks ago


    Singapore ARCBLUE SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesCompany OverviewThe hiring company is a leading media company offering an extensive array of digital entertainment platforms. With a strong reputation in Asia and in order to stay ahead of consumer demands, they are currently looking for an experienced Lead Data Scientist to join their Consumer Insights & Analytics team to play a key...


  • Singapore QUINNOX SOLUTIONS PTE. LTD. Full time

    Roles & ResponsibilitiesKey Responsibilities:Provide expertise in mechanical systems for data centres, focusing on ACMV, hydraulic systems, and thermodynamics refrigeration cycles. Perform engineering calculations, including cooling load, hydraulic analysis, and equipment sizing (chillers, pumps, cooling towers, air handlers). ...


  • Singapore Data Intellect Full time

    Data Intellect is a data and technology consultancy firm that's constantly pushing the boundaries of what's possible. We're seeking an Executive Director of Client Development who can help us drive business expansion in APAC.The ideal candidate will have a strong understanding of the Services industry, particularly Financial Services, and be able to advise...

  • Data Engineer

    3 weeks ago


    Singapore NTT DATA BUSINESS SOLUTIONS SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesYour career at NTT DATA Business SolutionsEmpowering You – to feel our passion for SAP / Analytic Solutions / ServiceNowAs NTT DATA Business Solutions, we are more than just a company. We are a team of innovative and passionate people who specialize in creating value-added SAP and Analytic solutions and guiding our customers on...

  • Lead Data Architect

    3 weeks ago


    Singapore DOCTOR WORLD PTE. LTD. Full time

    Roles & ResponsibilitiesKey ResponsibilitiesDesign and implement data management solutions, data models, and database architectures Develop and maintain a comprehensive data strategy aligned with business goals Collaborate with stakeholders to understand data requirements and translate them into technical specifications Architect and implement robust...

  • Lead Data Architect

    3 weeks ago


    Singapore DOCTOR WORLD PTE. LTD. Full time

    Roles & ResponsibilitiesKey Responsibilities Design and implement data management solutions, data models, and database architectures Develop and maintain a comprehensive data strategy aligned with business goals Collaborate with stakeholders to understand data requirements and translate them into technical specifications Architect and implement robust...

  • Test Lead

    3 weeks ago


    Singapore NTT DATA SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesEmployer: NTT DATA Singapore, 12 months Contract.NTT DATA Singapore (Ranked as the world's 6th most valuable IT services provider)Role : Test Lead (Automation ) Work Location : Shenton Way, SGX CentreSalary : $ 6000 - 8000Client: One of Southeast Asia's leading financial services providers, specializing in digital insurance and...

  • Data Science Lead

    2 weeks ago


    Singapore TIKTOK PTE. LTD. Full time

    Roles & ResponsibilitiesTikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join UsAt TikTok, our people are humble, intelligent, compassionate and creative. We create...


  • Singapore Crypto Full time

    The Data Operations and Management Lead at Crypto will be responsible for overseeing the development and maintenance of our data warehouse, ensuring the accuracy, integrity, and availability of our data.This role involves collaborating with cross-functional teams to collect, analyze, and maintain data, driving informed decision-making and business growth.The...

  • Data Engineer

    6 days ago


    Singapore DELTA SG CONSULTING Full time

    Roles & ResponsibilitiesLocation: SingaporeJob Type: Renewable Contract (Work onsite with a leading bank)Experience Level: Associate LevelWe are seeking a Data Engineer to design, develop, and implement data solutions leveraging Quantexa's cutting-edge technology. The ideal candidate will have strong expertise in data engineering, ETL pipelines, and big...