Lead Data Engineer

4 weeks ago

Singapore D L RESOURCES PTE LTD Full time
Roles & Responsibilities

Key Skills:

Quantexa certification mandatory

Job Objectives

We seek individuals with highly developed conceptual, strategic, and analytical skills, capable of striking a balance between visionary thinking and practical solutions. The ability to comprehend, inspire, and mobilize others is crucial. A business-oriented mindset coupled with effective storytelling will drive your success. We are looking for self-starters ready to take on responsibilities with enthusiasm.

Key Responsibilities

As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution. You will be working with Quantexa platform.



Your main responsibilities will include:

• Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.

• Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.

• Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.

• Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.

• Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.

• Optimize data engineering workflows for containerized deployment and efficient resource utilization.

• Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.

• Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.

• Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.

• Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.

• Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.

• Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank.




Key Requirements

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.

• At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments.

• Must be Quantexa certified data engineer / data architect and proficient with the tool.

• Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)

• Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes.

• Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java.

• Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)

• Experience with Grafana, Prometheus, Splunk will be an added benefit

• Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges.

• Excellent collaboration and communication skills to work effectively with cross-functional teams.

• Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment.

• Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus.


Tell employers what skills you have

Scalability
Scala
Azure
Pipelines
Architect
Hadoop
Automation Tools
Information Technology
Data Engineering
Logging
Python
Containerization
Visionary Thinking
Java
Orchestration

  • The Chemical Engineer | Process Control Senior Technologist

    4 weeks ago

    Singapore The Chemical Engineer Full time

    What’s the role?Your primary accountabilities as a Process Control Senior Engineer include: Improve robustness of Base Layer control performance, delivery of high uptime and low operator intervention rates. Conduct daily base layer performance monitoring, provide technical support and lead troubleshooting for control and safeguarding related issues....

  • Data Engineer Specialist

    1 month ago

    Singapore NTT Data Singapore Full time

    NTT Data Singapore is seeking a highly skilled Data Engineer to join our team. This role will be responsible for designing, developing, and maintaining large-scale data systems.Job Description:Key Responsibilities: Design, develop, and maintain large-scale data systems using Scala programming and functional programming principles. Collaborate with...

  • Data Engineering Expert for Big Data Technologies

    3 weeks ago

    Singapore NTT Data Singapore Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore.This is an excellent opportunity for someone with experience in big data technologies to take their career to the next level and work on challenging projects.ResponsibilitiesDesign, develop, and deploy large-scale data processing systems using Hadoop ecosystem...

  • Lead Data Engineer

    3 weeks ago

    Singapore EPAM Systems Full time

    As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data...

  • Lead Data Engineer

    5 days ago

    Singapore SYNECHRON TECHNOLOGIES SINGAPORE PTE. LTD. Full time

    About Synechron: Synechron Singapore is a leading consulting and technology services firm specializing in the financial services sector. Our Singapore office focuses on delivering innovative digital transformation solutions to top-tier banks, insurers, and capital markets. We cultivate a diverse and inclusive environment where talent thrives, offering...

  • NTT DATA | Command Centre Operation Engineer

    2 days ago

    Singapore NTT DATA Full time

    Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at NTT DATA The Managed...

  • Data Engineering Lead

    4 weeks ago

    Singapore National University of Singapore Full time

    About NUS IT NUS Information Technology is the cornerstone to providing reliable, high-performance and secure IT solutions and effective IT governance for the campus. Here at NUS IT, we aim to transform NUS into a borderless computing community providing knowledge at its fingertips by enhancing the use of effective applications and services for teaching and...

  • Lead Data Engineer

    2 weeks ago

    Singapore SYNECHRON TECHNOLOGIES SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesAbout Synechron:Synechron Singapore is a leading consulting and technology services firm specializing in the financial services sector. Our Singapore office focuses on delivering innovative digital transformation solutions to top-tier banks, insurers, and capital markets. We cultivate a diverse and inclusive environment where talent...

  • Lead Data Engineer

    2 weeks ago

    Singapore SYNECHRON TECHNOLOGIES SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesAbout Synechron:Synechron Singapore is a leading consulting and technology services firm specializing in the financial services sector. Our Singapore office focuses on delivering innovative digital transformation solutions to top-tier banks, insurers, and capital markets. We cultivate a diverse and inclusive environment where talent...

  • M & e manager / a1 main con > perm

    2 months ago

    Singapore Preferably Professional Engineer Full time

    Client: MNC, A1 Main Contractor. Mega Projects Staff Benefits: 17 days annual leave, Year-End Bonus + VB. Salary: Negotiable Based on Qualification, Experience & Capability Location: Central Job Requirements: Degree in Electrical or Mechanical Engineering, recognized by PE Board Singapore/ BCA / IES Min 10 year experience in Mechanical & Electrical...

  • Aboitiz Data Innovation | Data Scientist

    3 weeks ago

    Singapore Aboitiz Data Innovation Full time

    Aboitiz Data Innovation (ADI)is one of the leading up and coming start-ups in the field of Data Science and Artificial Intelligence. We believe data can drive change for a better world by advancing businesses across industries and communities. We are looking for 2 x Mid-Senior Data Scientists to develop and deploy multiple machine learning models leveraging...

  • Data Engineering Lead

    1 month ago

    Singapore SOURCEO PTE. LTD. Full time

    Job Title: Data Engineering Lead - Big Data ExpertAbout the Role:We are seeking an experienced Data Engineering Lead to join our team at SOURCEO PTE. LTD. as a Big Data Expert.Responsibilities:The successful candidate will be responsible for guiding and de-risking the delivery of data projects, promoting iterative and incremental value addition. They will...

  • Senior/Lead Data Engineer

    5 days ago

    Singapore CHARLES & KEITH GROUP Full time

    Get AI-powered advice on this job and more exclusive features. Job Overview: The Senior/Lead Data Engineer is responsible for designing, developing, and optimizing scalable data solutions on Alibaba Cloud’s big data platforms, including MaxCompute and Hologres. This role will take a hands-on approach in building robust data pipelines, ensuring data...

  • Data Centre Business Development Lead

    1 month ago

    Singapore KEPPEL DATA CENTRES HOLDING PTE. LTD. Full time

    Job OverviewWe are seeking an experienced Data Centre Business Development Lead to join our team in Singapore. The successful candidate will drive data centre development projects in Singapore and overseas, contributing to the growth of our business.Estimated Salary: SGD 120,000 - 180,000 per annumThis is a challenging role that requires strong business...

  • Data Engineering and Operations Lead

    4 weeks ago

    Singapore United Overseas Bank Full time

    Join United Overseas Bank as a Data Engineering and Operations Lead to shape the future of data architecture and pipeline in support of AML uplift analytics needs. This role is part of our Compliance function, which plays a critical strategic partner and business enabler to the Board and senior management.As a Data Engineering and Operations Lead, you will...

  • Lead Data Visualization Engineer

    1 month ago

    Singapore PERSOLKELLY SINGAPORE PTE. LTD. Full time

    About the RolePERSOLKELLY SINGAPORE PTE. LTD. is seeking an experienced Lead Data Visualization Engineer to join our team. As a key member of our data engineering department, you will play a critical role in designing, developing, and implementing innovative data visualization solutions for our clients.Key Responsibilities• Design and develop complex data...

  • ECF Data, LLC | Data Scientist

    4 weeks ago

    Singapore ECF Data, LLC Full time

    Description: Location: Singapore Budget : 8000 -10000Responsibilities: 1. Drive analytic projects for all pillars of the bank’s treasury and markets department, such as bond analytics, news generation FX trading signals, product recommendation and customer hyper-personalization. 2. Conduct literature review and evaluate trade-offs between different...

  • Lead Engineer, Network Engineering

    5 days ago

    Singapore ICE DATA SERVICES SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesJob DescriptionIntercontinental Exchange, Inc. (ICE) presents a unique opportunity to work on cutting-edge technology and business challenges in the financial sector. The ICE Network Engineer provides leadership in network design, deployment, maintenance, and operational supports. The successful candidate will be able to multitask...

  • Lead Engineer, Network Engineering

    5 days ago

    Singapore ICE DATA SERVICES SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesJob DescriptionIntercontinental Exchange, Inc. (ICE) presents a unique opportunity to work on cutting-edge technology and business challenges in the financial sector. The ICE Network Engineer provides leadership in network design, deployment, maintenance, and operational supports. The successful candidate will be able to multitask...

  • Technical Lead

    1 month ago

    Singapore OPUS IT SERVICES PTE LTD Full time

    Roles & ResponsibilitiesThe Technical Lead will be a key member of the data engineering team. He/she will support the delivery team in designing technology solution to help address complex business and process challenges in client organisation. He/she will ensure the proper and smooth implementation of the technical design.Job Responsibilities· Collaborate...