Lead Data Engineer
3 weeks ago
Your Role
As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.
Your main responsibilities will include:
· Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.
· Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.
· Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
· Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.
· Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.
· Optimize data engineering workflows for containerized deployment and efficient resource utilization.
· Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.
· Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.
· Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.
· Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.
· Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.
· Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank.
Requirements
· Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
· At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments.
· Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)
· Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes.
· Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java.
· Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
· Experience with Grafana, Prometheus, Splunk will be an added benefit
· Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges.
· Excellent collaboration and communication skills to work effectively with cross-functional teams.
· Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment.
· Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus.
Tell employers what skills you have
Scalability
Scala
Azure
Pipelines
Hadoop
Automation Tools
Information Technology
Data Engineering
Logging
Python
Containerization
Ansible
Java
Technical Leadership
Orchestration
-
Strategic Data Insights Lead
5 days ago
Singapore NTT DATA Full timeTransform Business with NTT DATAAbout the Role:We are seeking a seasoned Strategic Data Insights Lead to join our team at NTT DATA. As a key member of our analytics capacity, you will be responsible for driving business value using data analytics, data visualization, and data modelling techniques and technologies.Key Responsibilities:Gather relevant...
-
Bioprocessing data engineer
16 hours ago
Singapore Data Privacy Full timeJob Description We're seeking a talentedBioprocess Data Engineerto join our team and play a crucial role in establishing the framework and systems for bioprocess data in a greenfield facility. As a Bioprocess Data Engineer, you'll bridge the gap between digital infrastructure and process engineers, creating a user-friendly interface for data...
-
Process engineer
2 days ago
Singapore The Chemical Engineer Full timeWhy Patients Need You Whether you are involved in the design and development of manufacturing processes for products or supporting maintenance and reliability, engineering is vital to making sure customers and patients have the medicines they need, when they need them. Working with our innovative engineering team, you'll help bring medicines to the...
-
Process engineer
2 days ago
Singapore The Chemical Engineer Full timePOSITION SUMMARY Incumbent will be responsible for leading product manufacturing campaigns from the start of production, and providing technical supervision with respect to quality and EHS until completion and release of the material.POSITION RESPONSIBILITIESDevelop Manufacturing/Operating Instructions and necessary Standard Operating Procedures.Develop,...
-
Senior technical engineer
2 weeks ago
Singapore The Chemical Engineer Full timeAbout us At Exxon Mobil, our vision is to lead in energy innovations that advance modern living and a net-zero future. As one of the world’s largest publicly traded energy and chemical companies, we are powered by a unique and diverse workforce fueled by the pride in what we do and what we stand for. The success of our Upstream, Product Solutions and Low...
-
Strategic Enterprise Data Architect Lead
5 days ago
Singapore NTT DATA Full timeJob OverviewThe Senior Principal Enterprise Architect at NTT DATA plays a pivotal role in making an impact with our company. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.This is...
-
Data Engineering Lead
3 days ago
Singapore Epam Full timeWe are looking for a highly skilled Data Engineering Lead to join our team at EPAM Singapore.About the RoleThis is an exciting opportunity to work on challenging projects and contribute to the growth of our data engineering capabilities. As a seasoned professional, you will lead the design, development, and maintenance of big data solutions using Cloudera...
-
Software Engineering Lead
3 days ago
Singapore NTT Data Singapore Full timeNTT Data Singapore seeks a highly skilled Software Engineering Lead to spearhead the development of our enterprise-wide visualization platform. The successful candidate will have 3-7 years of experience in system development and be proficient in Python, SQL, and Oracle.Key Responsibilities:Design and implement a continuous integration and deployment (CI/CD)...
-
Data Engineering Lead
3 weeks ago
Singapore REAL ESTATE ANALYTICS PTE. LTD. Full timeData Engineering LeadAt REAL ESTATE ANALYTICS PTE. LTD., we are seeking a highly skilled Data Engineering Lead to join our team.Key Responsibilities:We are looking for someone with strong experience in SQL programming, database design, and dealing with large volumes of data.The ideal candidate will have a solid understanding of data modeling, indexing...
-
Software Engineering Lead Trainer
6 months ago
Singapore Institute of Data Full timeAbout the Institute of DataThe Institute of Data is a Professional Network of Data Scientists, Cyber Security Analysts and Digital Marketers. In partnership with leading universities in Australia, New Zealand & America, the government of Singapore and hundreds of employers, we transform careers for a data-driven world.The Institute of Data operates in the...
-
Data Engineer Lead
3 weeks ago
Singapore Careers@Gov Full timeAbout the RoleCareers@Gov is seeking a highly skilled Data Engineer Lead to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data systems that support our organisation's analytical needs.Key ResponsibilitiesMaintain a data warehouse and analytics environmentDevelop...
-
Data Engineer Lead
2 weeks ago
Singapore Careers@Gov Full timeJob Title: Senior / Executive Data Engineer / Data EngineerAt Careers@Gov, we're seeking a highly skilled Data Engineer Lead to join our team of data experts. In this role, you'll play a crucial part in deriving business insights from large and real-time datasets to support strategic planning and operational efficiency.Key Responsibilities:Maintain and...
-
Senior / Lead Data Engineer
3 weeks ago
Singapore VANGUARD SOFTWARE PTE. LTD. Full timeRoles & ResponsibilitiesJOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data...
-
Senior Big Data Engineer
4 weeks ago
Singapore NTT Data Singapore Full timeRequirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...
-
Senior Big Data Engineer
4 weeks ago
Singapore NTT Data Singapore Full timeJob Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...
-
M & e manager / a1 main con > perm
2 weeks ago
Singapore Preferably Professional Engineer Full timeClient: MNC, A1 Main Contractor. Mega Projects Staff Benefits: 17 days annual leave, Year-End Bonus + VB. Salary: Negotiable Based on Qualification, Experience & Capability Location: Central Job Requirements: Degree in Electrical or Mechanical Engineering, recognized by PE Board Singapore/ BCA / IES Min 10 year experience in Mechanical & Electrical...
-
Lead Data Engineer
2 months ago
Singapore D L RESOURCES PTE LTD Full timeRoles & ResponsibilitiesKey Skills:Quantexa certification mandatoryJob ObjectivesWe seek individuals with highly developed conceptual, strategic, and analytical skills, capable of striking a balance between visionary thinking and practical solutions. The ability to comprehend, inspire, and mobilize others is crucial. A business-oriented mindset...
-
Frontend Software Engineer
5 days ago
Singapore Twelve Data Full timeAt Twelve Data, we're on a mission to make the world's financial data universally accessible. We aggregate and store vast volumes of data, leveraging custom terabytes-scale distributed time series databases to deliver high-speed performance.We're seeking an experienced Frontend Software Engineer to join our core development team. As a key member, you'll...
-
Data Engineering Lead for Aerospace Projects
3 days ago
Singapore ST Engineering Full timeJob Title: Data Engineering Lead for Aerospace ProjectsST Engineering, a leading player in the aerospace industry, is seeking a highly skilled Data Engineer to lead our data pipeline development efforts.About the RoleWe are looking for a seasoned professional with expertise in designing, executing, and maintaining critical data pipelines that support various...
-
Global Leader in Data Center Design
4 days ago
Singapore NTT DATA Full timeAbout the RoleWe are seeking an experienced Senior Director, Design Management to lead our team of design professionals and oversee the development of high-quality data center designs for our clients in Japan and APAC regions.Job DescriptionThe ideal candidate will have a strong background in designing critical facilities and complex infrastructure, with at...