Lead Data Engineer

3 weeks ago


Singapore SYNECHRON TECHNOLOGIES SINGAPORE PTE. LTD. Full time
Roles & Responsibilities

Your Role

As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.

Your main responsibilities will include:

· Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.

· Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.

· Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.

· Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.

· Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.

· Optimize data engineering workflows for containerized deployment and efficient resource utilization.

· Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.

· Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.

· Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.

· Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.

· Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.

· Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank.


Requirements

· Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.

· At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments.

· Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)

· Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes.

· Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java.

· Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)

· Experience with Grafana, Prometheus, Splunk will be an added benefit

· Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges.

· Excellent collaboration and communication skills to work effectively with cross-functional teams.

· Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment.

· Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus.


Tell employers what skills you have

Scalability
Scala
Azure
Pipelines
Hadoop
Automation Tools
Information Technology
Data Engineering
Logging
Python
Containerization
Ansible
Java
Technical Leadership
Orchestration

  • Singapore NTT DATA Full time

    Transform Business with NTT DATAAbout the Role:We are seeking a seasoned Strategic Data Insights Lead to join our team at NTT DATA. As a key member of our analytics capacity, you will be responsible for driving business value using data analytics, data visualization, and data modelling techniques and technologies.Key Responsibilities:Gather relevant...


  • Singapore Data Privacy Full time

    Job Description We're seeking a talentedBioprocess Data Engineerto join our team and play a crucial role in establishing the framework and systems for bioprocess data in a greenfield facility. As a Bioprocess Data Engineer, you'll bridge the gap between digital infrastructure and process engineers, creating a user-friendly interface for data...

  • Process engineer

    2 days ago


    Singapore The Chemical Engineer Full time

    Why Patients Need You Whether you are involved in the design and development of manufacturing processes for products or supporting maintenance and reliability, engineering is vital to making sure customers and patients have the medicines they need, when they need them. Working with our innovative engineering team, you'll help bring medicines to the...

  • Process engineer

    2 days ago


    Singapore The Chemical Engineer Full time

    POSITION SUMMARY Incumbent will be responsible for leading product manufacturing campaigns from the start of production, and providing technical supervision with respect to quality and EHS until completion and release of the material.POSITION RESPONSIBILITIESDevelop Manufacturing/Operating Instructions and necessary Standard Operating Procedures.Develop,...


  • Singapore The Chemical Engineer Full time

    About us At Exxon Mobil, our vision is to lead in energy innovations that advance modern living and a net-zero future. As one of the world’s largest publicly traded energy and chemical companies, we are powered by a unique and diverse workforce fueled by the pride in what we do and what we stand for. The success of our Upstream, Product Solutions and Low...


  • Singapore NTT DATA Full time

    Job OverviewThe Senior Principal Enterprise Architect at NTT DATA plays a pivotal role in making an impact with our company. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.This is...


  • Singapore Epam Full time

    We are looking for a highly skilled Data Engineering Lead to join our team at EPAM Singapore.About the RoleThis is an exciting opportunity to work on challenging projects and contribute to the growth of our data engineering capabilities. As a seasoned professional, you will lead the design, development, and maintenance of big data solutions using Cloudera...


  • Singapore NTT Data Singapore Full time

    NTT Data Singapore seeks a highly skilled Software Engineering Lead to spearhead the development of our enterprise-wide visualization platform. The successful candidate will have 3-7 years of experience in system development and be proficient in Python, SQL, and Oracle.Key Responsibilities:Design and implement a continuous integration and deployment (CI/CD)...

  • Data Engineering Lead

    3 weeks ago


    Singapore REAL ESTATE ANALYTICS PTE. LTD. Full time

    Data Engineering LeadAt REAL ESTATE ANALYTICS PTE. LTD., we are seeking a highly skilled Data Engineering Lead to join our team.Key Responsibilities:We are looking for someone with strong experience in SQL programming, database design, and dealing with large volumes of data.The ideal candidate will have a solid understanding of data modeling, indexing...


  • Singapore Institute of Data Full time

    About the Institute of DataThe Institute of Data is a Professional Network of Data Scientists, Cyber Security Analysts and Digital Marketers. In partnership with leading universities in Australia, New Zealand & America, the government of Singapore and hundreds of employers, we transform careers for a data-driven world.The Institute of Data operates in the...

  • Data Engineer Lead

    3 weeks ago


    Singapore Careers@Gov Full time

    About the RoleCareers@Gov is seeking a highly skilled Data Engineer Lead to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data systems that support our organisation's analytical needs.Key ResponsibilitiesMaintain a data warehouse and analytics environmentDevelop...

  • Data Engineer Lead

    2 weeks ago


    Singapore Careers@Gov Full time

    Job Title: Senior / Executive Data Engineer / Data EngineerAt Careers@Gov, we're seeking a highly skilled Data Engineer Lead to join our team of data experts. In this role, you'll play a crucial part in deriving business insights from large and real-time datasets to support strategic planning and operational efficiency.Key Responsibilities:Maintain and...


  • Singapore VANGUARD SOFTWARE PTE. LTD. Full time

    Roles & ResponsibilitiesJOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data...


  • Singapore NTT Data Singapore Full time

    Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...


  • Singapore NTT Data Singapore Full time

    Job Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...


  • Singapore Preferably Professional Engineer Full time

    Client: MNC, A1 Main Contractor. Mega Projects Staff Benefits: 17 days annual leave, Year-End Bonus + VB. Salary: Negotiable Based on Qualification, Experience & Capability Location: Central Job Requirements: Degree in Electrical or Mechanical Engineering, recognized by PE Board Singapore/ BCA / IES Min 10 year experience in Mechanical & Electrical...

  • Lead Data Engineer

    2 months ago


    Singapore D L RESOURCES PTE LTD Full time

    Roles & ResponsibilitiesKey Skills:Quantexa certification mandatoryJob ObjectivesWe seek individuals with highly developed conceptual, strategic, and analytical skills, capable of striking a balance between visionary thinking and practical solutions. The ability to comprehend, inspire, and mobilize others is crucial. A business-oriented mindset...


  • Singapore Twelve Data Full time

    At Twelve Data, we're on a mission to make the world's financial data universally accessible. We aggregate and store vast volumes of data, leveraging custom terabytes-scale distributed time series databases to deliver high-speed performance.We're seeking an experienced Frontend Software Engineer to join our core development team. As a key member, you'll...


  • Singapore ST Engineering Full time

    Job Title: Data Engineering Lead for Aerospace ProjectsST Engineering, a leading player in the aerospace industry, is seeking a highly skilled Data Engineer to lead our data pipeline development efforts.About the RoleWe are looking for a seasoned professional with expertise in designing, executing, and maintaining critical data pipelines that support various...


  • Singapore NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Director, Design Management to lead our team of design professionals and oversee the development of high-quality data center designs for our clients in Japan and APAC regions.Job DescriptionThe ideal candidate will have a strong background in designing critical facilities and complex infrastructure, with at...