Lead Data Engineer
1 day ago
Your Role
As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.
Your main responsibilities will include:
· Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data.
· Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data.
· Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
· Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions.
· Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques.
· Optimize data engineering workflows for containerized deployment and efficient resource utilization.
· Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability.
· Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements.
· Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure.
· Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference.
· Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps.
· Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank.
Requirements
· Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
· At least 10 years of experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments.
· Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Ranger, etc)
· Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes.
· Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java.
· Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
· Experience with Grafana, Prometheus, Splunk will be an added benefit
· Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges.
· Excellent collaboration and communication skills to work effectively with cross-functional teams.
· Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment.
· Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus.
Tell employers what skills you have
Scalability
Scala
Azure
Pipelines
Hadoop
Automation Tools
Information Technology
Data Engineering
Logging
Python
Containerization
Ansible
Java
Technical Leadership
Orchestration
-
Data Engineer
4 weeks ago
Singapore NTT Data Singapore Full timeJob SummaryWe are seeking a highly skilled and experienced Big Data Engineer to join our team at NTT Data Singapore. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environments, such as Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/Senior Consultant preferably.Key...
-
Data Engineer
4 weeks ago
Singapore NTT Data Singapore Full timeJob SummaryWe are seeking a highly skilled and experienced Data Engineer to join our team at NTT Data Singapore.Key ResponsibilitiesAnalyze the Authority's data needs and document the requirements.Refine data collection/consumption by migrating data collection to more efficient channels.Plan, design and implement data engineering jobs and reporting solutions...
-
Data Engineering Lead
17 hours ago
Singapore REAL ESTATE ANALYTICS PTE. LTD. Full timeData Engineering LeadAt REAL ESTATE ANALYTICS PTE. LTD., we are seeking a highly skilled Data Engineering Lead to join our team.Key Responsibilities:We are looking for someone with strong experience in SQL programming, database design, and dealing with large volumes of data.The ideal candidate will have a solid understanding of data modeling, indexing...
-
Software Engineering Lead Trainer
5 months ago
Singapore Institute of Data Full timeAbout the Institute of DataThe Institute of Data is a Professional Network of Data Scientists, Cyber Security Analysts and Digital Marketers. In partnership with leading universities in Australia, New Zealand & America, the government of Singapore and hundreds of employers, we transform careers for a data-driven world.The Institute of Data operates in the...
-
Senior / Lead Data Engineer
1 day ago
Singapore VANGUARD SOFTWARE PTE. LTD. Full timeRoles & ResponsibilitiesJOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data...
-
Senior Big Data Engineer
1 week ago
Singapore NTT Data Singapore Full timeRequirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...
-
Senior Big Data Engineer
4 days ago
Singapore NTT Data Singapore Full timeJob Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...
-
Lead Data Engineer
1 month ago
Singapore D L RESOURCES PTE LTD Full timeRoles & ResponsibilitiesKey Skills:Quantexa certification mandatoryJob ObjectivesWe seek individuals with highly developed conceptual, strategic, and analytical skills, capable of striking a balance between visionary thinking and practical solutions. The ability to comprehend, inspire, and mobilize others is crucial. A business-oriented mindset...
-
Data Scientist
4 weeks ago
Singapore NTT Data Singapore Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore.Key Responsibilities:Design and develop large-scale data processing systems using Hadoop and Spark.Implement data pipelines using Spring Boot and integrate data from multiple sources.Develop and maintain data warehouses using NoSQL databases such as HBase and...
-
Senior Data Engineering Lead
5 days ago
Singapore Singtel Group Full timeWe are seeking a seasoned Data Engineering Lead to join our team at Singtel Group. As a key member of our Data Analytics Practice, you will be responsible for overseeing the implementation of data analytics strategies for our clients, providing team leadership, and guiding and coaching a Data & AI delivery squad.Key Responsibilities:Oversee the...
-
Data Analytics Lead
17 hours ago
Singapore NETWORK GUARD PTE. LTD. Full timeAbout the RoleWe are seeking a skilled Data Analytics Lead to join our team at NETWORK GUARD PTE. LTD. The successful candidate will be responsible for leading a group of data analysts, identifying skill gaps, and developing a team that can independently deliver high-quality advice, experimentation, analytics, and dashboards.Key Responsibilities:Manage,...
-
Singapore ARISTON SERVICES PTE. LTD. Full timeRoles & ResponsibilitiesYour RoleAs a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders...
-
Data Architect Lead
2 days ago
Singapore Infosys Singapore & Australia Full timeWe are seeking a highly skilled Data Architect to lead our data engineering efforts. The ideal candidate will have extensive experience in architecting solutions for data engineering, focusing on ELT and PySpark/Hadoop workloads.Key Responsibilities:Design and implement high-performance, scalable, and secure data architectures.Collaborate with business...
-
Data Solutions Engineer
17 hours ago
Singapore DATA CONNECT TECHNOLOGIES PTE. LTD. Full timeTechnical Consultant RoleAt DATA CONNECT TECHNOLOGIES PTE. LTD., we are seeking a skilled Technical Consultant to join our team. The ideal candidate will possess a strong understanding of systems administration, networking protocols, and cybersecurity principles.Pre-Sales Technical ServicesAssist potential clients in understanding our suite of products and...
-
Data Analytics Lead
17 hours ago
Singapore SMRT TRAINS LTD. Full timeJob SummaryAt SMRT TRAINS LTD., we are looking for a highly skilled Data Analytics Lead who can leverage Artificial Intelligence (AI) and data-driven insights to enhance workplace safety and operational efficiency. The ideal candidate will oversee the development and implementation of AI-integrated analytics strategies to optimize safety processes, identify...
-
Lead Engineer
1 day ago
Singapore SYNAPXE PTE. LTD. Full timeRoles & ResponsibilitiesThe candidate is part of the Data Loss Prevention (DLP) project and operation team. He/she will be reporting to the DLP Team Lead and be assigned with project and operational work and responsibilities which includes but not limited to, processing requests, investigate operational issues and facilitate security investigation by...
-
Lead Consultant, Data
3 months ago
Singapore Singtel Group Full timeAbout the job Are you looking for value adding and impactful work? Do you want to make a difference with your expertise? With us, you’ll be able to make it happen. NCS is the leading technology services firm, operating across Asia Pacific in over 20 cities, providing services and solutions in consulting, digital services,...
-
Data Analytics Lead
2 weeks ago
Singapore The Edge Partnership Full timeJob Title: Data Analytics LeadAt The Edge Partnership, we are seeking a highly skilled Data Analytics Lead to join our team. As a key member of our organization, you will be responsible for leading the development of our Information & Analytics products, leveraging your expertise in data preparation, insight generation, and visualization.Key...
-
Oliver Wyman
5 months ago
Singapore Oliver Wyman Group Full timeAbout Oliver WymanOliver Wyman is a global leader in management consulting. With offices in more than 70 cities across 30 countries, Oliver Wyman combines deep industry knowledge with specialized expertise in strategy, operations, risk management, and organization transformation. The firm has more than 6,000 professionals around the world who work with...
-
Lead Data Architect
1 day ago
Singapore DOCTOR WORLD PTE. LTD. Full timeRoles & ResponsibilitiesKey Responsibilities Design and implement data management solutions, data models, and database architectures Develop and maintain a comprehensive data strategy aligned with business goals Collaborate with stakeholders to understand data requirements and translate them into technical specifications Architect and implement robust...