Lead Data Engineer

3 weeks ago


Singapore Transformhub Consulting Full time

Required Qualifications:

  • Bachelor qualification in a computer science or STEM (science, technology, engineering, or mathematics) related field.
  • At least 8+ years of strong data warehousing experience using RDBMS and Non-RDBMS databases.
  • At least 5 years of recent hands-on professional experience (actively coding) working as a data engineer (back-end software engineer considered).
  • Professional experience working in an agile, dynamic and customer facing environment is required.
  • Understanding of distributed systems and cloud technologies (AWS) is highly preferred.
  • Understanding of data streaming and scalable data processing is preferred to have.
  • Experience with large scale datasets, data lake and data warehouse technologies on at least TB scale (ideally PB scale of datasets) (Preferred: Snowflake)
  • At least 2+ years of experience in ETL (AWS Glue), Amazon S3, Amazon Redshift, Amazon RDS, Amazon Kinesis, Amazon Lambda, Apache Airflows, Amazon Step Functions.
  • At least 2+ experience in Snowflake is highly preferred.
  • Strong knowledge in scripting languages like Python, UNIX shell and Spark is required.
  • Understanding of RDBMS, Data ingestions, Data flows, Data Integrations etc.
  • Technical expertise with data models, data mining and segmentation techniques.
  • Experience with full SDLC lifecycle and Lean or Agile development methodologies.
  • Knowledge of CI/CD and GIT Deployments.
  • Ability to work in team in diverse/ multiple stakeholder environment.
  • Ability to communicate complex technology solutions to diverse teams namely, technical, business and management teams.

Responsibilities:

  • Work with stakeholders to understand needs for data structure, availability, scalability, and accessibility.
  • Develop tools to improve data flows between internal/external systems and the data lake/warehouse.
  • Build robust and reproducible data ingest pipelines to collect, clean, harmonize, merge, and consolidate data sources.
  • Understanding existing data applications and infrastructure architecture
  • Build and support new data feeds for various Data Management layers and Data Lakes
  • Evaluate business needs and requirements
  • Support migration of existing data transformation jobs in Oracle, and MS-SQL to Snowflake.
  • Lead the migration of the existing data transformation jobs in Oracle, Hive, Impala etc. into Spark, Python on Glue etc.
  • Able to document the processes and steps.
  • Develop and maintain datasets.
  • Improve data quality and efficiency.
  • Lead Business requirements and deliver accordingly.
  • Collaborate with Data Scientists, Architect and Team on several projects

  • Data Engineering Lead

    3 weeks ago


    Singapore GMP Technologies Full time

    Job Responsibilities:To oversee data projects, operations and initiatives in the Great China & Intercontinental region in both Commercial and Medical areasTo be able to support and partner in country data operations and engineering leadsTo oversee analytics operations in the region in alignment with global data operations/engineering teamsDevelop strategies...


  • Singapore Vanguard Software Pte. Ltd. Full time

    JOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data engineering, expertise in...


  • Singapore VANGUARD SOFTWARE PTE. LTD. Full time

    Roles & ResponsibilitiesJOB SUMMARYWe are looking for a talented and experienced Lead Data Engineer to join our innovative team. The Lead Data Engineer will play a critical role in leading the design, development, and maintenance of our data infrastructure, pipelines, and analytical tools. The ideal candidate will have a strong technical background in data...


  • Singapore Standard Chartered Full time

    Want to push the boundaries of traditional technology, finance, and operations? At nexus, courage is a defining trait for all team members. Courage to challenge the status quo, to do better, to deliver against the grain and to achieve spectacular results in massive ambiguity. We want to change how ecosystems interact and only have the blueprint for the baby...


  • Singapore Standard Chartered Full time

    Want to push the boundaries of traditional technology, finance, and operations? At nexus, courage is a defining trait for all team members. Courage to challenge the status quo, to do better, to deliver against the grain and to achieve spectacular results in massive ambiguity. We want to change how ecosystems interact and only have the blueprint for the baby...


  • Singapore Bridge Data Centres (international) Pte. Ltd. Full time

    Key Responsibilities:. Conduct in-depth design reviews and oversee the commissioning/Integrated Systems Testing (IST) processes for new data centre projects, as well as modifications and expansions to existing data centres. Enhance and standardize data centre operation and maintenance policies and procedures to ensure efficiency and compliance. Perform...

  • Data Engineer

    3 weeks ago


    Singapore NTT DATA SINGAPORE PTE. LTD. Full time

    Roles & Responsibilities EMAIL ID : Interested candidates, please email your resume to mike.ramos@nttdata.comOnly shortlisted candidates would be contacted for interview.Role: Data Engineer - 2 years Renewable contractExperience: Minimum of 5 yearsLocation : Tanjong PagarBenefits: Renewable contract (Performance-based, with possibility of conversion to...

  • Data Engineer

    3 weeks ago


    Singapore Ntt Data Singapore Pte. Ltd. Full time

    ' EMAIL ID : Interested candidates, please email your resume to [HIDDEN TEXT]Only shortlisted candidates would be contacted for interview.Role: Data Engineer - 2 years Renewable contractExperience: Minimum of 5 yearsLocation : Tanjong PagarBenefits:Renewable contract (Performance-based, with possibility of conversion to perm)Medical Insurance CoverageLeave...


  • Singapore Standard Chartered Full time

    Want to push the boundaries of traditional technology, finance, and operations? At nexus, courage is a defining trait for all team members. Courage to challenge the status quo, to do better, to deliver against the grain and to achieve spectacular results in massive ambiguity. We want to change how ecosystems interact and only have the blueprint for...


  • Singapore Standard Chartered Full time

    Want to push the boundaries of traditional technology, finance, and operations? At nexus, courage is a defining trait for all team members. Courage to challenge the status quo, to do better, to deliver against the grain and to achieve spectacular results in massive ambiguity. We want to change how ecosystems interact and only have the blueprint for...


  • Singapore Shopee Full time

    Job Description:Lead the Big Data SRE team and oversee the development and maintenance of the big data ops automation platform, and improve the operation and maintenance management level of big dataMonitor the performance and system stability of big data system in Shopee such as Hadoop/Spark/Storm/KafkaProvide technical expertise on big data-related...

  • Lead Data Engineer

    3 weeks ago


    Singapore GMP Technologies Full time

    Responsibilities: Overseeing data projects, operations and initiatives regionally in both Commercial and Medical areas Overseeing analytics operations in the region in alignment with global data operations/engineering teams To be able to support and partner in country data operations and engineering leads, specifically China, Japan & Canada Develop...

  • Data Product Lead

    5 days ago


    Singapore GVT Government Technology Agency Full time

    The Government Technology Agency (GovTech) is the lead agency driving Singapore's Smart Nation initiatives and public sector digital transformation. As the Centre of Excellence for Infocomm Technology and Smart Systems (ICT & SS), GovTech develops the Singapore Government's capabilities in Data Science & Artificial Intelligence, Application Development,...

  • Data Engineer

    4 weeks ago


    Singapore Cognizant Full time

    Do you have software developer experience, and are you seeking a new job in Singapore? Cognizant is looking for a Data Engineer, and the full-time role comes with an attractive salary and benefits package.As a Data Engineer, you will design, develop and implement data-processing pipelines to process large volumes of structured and unstructured data....

  • Data Engineer

    4 weeks ago


    Singapore Cognizant Full time

    Do you have software developer experience, and are you seeking a new job in Singapore? Cognizant is looking for a Data Engineer, and the full-time role comes with an attractive salary and benefits package.As a Data Engineer, you will design, develop and implement data-processing pipelines to process large volumes of structured and unstructured data. You...

  • Data Engineer

    3 weeks ago


    Singapore Ntt Data Business Solutions Singapore Pte. Ltd. Full time

    Your career at NTT DATA Business SolutionsEmpowering You - to feel our passion for SAP / Analytic Solutions / ServiceNowAs NTT DATA Business Solutions, we are more than just a company. We are a team of innovative and passionate people who specialize in creating value-added SAP and Analytic solutions and guiding our customers on their journey to becoming...

  • Data Engineer

    5 days ago


    Singapore NTT DATA BUSINESS SOLUTIONS SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesYour career at NTT DATA Business SolutionsEmpowering You – to feel our passion for SAP / Analytic Solutions / ServiceNowAs NTT DATA Business Solutions, we are more than just a company. We are a team of innovative and passionate people who specialize in creating value-added SAP and Analytic solutions and guiding our customers on...


  • Singapore DATA DISTINCTION Full time

    b'This description is supported by a more extensive requirements document. Overview We are looking to establish a text classification model that will create a mapping between a dictionary (JSON) of trip types with locales or destinations which are individually described in "articles" in Wiki Voyage. There are three specific outcomes that can be achieved...


  • Singapore Singtel Group Full time

    NCS is a leading information and communications technology (ICT) and communications engineering services provider across the Asia-Pacific region. We are headquartered in Singapore and a wholly owned subsidiary of the Singtel Group. We have in-depth domain knowledge and unique capabilities that create business value for customers. We offer a broad range...


  • Singapore SingTel Full time

    Job Description :NCS is a leading information and communications technology (ICT) and communications engineering services provider across the Asia-Pacific region. We are headquartered in Singapore and a wholly owned subsidiary of the Singtel Group. We have in-depth domain knowledge and unique capabilities that create business value for customers. We offer a...