Senior Data Engineer

2 days ago

Singapore EXASOFT PTE. LTD. Full time $120,000 - $180,000 per year

Job Description: We are seeking a skilled Data Engineer to do the migration of our existing data warehouse and data model from MariaDB to the Hadoop Big Data Cloudera on-premise platform. The ideal candidate must be proficient in SQL, Hive SQL, Spark, and data modeling. Additionally, they should possess a strong understanding of the production deployment process, including design, development, testing, UAT, and production deployment. Experience of scheduling jobs using Autosys is essential for this role.

Key Responsibilities

• Migrate existing data warehouse and data model from MariaDB to Hadoop Big Data Cloudera on-premises platform.
•Develop and optimize SQL, Hive SQL, and Spark scripts to ensure efficient data processing.
•Design and implement data models to support business requirements and optimize performance.
•Collaborate with cross-functional teams to understand data requirements and ensure data integrity throughout the migration process.

  • Develop and execute test plans to validate data accuracy and system performance.
  • Coordinate with stakeholders (internal) to plan and execute production deployments.
  • Schedule and monitor jobs using Autosys to ensure timely execution and minimize downtime.
  • Provide technical expertise and guidance to team members throughout the migration project.
  • Document processes, procedures, technical specifications, and best practices to facilitate knowledge sharing and ensure scalability.
  • Create and maintain unit test case documents to ensure code quality and reliability.

Qualifications

• Bachelor's (or Higher) degree in computer science, Engineering, or a related field.
•Proven experience of 5+ years in data engineering and migration projects for big data (Hortonworks / Cloudera)
•Strong hands-on experience on Cloudera and related ecosystem components.
•Strong experience in implementing ETL (Extract, Transform, Load) processes.

  • Highly proficient in SQL, Hive SQL, Spark, and data modeling.
  • Strong understanding of the production deployment process.
  • Experience with scheduling jobs using Autosys or similar.
  • Experience with version control systems such as Bitbucket, GIT etc.
  • Ability to troubleshoot and resolve data related issues efficiently.
  • Good communication and interpersonal skills
  • Experience with Dataiku is a plus, but not mandatory.

  • Senior Backup Engineer SME

    1 week ago

    Singapore NTT DATA, Inc. Full time

    Join to apply for the Senior Backup Engineer SME role at NTT DATA, Inc. 2 days ago Be among the first 25 applicants Join to apply for the Senior Backup Engineer SME role at NTT DATA, Inc. Get AI-powered advice on this job and more exclusive features. Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are...

  • Senior Data Scientist

    2 weeks ago

    Singapore Data Analytics Full time $120,000 - $180,000 per year

    Get to Know the TeamOur Search and Personalisation Team focuses on advancing scalable machine learning and experimentation frameworks that power search relevance, ranking, and recommendations across Grab. By leveraging cutting-edge analytics and statistical methods, we continuously optimize how users connect with merchants, services, and content.Get to Know...

  • Azure Data engineer

    4 days ago

    Singapore NTT Data Singapore Full time

    Job Responsibilities: Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    1 week ago

    Singapore NTT DATA, Inc. Full time

    Overview Senior Associate Infrastructure Engineering and Operations Engineer at NTT DATA, Inc. Make an impact with NTT DATA. Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and...

  • Azure Data engineer

    6 days ago

    Singapore NTT Data Singapore Full time $60,000 - $180,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Senior Data Engineer

    1 week ago

    Singapore Handshakes by DC Frontiers Pte Ltd Full time

    Handshakes is an award-winning DataTech company. We enable our clients to make safer, more informed decisions by delivering meaningful insights, harnessed from reliable data. We apply machine learning technology to analyze and provide corporate data from public and private domains. Our award-winning Handshakes App fuses these data to provide strategic...

  • Director

    2 weeks ago

    Singapore GALA ENGINEER PTE. LTD. Full time

    The Operation Director is responsible for leading, guiding and directing all operational areas to ensure successful achievement of operational objectives. They must create operational plans that fully align with the organizations’ overall strategic plan, while maintaining a focus on operational effectiveness, efficiency, and customer satisfaction. This...

  • Senior Hyperscale Solutions

    4 days ago

    Singapore NTT DATA, Inc. Full time

    Senior Hyperscale Solutions & Sales Engineer, APAC Join to apply for the Senior Hyperscale Solutions & Sales Engineer, APAC role at NTT DATA, Inc. Overview The APAC Region Senior Hyperscale Solutions & Sales Engineer provides technical support to APAC Region Sales Engineering activities related to hyperscale clients and is responsible for developing and...

  • Data Engineer/Senior Engineer

    1 week ago

    Singapore Sea Full time

    Join to apply for the Data Engineer/Senior Engineer role at Sea Join to apply for the Data Engineer/Senior Engineer role at Sea About Us Sea is the leading consumer internet company in Southeast Asia and Taiwan. Our mission is to better the lives of the consumers and small businesses of our region with technology. About the Team: About Us Sea is the leading...

  • Senior / Data Engineer

    1 week ago

    Singapore H2 Games Full time

    We are looking for a Senior / Data Engineer to build and manage the data pipelines and infrastructure supporting the cutting-edge user growth platform. This role will ensure seamless data integration and real-time analytics capabilities. Job Responsibilities: Design, develop, and maintain scalable data pipelines to process and analyze data from multiple...