Data Engineer

7 days ago

Singapore ANTAS PTE. LTD. Full time $104,000 - $130,878 per year

POSITION OVERVIEW : Software Development Analyst

POSITION GENERAL DUTIES AND TASKS :

Job Description: We are seeking a skilled Data Engineer to do the migration of our existing data warehouse and data model from MariaDB to the Hadoop Big Data Cloudera on-premise platform. The ideal candidate must be proficient in SQL, Hive SQL, Spark, and data modeling. Additionally, they should possess a strong understanding of the production deployment process, including design, development, testing, UAT, and production deployment. Experience of scheduling jobs using Autosys is essential for this role.

Key Responsibilities


• Migrate existing data warehouse and data model from MariaDB to Hadoop Big Data Cloudera on-premises platform.
•Develop and optimize SQL, Hive SQL, and Spark scripts to ensure efficient data processing.
•Design and implement data models to support business requirements and optimize performance.
•Collaborate with cross-functional teams to understand data requirements and ensure data integrity throughout the migration process.

  • Develop and execute test plans to validate data accuracy and system performance.
  • Coordinate with stakeholders (internal) to plan and execute production deployments.
  • Schedule and monitor jobs using Autosys to ensure timely execution and minimize downtime.
  • Provide technical expertise and guidance to team members throughout the migration project.
  • Document processes, procedures, technical specifications, and best practices to facilitate knowledge sharing and ensure scalability.
  • Create and maintain unit test case documents to ensure code quality and reliability.

Qualifications


• Bachelor's (or Higher) degree in computer science, Engineering, or a related field.
•Proven experience of 5+ years in data engineering and migration projects for big data (Hortonworks / Cloudera)
•Strong hands-on experience on Cloudera and related ecosystem components.
•Strong experience in implementing ETL (Extract, Transform, Load) processes.

  • Highly proficient in SQL, Hive SQL, Spark, and data modeling.
  • Strong understanding of the production deployment process.
  • Experience with scheduling jobs using Autosys or similar.
  • Experience with version control systems such as Bitbucket, GIT etc.
  • Ability to troubleshoot and resolve data related issues efficiently.
  • Good communication and interpersonal skills
  • Experience with Dataiku is a plus, but not mandatory.

  • Azure Data engineer

    1 day ago

    Singapore NTT Data Singapore Full time $80,000 - $120,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Senior Data Scientist

    7 days ago

    Singapore Data Analytics Full time $120,000 - $180,000 per year

    Get to Know the TeamOur Search and Personalisation Team focuses on advancing scalable machine learning and experimentation frameworks that power search relevance, ranking, and recommendations across Grab. By leveraging cutting-edge analytics and statistical methods, we continuously optimize how users connect with merchants, services, and content.Get to Know...

  • Lead Data Scientist

    2 weeks ago

    Singapore Data Analytics Full time $180,000 - $250,000 per year

    Get to Know the TeamAt Grab, the Market Insights team drives impactful data-driven decisions by uncovering insights about the region's users, merchants, and driver partners. By analysing vast datasets across transactional and behavioural activities, we bridge the gap between raw data science capabilities and actionable business strategies. Our work is...

  • Director

    2 weeks ago

    Singapore GALA ENGINEER PTE. LTD. Full time

    The Operation Director is responsible for leading, guiding and directing all operational areas to ensure successful achievement of operational objectives. They must create operational plans that fully align with the organizations’ overall strategic plan, while maintaining a focus on operational effectiveness, efficiency, and customer satisfaction. This...

  • Ms Engineer

    2 weeks ago

    Singapore NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** The...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    3 days ago

    Singapore NTT DATA, Inc. Full time

    Overview Senior Associate Infrastructure Engineering and Operations Engineer at NTT DATA, Inc. Make an impact with NTT DATA. Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and...

  • Senior Backup Engineer SME

    3 days ago

    Singapore NTT DATA, Inc. Full time

    Join to apply for the Senior Backup Engineer SME role at NTT DATA, Inc. 2 days ago Be among the first 25 applicants Join to apply for the Senior Backup Engineer SME role at NTT DATA, Inc. Get AI-powered advice on this job and more exclusive features. Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are...

  • DevSecOps Engineer

    7 days ago

    Singapore NTT DATA Full time $120,000 - $200,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAWe are seeking a...

  • Customer Success Services Engineer

    2 weeks ago

    Singapore Data Direct Networks Full time

    Overview: This is an incredible opportunity to be part of a company that has been at the forefront of AI and high performance data storage innovation for over two decades. DDN Storage is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare, to financial...

  • Senior Manager, Mechanical Engineering, Apac

    2 weeks ago

    Singapore Vantage Data Centers Full time

    About Vantage Data Centers Vantage is committed to being a workplace of inclusion, equity, respect and acceptance. We celebrate diversity and intentionally seek out opportunities to learn from one another’s experience. Vantage Data Centers powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud providers and...