Data Engineer

2 weeks ago

Singapore ANTAS PTE. LTD. Full time $104,000 - $130,878 per year

POSITION OVERVIEW : Software Development Analyst

POSITION GENERAL DUTIES AND TASKS :

Job Description: We are seeking a skilled Data Engineer to do the migration of our existing data warehouse and data model from MariaDB to the Hadoop Big Data Cloudera on-premise platform. The ideal candidate must be proficient in SQL, Hive SQL, Spark, and data modeling. Additionally, they should possess a strong understanding of the production deployment process, including design, development, testing, UAT, and production deployment. Experience of scheduling jobs using Autosys is essential for this role.

Key Responsibilities


• Migrate existing data warehouse and data model from MariaDB to Hadoop Big Data Cloudera on-premises platform.
•Develop and optimize SQL, Hive SQL, and Spark scripts to ensure efficient data processing.
•Design and implement data models to support business requirements and optimize performance.
•Collaborate with cross-functional teams to understand data requirements and ensure data integrity throughout the migration process.

  • Develop and execute test plans to validate data accuracy and system performance.
  • Coordinate with stakeholders (internal) to plan and execute production deployments.
  • Schedule and monitor jobs using Autosys to ensure timely execution and minimize downtime.
  • Provide technical expertise and guidance to team members throughout the migration project.
  • Document processes, procedures, technical specifications, and best practices to facilitate knowledge sharing and ensure scalability.
  • Create and maintain unit test case documents to ensure code quality and reliability.

Qualifications


• Bachelor's (or Higher) degree in computer science, Engineering, or a related field.
•Proven experience of 5+ years in data engineering and migration projects for big data (Hortonworks / Cloudera)
•Strong hands-on experience on Cloudera and related ecosystem components.
•Strong experience in implementing ETL (Extract, Transform, Load) processes.

  • Highly proficient in SQL, Hive SQL, Spark, and data modeling.
  • Strong understanding of the production deployment process.
  • Experience with scheduling jobs using Autosys or similar.
  • Experience with version control systems such as Bitbucket, GIT etc.
  • Ability to troubleshoot and resolve data related issues efficiently.
  • Good communication and interpersonal skills
  • Experience with Dataiku is a plus, but not mandatory.

  • Azure Data engineer

    4 days ago

    Singapore NTT Data Singapore Full time

    Job Responsibilities: Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Azure Data engineer

    6 days ago

    Singapore NTT Data Singapore Full time $60,000 - $180,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Data Engineer

    3 days ago

    Singapore NTT Data Singapore Full time

    **Competency Required**: - Possess a degree in Computer Science/Information Technology or related fields. - At least 3 years of experience in a role focusing on development and support of data ingestion pipelines. - **Experience with building on data platforms, e.g. Snowflake.**: - **Proficient in SQL and Python.**: - Experience in continuous integration...

  • Senior Data Scientist

    2 weeks ago

    Singapore Data Analytics Full time $120,000 - $180,000 per year

    Get to Know the TeamOur Search and Personalisation Team focuses on advancing scalable machine learning and experimentation frameworks that power search relevance, ranking, and recommendations across Grab. By leveraging cutting-edge analytics and statistical methods, we continuously optimize how users connect with merchants, services, and content.Get to Know...

  • Data Engineer

    5 days ago

    Singapore COMMUNITY DATA LABS PTE. LTD. Full time

    We are a San Francisco and Singapore based stealth startup developing a new framework for data on the internet powered by data interoperability and new economic systems. We are a stealth startup where everyone directly contributes to the core product. You’ll need to be comfortable wearing many hats and managing projects end-to-end independently. We are...

  • SME Critical Facilities Data Center Operations Engineer

    4 days ago

    Singapore NTT Global Data Centers Full time

    Overview The SME Critical Facilities Data Center Operations Engineer is a developing subject matter expert typically responsible for designing, planning, installing, and maintaining mechanical, electrical, plumbing systems. Proficient understanding of sensitive facility components that may be impacted by failures or malfunctions of infrastructure systems....

  • Director

    2 weeks ago

    Singapore GALA ENGINEER PTE. LTD. Full time

    The Operation Director is responsible for leading, guiding and directing all operational areas to ensure successful achievement of operational objectives. They must create operational plans that fully align with the organizations’ overall strategic plan, while maintaining a focus on operational effectiveness, efficiency, and customer satisfaction. This...

  • Ms Engineer

    1 week ago

    Singapore NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** The...

  • Network Engineering Specialist

    3 days ago

    Singapore NTT DATA Full time

    **Req ID**: 133019 We are currently seeking a Network Engineering Specialist to join our team in Singapore, Central Singapore (SG-01), Singapore (SG). Job Title: Network Engineering Specialist Summary: The Networking Engineering Specialist will be responsible for production and non-production Device Design/Policy/functionality for service implementation....

  • Senior Associate Infrastructure Engineering and Operations Engineer

    1 week ago

    Singapore NTT DATA, Inc. Full time

    Overview Senior Associate Infrastructure Engineering and Operations Engineer at NTT DATA, Inc. Make an impact with NTT DATA. Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and...