Data Engineer

4 days ago

Singapore INFOSIGHT Solutions Full time

Data Engineer - (PySpark + Hadoop)

Roles & Responsibilities
- Create Spark Scala/Pyspark jobs for data transformation and aggregation.
- Produce unit tests for Spark transformations and helper methods.
- Used Spark and Spark-SQL to read the parquet data and create the tables in hive using the Scala API.
- Work closely with Business Analysts team to review the test results and obtain sign off
- Prepare necessary design/operations documentation for future usage
- Perform peers Code quality review and be gatekeeper for quality checks
- Hands-on coding, usually in a pair programming environment
- Working in highly collaborative teams and building quality code
- Familiar with Oracle, Spark streaming, Kafka, ML.
- Ensure smooth production deployments as per plan and post-production deployment verification.

**Requirements**:

- Hadoop data engineer with total 4 - 10 years of experience and should have strong experience in Hadoop, Spark, Scala, Pyspark, Python, Hive, Impala, CI/CD, Git, Jenkins, Agile Methodologies, DevOps, Cloudera Distribution.
- Strong Knowledge in data warehousing Methodology.
- Min Relevant 4 years of Hadoop & Spark/Pyspark experience is mandatory.
- Strong in enterprise data architectures and data models
- Good experience in Core Banking, Finance domain
- Good to have Cloud experience on AWS.

**Job Type**: Contract
Contract length: 6 months

Pay: $4,500.00 - $5,100.00 per month

  • Data Engineer

    2 weeks ago

    Singapore NTT Data Singapore Full time

    Open roles:  Lead Data EngineerSenior Data EngineerJunior Data EngineerJunior Data Engineer 1 year renewable contract Primary Skills Informatica/IDMC, Databricks/Python (Hands on strong knowledge in any combination of two) Team Lead experience Secondary SkillsTableau/OAS, AWS Cloud PreferenceAWS Architect / Databricks / IDMC certification Key...

  • Data Engineer

    2 weeks ago

    Singapore NTT Data Singapore Full time

    Open roles:  Lead Data EngineerSenior Data EngineerJunior Data EngineerJunior Data Engineer 1 year renewable contract Primary Skills Informatica/IDMC, Databricks/Python (Hands on strong knowledge in any combination of two) Team Lead experience Secondary Skills Tableau/OAS, AWS Cloud Preference AWS Architect / Databricks / IDMC certification Key...

  • Data Engineer

    2 weeks ago

    Singapore NTT Data Singapore Full time

    **The primary responsibilities**: - Lead architecture design, develop, document, and implement end-to-end data pipelines and data-driven solutions. - Define roadmap to transform data architecture focusing on scalability, performance, and stability for the entire data lifecycle; - Build data flow for data acquisition, aggregation, and modelling, using both...

  • Azure Data engineer

    6 days ago

    Singapore NTT Data Singapore Full time

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations...

  • Data Center Project Engineer

    2 weeks ago

    Singapore NTT DATA, Inc. Full time

    Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive. Your day at NTT DATA The Data...

  • Data Engineer

    1 week ago

    Singapore NTT DATA Asia Pacific Full time

    Data Engineer (Junior to Lead roles available)Join to apply for the Data Engineer (Junior to Lead roles available)role at NTT DATA Asia Pacific Open roles Lead Data Engineer Senior Data Engineer Junior Data Engineer Junior Data Engineer 1 year renewable contract Primary Skills Informatica/IDMC, Databricks/Python (Hands on strong knowledge in any combination...

  • M-2025-1636/37/38/39 - Data Engineer

    1 week ago

    Singapore NTT Data Singapore Full time

    Open roles: M- - Lead Data Engineer M- - Senior Data Engineer M- - Junior Data Engineer M- - Junior Data Engineer 1 year renewable contract Primary Skills Informatica/IDMC, Databricks/Python (Hands on strong knowledge in any combination of two)Team Lead experience Secondary Skills Tableau/OAS, AWS Cloud Preference AWS Architect / Databricks / IDMC...

  • Senior Data Scientist

    2 weeks ago

    Singapore Data Science Full time

    Get to Know the Team You'll be part of the SNP Data Science Personalization team, an established group dedicated to enhancing personalization experiences for Grab Food/Mart/Transport. We aim to develop cutting‐edge Deep Learning models to deliver best user experiences. Our team comprises passionate Data Scientists with various expertise. If personalization...

  • Lead Data Engineer

    1 week ago

    Singapore NTT Data Singapore Full time

    A technology solutions provider in Singapore is seeking a Lead Data Engineer to design and optimize scalable data pipelines. The ideal candidate will have strong hands-on experience with Informatica/IDMC or Databricks with Python. Responsibilities include integrating data from multiple sources and collaborating with analytics teams to ensure data quality and...

  • Data Engineer: Build Scalable ETL Pipelines

    1 week ago

    Singapore NTT DATA Asia Pacific Full time

    An IT consulting firm in Singapore is seeking Data Engineers across various levels from Junior to Lead. The role involves optimizing data pipelines, integrating data using tools like Informatica, and developing solutions in Databricks with Python. Candidates should possess strong skills in data management and integration, and familiarity with Tableau for...