Data Engineer

3 days ago


Singapore Rapsys Technologies Full time

Description:
Roles & Responsibilities:

- Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework.
- Should have strong hands-on working expertise in creating the optimized data pipelines in Pyspark/Python/Scala. Produce unit tests for Spark transformations and helper methods.
- Understand the complex transformation logic and translate and build data pipelines in Pyspark/Spark-SQL/Hive logic to ingest the data from source systems to Data Lake (Hive/Hbase/Parquet)/ Enterprise Data Domain tables.
- Work closely with Business Analysts team to review the test results and obtain sign off.
- Prepare necessary design/operations documentation for future usage.
- Perform peers Code quality review and be gatekeeper for quality checks.
- Hands-on coding, usually in a pair programming environment.
- Working in highly collaborative teams and building quality code.
- Familiar with Oracle, Spark streaming, Kafka, ML.
- Have knowledge on RDMS concepts, hands on experience on PLSQL etc.
- Ensure smooth production deployments as per plan and post-production deployment verification.

Technical Requirements:

- Hadoop data engineer with total 6 - 9.5 years of experience and should have strong experience in Hadoop, Spark, Pyspark, Scala, Hive, Spark-SQL, Python, Impala, CI/CD, Git, Jenkins, Agile Methodologies, DevOps, Cloudera Distribution.
- Strong Knowledge in data warehousing Methodology and Change Data Capture.
- Relevant 5+ years of Hadoop & Spark/Pyspark experience is mandatory.
- Good Knowledge and experience in any RDBMS database (MariaDB or SQL Server OR MySQL or Oracle ). Knowledge on stored procedures is an added advantage.
- Have exposure to TWS jobs for scheduling.
- Strong in enterprise data architectures and data models.
- Good experience in Core Banking, Finance domain.
- Exposure in AML Domain preferred, not mandatory.

**Job Type**: Contract
Contract length: 12 months

Pay: $4,000.00 - $6,000.00 per month

**Experience**:

- Data Engineer: 5 years (preferred)
- Pyspark: 4 years (preferred)
- SCALA: 4 years (preferred)
- Python: 4 years (preferred)
- banking domain: 3 years (preferred)



  • Singapore NTT DATA Full time

    Requirements / Qualifications - Bachelor's or master's degree in computer science, data engineering, or a related field. - Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC. - Proficiency in programming languages such as Python, Java, or Scala for building data pipelines. - Evaluate...

  • Data Engineer

    3 days ago


    Singapore DOZER DATA PTE. LTD. Full time

    **About the job**: This position is needed to build a highly scalable, reliable, and efficient data platform that makes it easy for users to get deep insights from vast amounts of distributed data. This role will be responsible for driving key technical decisions to completion for the organization and also guiding other engineers. **Responsibilities**: In...

  • Azure Data engineer

    2 weeks ago


    Singapore NTT Data Singapore Full time

    Job Responsibilities: Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Azure Data engineer

    2 weeks ago


    Singapore NTT Data Singapore Full time $80,000 - $120,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...


  • Singapore NTT DATA Singapore PTE Ltd Full time

    Job Description - About Us - Roles and Responsibility **Job scopes**: - The Data Engineer will lead the enterprise-wide data transformations projects. The person will involve in developing and automating data processing pipelines for financial and Investment data modelling, analysis, and reporting from various data sources systems; The primary...

  • Data Engineer

    2 weeks ago


    Singapore TRIVI DATA CONSULTING PTE. LTD. Full time

    Key Responsibilities - Design, implement, and maintain ETL workflows for managing large-scale datasets. - Develop cloud-based data pipelines (AWS, Azure) for scalable and real-time analytics delivery. - Optimize data models to ensure high performance and reliable reporting. - Create advanced dashboards and visualizations using tools like Qlik Sense and...


  • Singapore Data Science Full time

    Get to know the team You'll be part of the SNP Data Science Personalization team, an established group dedicated to enhancing personalization experiences for Grab Food/Mart/Transport. We aim to develop cutting-edge Deep Learning models to deliver best user experiences. Our team comprises passionate Data Scientists with various expertise. If personalization...


  • Singapore NTT DATA Full time

    **Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** The...


  • Singapore NTT Data Singapore Full time

    Responsibilities: - Installation of SQL Server / Oracle - Creation of new database - Database Instance Hardening, Patching on identification of Vulnerabilities - Configuration and Testing of Backup-Restore scripts - Configuration of AOAG (SQL Server) / DataGuard (Oracle) - Testing of AOAGroup/Instance Switchover to D.R. and Switchback to PROD - Migration of...

  • Linux L3 Engineer

    3 days ago


    Singapore NTT DATA Full time

    **Req ID**: 281067 We are currently seeking a Linux L3 Engineer to join our team in Singapore, SG.01, Central Singapore (SG-01), Singapore (SG). Expertise in OS, S/W, Patch Management, Product upgrades Perform performance tuning and hardware upgrades as required Experience in Shell, Perl, and/or Python scripting Experience in clustering Ability to build...