Current jobs related to Data Engineer(Python, Pyspark, Spark - Singapore - Unison Consulting Pte Ltd

  • Data Engineer(Python, Pyspark, Spark

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Job Type** **Seniority** Mid **Years of Experience** Information not provided **Tech Stacks** ETL AWS Google Cloud pySpark BigQuery RedShift Apache Azure Spark SQL Python **Key Responsibilities**: - Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL - Collaborate with...

  • Data Engineer

    1 week ago

    Singapore Rapsys Technologies Full time

    Description: Roles & Responsibilities: - Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework. - Should have strong hands-on working expertise in creating the optimized data pipelines in Pyspark/Python/Scala....

  • Data Engineer

    6 days ago

    Singapore Capgemini Full time

    Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework. - Should have strong hands-on working expertise in creating the optimized data pipelines in Pyspark/Python/Scala. Produce unit tests for Spark...

  • Big Data Engineer

    3 days ago

    Singapore Capgemini Full time

    **Roles and Responsibilities: - **Develop Spark Scala and PySpark jobs for data transformation and aggregation.** - **Create unit tests for Spark transformations and helper methods.** - **Utilize Spark and Spark SQL to read parquet data and create tables in Hive using the Scala API.** - **Collaborate closely with the Business Analysts team to review test...

  • Data Engineer

    3 days ago

    Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using Spark, Python, and PySpark. Ensure the smooth flow of data from various sources to destination systems. Utilize SAS, Spark, Python, and PySpark for efficient data processing. Design and implement data models to support business requirements. Work closely with data architects to...

  • Data Engineer

    2 days ago

    Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time

    **Responsibilities**: - Architect and implement robust ETL/ELT pipelines to ingest and transform large volumes of data from heterogeneous systems (e.g., Mainframe, Oracle, SQL Server, DB2, Teradata) into distributed data platforms such as Hadoop and cloud-based data lakes. - Utilize tools such as Apache Spark, PySpark, and Hive for batch and stream data...

  • Python Data Engineer

    3 days ago

    Singapore NEURONES IT ASIA PTE. LTD. Full time

    As a Python Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks. **Your role is**: - Designing and delivering software using an agile and iterative approach based on Scrum or Kanban - Following and contributing to improvement of client's software engineering...

  • Senior Scala,pyspark

    1 day ago

    Singapore Rapsys Technologies Full time

    JD: "10+ yrs Extensive working knowledge and expertise in Spark on Scala or PySpark and Hive. Experience in Design, development and performance tuning in Spark. Strong programming skills in Java or Scala or Python Familiarity with big data processing tools and techniques. Experience with the Hadoop ecosystem Good understanding of distributed systems Should...

  • Pyspark Developer with SAS Knowledge

    2 weeks ago

    Singapore Unison Consulting Pte Ltd Full time

    Key Responsibilities: - Analyze and understand existing SAS programs and datasets - Design and develop PySpark code that replicates the functionality of existing SAS programs - Optimize PySpark code for performance and scalability - Test and validate PySpark code to ensure it produces the same results as the original SAS programs - Identify and resolve...

  • Data Engineer

    2 days ago

    Singapore Cognizioni IT Solutions LLC Full time

    **Role : Data Engineer** **Location : Singapore** **Long Term Contract** JD Roles & Responsibilities: - Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework. - Should have strong hands-on working expertise in...

Data Engineer(Python, Pyspark, Spark

2 weeks ago

Singapore Unison Consulting Pte Ltd Full time

**Key Responsibilities**:

- Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL.
- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver appropriate data solutions.
- Develop and optimize ETL (Extract, Transform, Load) processes to ensure data quality and integrity.
- Implement data processing workflows for batch and real-time data integration.
- Monitor and troubleshoot data pipelines, ensuring data accuracy and availability.
- Perform data exploration and analysis to support business decisions.
- Document data processes, architectures, and workflows.

**Requirements**:
Proven 4+Years of experience as a Data Engineer or in a similar role.

Strong proficiency in **Python and PySpark** for data manipulation and transformation.

Hands-on experience with Apache Spark for large-scale data processing.

Advanced SQL skills for data querying, transformation, and performance tuning.

Familiarity with cloud platforms (AWS, Azure, Google Cloud) and their data services.

Experience with data warehousing concepts and technologies (e.g., Redshift, BigQuery, Snowflake).

Knowledge of best practices for data architecture and data governance.