Data Engineer(Python, Pyspark, Spark

1 day ago


Singapore Unison Consulting Pte Ltd Full time

**Key Responsibilities**:

- Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL.
- Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver appropriate data solutions.
- Develop and optimize ETL (Extract, Transform, Load) processes to ensure data quality and integrity.
- Implement data processing workflows for batch and real-time data integration.
- Monitor and troubleshoot data pipelines, ensuring data accuracy and availability.
- Perform data exploration and analysis to support business decisions.
- Document data processes, architectures, and workflows.

**Requirements**:
Proven 4+Years of experience as a Data Engineer or in a similar role.

Strong proficiency in **Python and PySpark** for data manipulation and transformation.

Hands-on experience with Apache Spark for large-scale data processing.

Advanced SQL skills for data querying, transformation, and performance tuning.

Familiarity with cloud platforms (AWS, Azure, Google Cloud) and their data services.

Experience with data warehousing concepts and technologies (e.g., Redshift, BigQuery, Snowflake).

Knowledge of best practices for data architecture and data governance.


  • Data Engineer

    2 weeks ago


    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$7,500 - S$11,500 / Monthly **Job Type** **Seniority** Senior **Years of Experience** At least 6 years **Tech Stacks** ETL AWS Analytics pySpark Apache Azure Spark NoSQL SQL Python **Position Overview**: As a PySpark Data Engineer, you will be responsible for designing, implementing, and maintaining data processing...

  • Software Engineer

    1 week ago


    Singapore QUESSCORP HOLDINGS PTE. LTD. Full time

    Key Responsibilities - Design, develop, and optimize data workflows and pipelines using Apache Hadoop, Cloudera, and Spark. - Develop microservice APIs using Python Django framework and utilized Redis as a data store - Manage and optimize data storage in data warehouse solutions, including Cloudera and Hive. - Implement and manage data processing workflows...

  • Data Engineer

    3 days ago


    Singapore Infosight consulting Full time

    **Core Java** - Fundamental knowledge of distributed **architectures, SOA, Micro-services, PaaS** - Good knowledge of data manipulation using distributed data processing systems such as **Spark SQL** - Required to have good SPARKS skills - Hands-on experience on **HDFS (Hadoop), spark, impala, hive, as well as database technology** - Source Code Control...

  • Pyspark Developer

    1 week ago


    Singapore NodeFlair Full time

    **Job Summary**: **Job Type** Permanent **Seniority** Senior Mid **Years of Experience** 4-6 years **Tech Stacks** Shell Script pySpark HDFS Shell UNIX Java Hive Spark SQL Scala Hadoop Python **Key Responsibilities**: - Working with the latest tools and techniques - Hands-on coding, usually in a pair programming environment - Working in highly...


  • Singapore Sembcorp Industries Ltd Full time

    About Sembcorp Sembcorp is a leading energy and urban solutions provider headquartered in Singapore. Led by its purpose to drive energy transition, Sembcorp delivers sustainable energy solutions and urban developments by leveraging its sector expertise and global track record. Purpose & Scope We are seeking a highly skilled and self-driven Azure Data...


  • Singapore Tangspac Full time

    S - Posted by Subagio Kusuma- Managing Partner Working as data engineer for a large international bank, you will be part of the global data engineering team involved in development and implementing data solutions You will be working as data engineer with a large global bank, you will be playing key role in building complex data solution Job...

  • Data Engineer

    3 days ago


    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    This is a 12 months contract assigned to our client Work Location: To be confirmed Salary Range : $6,000-$8,000 **Job Description**: 1. Primary Skill: Data Analytics 2. Data Engineering 3. Hands on programming skills using Python, Spark, ETL pipeline related such as Kafka, airflow. Skills Requirement (Intermidiate Level) 1. Apache Kafka 2. Apache...


  • Singapore Consulguru Pte. Ltd. Full time

    We are looking for an experienced and highly skilled Hadoop Data Engineer to join our dynamic team. The ideal candidate will have hands‐on expertise in developing optimized data pipelines using Python, PySpark, Scala, Spark‐SQL, Hive , and other big data technologies. You will be responsible for translating complex business and technical requirements...

  • Senior Data Engineer

    2 weeks ago


    Singapore OPENSOURCE PTE. LTD. Full time

    **Job Title**:Senior Data Engineer (5-10 Years of Experience) **Position Summary**: **Responsibilities**: - Design, build, and maintain scalable and reliable data pipelines to support real-time and batch processing requirements. - Manage, monitor, and optimize workflows in **Apache Airflow **to ensure data quality and integrity. - Develop and integrate...

  • Data Engineer

    2 weeks ago


    Singapore Unison Consulting Pte Ltd Full time

    Overview We are seeking a skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in SQL, Python, Spark, and Hive , with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver...