Current jobs related to 12200273 - Data Engineer (Python/pyspark) - Singapore - Capgemini

  • Data Engineer

    1 week ago

    Singapore Rapsys Technologies Full time

    Description: Roles & Responsibilities: - Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework. - Should have strong hands-on working expertise in creating the optimized data pipelines in Pyspark/Python/Scala....

  • Data Engineer(Python, Pyspark, Spark

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Job Type** **Seniority** Mid **Years of Experience** Information not provided **Tech Stacks** ETL AWS Google Cloud pySpark BigQuery RedShift Apache Azure Spark SQL Python **Key Responsibilities**: - Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL - Collaborate with...

  • Abinitio & Python(Pyspark) Developer

    4 weeks ago

    Singapore LION & ELEPHANTS CONSULTANCY PTE. LTD. Full time

    Roles & ResponsibilitiesHIRING AB INITIO + PYTHON (PYSPARK) DEVELOPERS Open to AllJob Title: Ab Initio ETL / Data Engineer (Ab Initio + Python / PySpark) – for a Leading Software Services CompanyExperience: 5 to 10 yearsWork Location: Singapore (Onsite/Hybrid)Contract Type: Fixed Term / Direct Contract Contact:Please reach out to priya@lionandelephants.com...

  • Data Engineer

    6 days ago

    Singapore Capgemini Full time

    Data Engineer should be able to understand the Business requirements, Functional and Technical requirements and should build effective data transformation jobs in Python, PySpark/SCALA, Python Framework. - Should have strong hands-on working expertise in creating the optimized data pipelines in Pyspark/Python/Scala. Produce unit tests for Spark...

  • Big Data Engineer

    3 days ago

    Singapore Capgemini Full time

    **Roles and Responsibilities: - **Develop Spark Scala and PySpark jobs for data transformation and aggregation.** - **Create unit tests for Spark transformations and helper methods.** - **Utilize Spark and Spark SQL to read parquet data and create tables in Hive using the Scala API.** - **Collaborate closely with the Business Analysts team to review test...

  • Data Engineer

    3 days ago

    Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using Spark, Python, and PySpark. Ensure the smooth flow of data from various sources to destination systems. Utilize SAS, Spark, Python, and PySpark for efficient data processing. Design and implement data models to support business requirements. Work closely with data architects to...

  • Python Data Engineer

    3 days ago

    Singapore NEURONES IT ASIA PTE. LTD. Full time

    As a Python Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks. **Your role is**: - Designing and delivering software using an agile and iterative approach based on Scrum or Kanban - Following and contributing to improvement of client's software engineering...

  • Python Data Engineer

    4 weeks ago

    Singapore LION & ELEPHANTS CONSULTANCY PTE. LTD. Full time

    Roles & ResponsibilitiesWe're looking for an experienced Python Data Engineer to join our growing team and help design scalable data pipelines that handle massive datasets. What You'll Do:Build robust ETL/ELT pipelines using Python and cloud technologiesDesign data warehouses and lakes for analytics workloadsImplement real-time data processing...

  • Python Data Engineer

    1 week ago

    Singapore NEURONES IT ASIA PTE. LTD. Full time

    Roles & ResponsibilitiesAs a Python Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks.Your role is : · Designing and delivering software using an agile and iterative approach based on Scrum or Kanban· Following and contributing to improvement of client's software...

  • Senior Scala,pyspark

    1 day ago

    Singapore Rapsys Technologies Full time

    JD: "10+ yrs Extensive working knowledge and expertise in Spark on Scala or PySpark and Hive. Experience in Design, development and performance tuning in Spark. Strong programming skills in Java or Scala or Python Familiarity with big data processing tools and techniques. Experience with the Hadoop ecosystem Good understanding of distributed systems Should...

12200273 - Data Engineer (Python/pyspark)

2 weeks ago

Singapore Capgemini Full time

**Responsibilities**:

- Develop enterprise grade Data Products using Spark and other Big Data frameworks
- Benchmark existing solutions, identify gaps, and propose solutions to eliminate them
- Be able to work within a team of data engineers, data scientists, Devops engineers, business analysts, project managers around the proposed solution and contribute to the team efforts
- Provide operational assistance and guidance for the resulting data product including monitoring, management, disaster recovery, security compliance/auditing, networking, storage, service brokers and build packs
- Establish and maintain continuous delivery pipelines for the deployment of Data Products and related products infrastructure
- Design and implement continuous integration and continuous delivery processes to deliver customer apps to production, fostering a culture of continuous process improvement
- Continuously learn and be at the leading edge of industry trends
- Become an agent of change within organizations

**Requirements**:

- Bachelor’s degree in Computer Engineering or equivalent
- Strong hands-on experience in distributed data architectures is a must
- Strong knowledge of PySpark is a must
- Working knowledge of Python is a must
- CI/CD experience (Jenkins, GitHub) is a must
- Working knowledge of Bash scripts is a must
- A clear understanding of cloud service and deployment models. AWS/GCP experience is nice to have
- Kubernetes or Docker experience is nice to have
- Experience in release management or production operations, leading teams performing systems automation and integration (leveraging an agile methodology / lean techniques)