PySpark Developer

2 weeks ago


Singapur, Singapore OPUS IT SERVICES PTE LTD Full time

Overview

  • Minimum 4 years experienced & motivated ETL Developer skilled in the design, development & deployment of Python & PySpark based XML Frameworks for ingestion of data from various source systems to Snowflake in AWS & Google Cloud Platform (GCP) environments.
  • Hands-on coding using Python, PySpark in cloud-based environment involving big-data frameworks in AWS resources like EMR, Lambda, S3 bucket, RDS, EC2, ECS, EKS etc.
  • Experience in ETL frameworks for data ingestion, data pipeline to Snowflake Data Warehouse at high-frequency & volume scenarios (giga / tera-bytes of data ingestions per day / month).
  • Optimize pyspark jobs for performance, efficiency, trouble shoot issues, ensure data quality and availability
  • Implement processes for automating data delivery, monitoring data quality, production deployment.
  • The candidate should be an expert in development with hands-on experience in leveraging Git / GitLab based repository management with understanding of Government Commercial Cloud (GCC) requirements.
  • Proficient in consulting communication skills – good in articulating business problem, approach for solution, respond to changes in scenarios for business problems.
  • Degree in Computer Science, Computer Engineering or any STEM equivalent
  • Familiar with working in Government Commercial Cloud (GCC) environment.
  • Independent and self-motivated contributor and passionate about software development
#J-18808-Ljbffr
  • Data engineer

    2 days ago


    Singapur, Singapore FLINTEX CONSULTING PTE. LTD. Full time

    Overview Data engineer (Azure) – Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure Devops Responsibilities Design, review and development of Pyspark scripts. Testing, troubleshooting of data pipelines, orchestration Implement and maintain datalake Establishing connections to source data systems such as on-prem databases, IOT...

  • Big Data Engineer

    2 weeks ago


    Singapur, Singapore Helius Technologies Pte Ltd Full time

    Big Data Engineer Job Requirement: 5 years of hands on work experience in Hadoop, Scala, Pyspark and Python. Work with Hadoop Distributed File System (HDFS) to manage and store large datasets Utilize tools like Hive, HBase, and Pig to process data in Hadoop Set up and manage data pipelines to extract, transform, and load (ETL) large datasets Develop and...

  • Data engineer

    3 weeks ago


    Singapur, Singapore FLINTEX CONSULTING PTE. LTD. Full time

    Benefits: 13th Month Salary Data Engineer (Azure) – Synapse and Pyspark, Python, Datawarehouse and Power BI, Azure Devops Skills & Experience Bachelor’s Degree in Computer Science or Engineering with 3-5 years of experience in Azure Data engineering, Python, Pyspark or Big Data development. Sound Knowledge of Azure Synapse analytics for pipelines,...

  • Data Architect

    4 days ago


    Singapur, Singapore Infosys Singapore & Australia Full time

    We are looking for a highly experienced and skilled Data Architect to join our team. The ideal candidate will have 12-15 years of experience in architecting solutions of data engineering, focusing on ELT and PySpark/Hadoop workloads. In addition to strong solution and delivery skills, the ideal candidate will also have a view on business growth and managing...


  • Singapur, Singapore QUESS SELECTION & SERVICES PTE. LTD. Full time

    Key Responsibilities: Develop and maintain applications using Oracle PaaS. Design and implement SOAP/REST APIs and process JSON data efficiently. Develop complex PL/SQL scripts, procedures, packages, and triggers. Build webpages and Business Objects in Oracle VBCS and integrate them with Oracle Cloud ERP. Integrate Oracle Cloud ERP with third-party systems...

  • Data Analyst

    3 weeks ago


    Singapur, Singapore Shopee Full time

    About The Team We are seeking a motivated and detail-oriented BI to join our team. The ideal candidate will work closely with cross-functional stakeholders, leveraging data and automation to enhance business processes, drive insights, and support performance analysis. The Brand Analytics team focuses on deep-diving into brand performance on Shopee, helping...

  • Sr. Data Scientist

    4 days ago


    Singapur, Singapore Digital Biz Solutions Full time

    Overview Dbiz is a high-performing product & engineering company that partners up with organizations to help build out digital solutions with the right technology at the right time, whilst being innovative by using technology in different ways. Requirements Mandatory skills: Statistical modelling for regressions, Anomaly detections, Clustering and...

  • Data Analyst

    2 days ago


    Singapur, Singapore Shopee Full time

    Overview Data Analyst - Brand Analytics, Regional Brand Partnership (Entry-Level) at Shopee. Join to apply for the Data Analyst - Brand Analytics, Regional Brand Partnership (Entry-Level) role at Shopee. About The Team We are seeking a motivated and detail-oriented BI to join our team. The ideal candidate will work closely with cross-functional stakeholders,...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore Sembcorp Industries Ltd Full time

    Join to apply for the Data Engineer role at Sembcorp Industries Ltd 3 days ago Be among the first 25 applicants Join to apply for the Data Engineer role at Sembcorp Industries Ltd About SembcorpSembcorp is a leading energy and urban solutions provider headquartered in Singapore. Led by its purpose to drive energy transition, Sembcorp delivers sustainable...

  • Data Analyst

    2 days ago


    Singapur, Singapore SHOPEE SINGAPORE PRIVATE LIMITED Full time

    Overview The Business Intelligence and Data Analytics team plays a critical role in conducting close-loop data-driven business iterations. As business intelligence specialists and data analysts, our scope of work is not limited to just performance monitoring and reporting, but also in proactively finding actionable insights to help drive high-impact business...