PySpark Developer

2 weeks ago

Singapore OPUS IT SERVICES PTE LTD Remote Work Freelance Full time

• Minimum 4 years experienced & motivated ETL Developer skilled in the design, development & deployment of Python & PySpark based XML Frameworks for ingestion of data from various source systems to Snowflake in AWS & Google Cloud Platform (GCP) environments.• Hands-on coding using Python, PySpark in cloud-based environment involving big-data frameworks in AWS resources like EMR, Lambda, S3 bucket, RDS, EC2, ECS, EKS etc.• Experience in ETL frameworks for data ingestion, data pipeline to Snowflake Data Warehouse at high-frequency & volume scenarios (giga / tera-bytes of data ingestions per day / month).• Optimize pyspark jobs for performance, efficiency, trouble shoot issues, ensure data quality and availability• Implement processes for automating data delivery, monitoring data quality, production deployment.• The candidate should be an expert in development with hands-on experience in leveraging Git / GitLab based repository management with understanding of Government Commercial Cloud (GCC) requirements.• Proficient in consulting communication skills - good in articulating business problem, approach for solution, respond to changes in scenarios for business problems.• Degree in Computer Science, Computer Engineering or any STEM equivalent• Familiar with working in Government Commercial Cloud (GCC) environment.• Independent and self-motivated contributor and passionate about software development

  • Pyspark Developer with SAS Knowledge

    5 days ago

    Singapore Unison Consulting Pte Ltd Full time

    Key Responsibilities: - Analyze and understand existing SAS programs and datasets - Design and develop PySpark code that replicates the functionality of existing SAS programs - Optimize PySpark code for performance and scalability - Test and validate PySpark code to ensure it produces the same results as the original SAS programs - Identify and resolve...

  • Data Engineer(Python, Pyspark, Spark

    5 days ago

    Singapore NodeFlair Full time

    **Job Summary**: **Job Type** **Seniority** Mid **Years of Experience** Information not provided **Tech Stacks** ETL AWS Google Cloud pySpark BigQuery RedShift Apache Azure Spark SQL Python **Key Responsibilities**: - Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL - Collaborate with...

  • Data Engineer

    1 week ago

    Singapore Manpower Full time

    Overview Job Reference: Industry: Information and Communications Technology Consultant Registration Number: R EA License Number: 02C3423Location: Singapore Recruiter: Kankariya Rupa Contact: Responsibilities Design, develop, and maintain data pipelines using PySpark on Databricks. Write clean, efficient, and reusable PySpark code for data ingestion,...

  • Pyspark Developer

    7 days ago

    Singapore RIDIK PTE. LTD. Full time

    Excellent Python - Technical aptitude, including understanding of data science concepts and tools (e.g. Hadoop, Spark, Python) - Knowledge on Spark, Python modelling packages, JIRA, Bitbucket - Experience of User Acceptance Testing and requirements gathering - Experience working with large and complex datasets - Effective communicator - Strong team player

  • Python Developer

    2 weeks ago

    Singapore STAFFLINK SERVICES PRIVATE LIMITED Remote Work Freelance Full time

    * 5 days work week * Location: Central * Strong in Python coding like Panda, PyArrow, PySpark * 5 months renewable contract Our client is a global leader IT consulting for 500 fortune company Responsibilities: Database Migration - Proven experience migrating relational databases to cloud-native data lake solutions Programming & Processing - Python (Pandas,...

  • Python Developer

    3 days ago

    Singapore VY SYSTEMS PTE. LTD. Full time

    Strong understanding and experience in completing projects using **Waterfall and Agile methodologies **. - Proficient in programming languages and frameworks, including **Java,.NET, Python, Dynamics CRM, SharePoint, Mobile development, Cloud technologies, **and **SSIS **. - Knowledge of automated software testing methodologies, such as:Unit...

  • Python Developer

    1 week ago

    Singapore STAFFLINK SERVICES PRIVATE LIMITED Full time

    Overview 5 days work week Location: Central Strong in Python coding like Panda, PyArrow, PySpark 5 months renewable contract Our client is a global leader IT consulting for 500 fortune company Responsibilities Database Migration – Proven experience migrating relational databases to cloud-native data lake solutions Programming & Processing – Python...

  • Python Developer

    5 days ago

    Singapore Unison Consulting Pte Ltd Full time

    Design the solution and develop the software using waterfall/Agile methodology. - Perform investigation, troubleshoot issues by developing scripts, SQL queries. - Work with deployment team/DevOps team on setting up continuous integration, build and release automation. - Provide estimates for stories during sprint planning for Agile project. - Convert...

  • Senior Data Engineer

    5 days ago

    Singapore NSEARCH GLOBAL PTE. LTD. Full time

    Our client, one of Asia-Pacific’s leading organizations is looking for: **Data Engineer** **Responsibilities**: - Design and implement enterprise-scale data architectures, including data lakes, warehouses, and real-time streaming platforms - Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data...

  • Data engineer

    1 week ago

    Singapore FLINTEX CONSULTING PTE. LTD. Full time

    Overview Data engineer (Azure)– Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure Devops Responsibilities Design, review and development of Pyspark scripts. Testing, troubleshooting of data pipelines, orchestration Implement and maintain datalake Establishing connections to source data systems such as on-prem databases, IOT devices,...