Senior Data Engineer

2 days ago

Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $70,000 - $120,000 per year

Summary

Python + Airflow + Snowflake Data Engineer

We are seeking a highly skilled Python / Airflow Data Engineer with 7 to 8 years of experience, specifically with a strong background in AWS technologies. The ideal candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline orchestration and management.

Responsibilities

· Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.

· Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.

· Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.

· Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.

· Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.

· Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.

· Develop and maintain documentation for data pipelines, processes, and system architecture.

· Stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements

· Bachelor's degree in Computer Science, Engineering, or a related field.

· Proficiency in Python, PySpark and SQL for data processing and manipulation.

· Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.

· Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.

· Understanding of Snowflake Data Lake is preferred.

· Experience with optimizing and scaling data pipelines for performance and efficiency.

· Good understanding of data modeling, ETL processes, and data warehousing concepts.

· Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.

· Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders.

Preferred Qualifications:

· AWS certification(s) related to data engineering or big data.

· Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.

· Familiarity with other data orchestration tools in addition to Apache Airflow.

· Knowledge of version control systems like Bitbucket, Git.

Job Type: Contract

Contract length: 12 months

Experience:

  • Data engineer: 6 years (Required)
  • Airflow: 4 years (Required)
  • Python: 3 years (Required)
  • AWS services, S3, Glue, EMR, Redshift, and AWS Lambda: 3 years (Required)

Work Location: In person

  • Data Engineer(Python + Airflow + Snowflake)

    2 weeks ago

    Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $100,000 - $120,000 per year

    ResponsibilitiesDesign, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.Implement data integration and transformation processes to ensure optimal performance and...

  • Senior Engineer, Data Center Facilities Management

    2 weeks ago

    Bedok New Town, Singapore 1-Net Singapore Pte Ltd Full time $80,000 - $120,000 per year

    COMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...

  • Software Engineer

    4 days ago

    Yishun New Town, Singapore ASMPT Singapore Pte Ltd Full time $90,000 - $120,000 per year

    Job Description:We are seeking a Software Engineer to develop and optimize software solutions for advanced packaging semiconductor equipment. This role involves designing, implementing, and testing high-performance software to enhance automation, control, and data processing for semiconductor manufacturing. You will collaborate with cross-functional teams to...

  • Senior Executive, Data Analytics

    7 hours ago

    Bedok New Town, Singapore PAP Community Foundation Full time $80,000 - $120,000 per year

    Key ResponsibilitiesData Management & Analytics• Collect, clean, and consolidate data from multiple sources, including Active Ageing Centre, senior care centers, operations, manpower deployment, finance, and client engagement metrics.• Conduct in-depth data analysis to identify trends, anomalies, and opportunities for improvement.• Ensure data...

  • Assistant Engineer, Data Center Facilities Management

    2 days ago

    Bedok New Town, Singapore 1-Net Singapore Pte Ltd Full time $40,000 - $60,000 per year

    COMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...

  • Software Engineer

    6 days ago

    Yishun New Town, Singapore STAFFKING PTE LTD Full time $90,000 - $120,000 per year

    Summary:5 Days, 8am-5:30pmLocation: Yishun Industrial ParkIndustry: Semiconductors & Electronics manufacturer in SingaporeMNC, great company prospectsAWS & Bonus givenResponsibilities:Design, develop, and maintain software for semiconductor equipment, including motion control, data acquisition, and process automationCollaborate with hardware and mechanical...

  • Data Engineer/ Analyst

    2 weeks ago

    Yishun New Town, Singapore Abhidi Solution Full time $120,000 - $240,000 per year

    Must have skillset with Min 4 to 8 years' experience on· Working knowledge of Azure cloud.· Hands on experience on logic app· Synapse· Strong SQL and spark knowledge· Good Communication and should self-Motivator and decision makerGood to have:· Databricks knowledge or experience.Job Types: Full-time, ContractContract length: 12 monthsPay: Up to...

  • Facilities Associate Engineer

    2 weeks ago

    Yishun New Town, Singapore Lumileds Singapore Pte Ltd Full time $3,000 - $60,000 per year

    Who We Are:Lumileds International is a global leader in LED technology, innovation, and solutions for the automotive, display, illumination, mobile, and other markets where light sources are essential. Our approximately 3,300 employees operate in over 15 countries to partner with our customers to deliver solutions for lighting, safety, and well-being.The...

  • Software Quality Engineer

    7 hours ago

    Yishun New Town, Singapore WSH Experts Pte Ltd Full time $60,000 - $100,000 per year

    Job Description & RequirementsResponsibilitiesDesign and implement comprehensive test plans, test cases, test data and test scripts to ensure the quality of software products.Collaborate with Product Owners and stakeholders to define test objectives, scope and entry/exit criteria for testing phases.Perform system testing (e.g., functional and performance...

  • Project Engineer

    2 weeks ago

    Ang Mo Kio New Town, Singapore ST Engineering Cloud & Data Centre Solutions Pte Ltd Full time $60,000 - $80,000 per year

    About ST EngineeringST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace,...