Senior Data Engineer

2 days ago

Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $70,000 - $120,000 per year

Summary

Python + Airflow + Snowflake Data Engineer

We are seeking a highly skilled Python / Airflow Data Engineer with 7 to 8 years of experience, specifically with a strong background in AWS technologies. The ideal candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline orchestration and management.

Responsibilities

· Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.

· Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.

· Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.

· Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.

· Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.

· Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.

· Develop and maintain documentation for data pipelines, processes, and system architecture.

· Stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements

· Bachelor's degree in Computer Science, Engineering, or a related field.

· Proficiency in Python, PySpark and SQL for data processing and manipulation.

· Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.

· Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.

· Understanding of Snowflake Data Lake is preferred.

· Experience with optimizing and scaling data pipelines for performance and efficiency.

· Good understanding of data modeling, ETL processes, and data warehousing concepts.

· Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.

· Effective communication skills and the ability to articulate technical concepts to non-technical stakeholders.

Preferred Qualifications:

· AWS certification(s) related to data engineering or big data.

· Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.

· Familiarity with other data orchestration tools in addition to Apache Airflow.

· Knowledge of version control systems like Bitbucket, Git.

Job Type: Contract

Contract length: 12 months

Experience:

  • Data engineer: 6 years (Required)
  • Airflow: 4 years (Required)
  • Python: 3 years (Required)
  • AWS services, S3, Glue, EMR, Redshift, and AWS Lambda: 3 years (Required)

Work Location: In person

  • Data Engineer(Python + Airflow + Snowflake)

    1 week ago

    Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $104,000 - $130,878 per year

    ResponsibilitiesDesign, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.Implement data integration and transformation processes to ensure optimal performance and...

  • Senior Data Engineer

    3 days ago

    Yishun, Singapore Rapsys Technologies Full time

    **Summary** Python + Airflow + Snowflake Data Engineer **Responsibilities** - Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. - Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment. - Implement data integration and...

  • Senior Engineer, Data Center Facilities Management

    1 week ago

    Bedok New Town, Singapore 1-Net Singapore Pte Ltd Full time $80,000 - $120,000 per year

    COMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...

  • Data Centre Operator

    1 week ago

    Yishun New Town, Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time $60,000 - $80,000 per year

    The DC Operations shall cover the following: 1. DC Environment Support; and 2. DC Facilities Management Services. DC Environment Support Shall monitor the facilities infrastructure in the DC to ensure that they are within the operating levels defined by the Customers. The infrastructure to be monitored shall minimally include: (a) Power infrastructure (such...

  • Drilling Engineer

    1 week ago

    Yishun New Town, Singapore Aurawoo International Full time $90,000 - $120,000 per year

    Yishun - **Job Country**: Singapore - **Job Industry**: Energy & Utilities - **Job Profile**: Drilling Engineer - **Training Duration**: 1 month - **Number of Jobs Opening**: 40 - **Salary**: Per Month) - **Job Type**:Full Time - **Visa**: Provided by company - **Food**:Provided by company - **Accommodation**:Provided by company - **Air...

  • Process Engineer

    1 week ago

    Yishun New Town, Singapore ASMPT Full time $90,000 - $120,000 per year

    Job Description As a Process Engineer at ASMPT Singapore, you will play a critical role in optimizing and characterizing semiconductor manufacturing processes. You will work closely with cross-functional teams to enhance product performance, drive automation improvements, and develop next-generation manufacturing solutions. Key Responsibilities:Optimize and...

  • Assistant Engineer, Data Center Facilities Management

    2 days ago

    Bedok New Town, Singapore 1-Net Singapore Pte Ltd Full time $45,000 - $55,000 per year

    COMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...

  • Prompt Engineering

    1 week ago

    Yishun New Town, Singapore Blue Ocean Systems Infotech Pvt. Ltd. Full time $150,000 - $200,000 per year

    HiUrgent opening for Prompt Engineering - Onsite Singapore12+ yrs experience requiredJob Summary:As a Prompt Engineer your responsibilities include working with our content creators, product teams and data scientists to develop and refine prompts, optimizing prompt performance and staying up-to-date on the latest advancements in AI, natural language...

  • Assistant Engineer, Data Centre Facilities Management

    1 week ago

    Bedok New Town, Singapore 1-Net Singapore Full time $40,000 - $60,000 per year

    COMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...

  • Data Engineer/ Analyst

    1 week ago

    Yishun New Town, Singapore Abhidi Solution Full time $10,000 - $120,000 per year

    Must have skillset with Min 4 to 8 years' experience on· Working knowledge of Azure cloud.· Hands on experience on logic app· Synapse· Strong SQL and spark knowledge· Good Communication and should self-Motivator and decision makerGood to have:· Databricks knowledge or experience.Job Types: Full-time, ContractContract length: 12 monthsPay: Up to...