Data Engineer(Python + Airflow + Snowflake)

2 weeks ago

Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time
Roles & Responsibilities

Role:Data Engineer(Python + Airflow + Snowflake)

JD:

Responsibilities


• Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.


• Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.


• Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.


• Optimize and fine-tune existing data pipelines / Airflow to improve efficiency,scalability, and maintainability.


• Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.


• Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.


• Develop and maintain documentation for data pipelines, processes, and system architecture.


• Stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements


• Bachelor's degree in Computer Science, Engineering, or a related field.


• Proficiency in Python, PySpark and SQL for data processing and manipulation.


• Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.


• Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.


• Understanding of Snowflake Data Lake is preferred.


• Experience with optimizing and scaling data pipelines for performance and efficiency.


• Good understanding of data modeling, ETL processes, and data warehousing concepts.


• Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.


• Effective communication skills and the ability to articulate technical concepts to

non-technical stakeholders.

Preferred Qualifications:


• AWS certification(s) related to data engineering or big data.


• Experience working with big data technologies like Snowflake, Spark, Hadoop, or

related frameworks.


• Familiarity with other data orchestration tools in addition to Apache Airflow.


• Knowledge of version control systems like Bitbucket, Git.

Tell employers what skills you have

Snowflake Cloud Data Warehouse
Version Control
PySpark
Airflow
Scalability
Data Modeling
Hadoop
ETL
Data Integration
Data Engineering
EMR
Python
Snowflake Computing
Orchestration
AWS Lambda
S3
System Architecture
Apache
Data Warehousing

  • Data Engineer(Python + Airflow + Snowflake)

    2 weeks ago

    Singapore Rapsys Technologies Pte Ltd. Full time

    Overview We're Hiring: Data Engineer (Python + Airflow + Snowflake) Location: Singapore, Singapore Work Mode: Work From Office Role: Data Engineer (Python + Airflow + Snowflake) What You'll Do Design and develop scalable data pipelines using Python and Airflow Build and optimize data warehouses in Snowflake Implement ETL/ELT processes for data...

  • Data Engineer(Python + Airflow + Snowflake)

    1 week ago

    Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Roles & Responsibilities Role:Data Engineer(Python + Airflow + Snowflake) JD: Responsibilities • Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. • Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment. •...

  • Snowflake Data Engineer

    2 weeks ago

    Singapore BLUE OCEAN SYSTEMS INFOTECH PTE. LTD. Full time

    Hi Immediate hiring **Job Overview**: **Key Responsibilities**: - Design and implement scalable and efficient data pipelines using **Snowflake**. - Develop and maintain **ETL/ELT workflows**to ingest, transform, and deliver clean data. - Optimize Snowflake performance (query tuning, resource management, clustering). - Collaborate with data analysts, data...

  • Data Engineer

    3 days ago

    Singapore TECHCOM SOLUTIONS (SINGAPORE) PTE. LTD. Full time

    Using Snowflake for data storage, querying, and developing optimized SQL queries, contributing to efficient data retrieval and reporting. - Leading design and implementation of ETL pipelines using GBI ETL tools, supporting data integration from multiple sources into the data warehouse, and ensuring robust data flows and system enhancements. - Creating...

  • Data Engineer

    2 weeks ago

    Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities•Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.•Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.•Implement data integration and transformation processes to ensure...

  • Data Engineer

    2 weeks ago

    Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Roles & Responsibilities Responsibilities •Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. •Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment. •Implement data integration and transformation processes to...

  • Snowflake Data Engineer

    7 days ago

    Singapore SEDHA CONSULTING PTE. LTD. Full time

    **Job Overview**: **Key Responsibilities**: - Architect and implement scalable data solutions using Snowflake Cloud Data Warehouse. - Design and develop robust ETL pipelines using Python, SnowSQL, and other data integration tools. - Write optimized SQL queries for data extraction, transformation, and analysis within Snowflake. - Develop automation scripts...

  • Senior Data Engineer

    2 weeks ago

    Singapore OX CONSULTANCY PTE. LTD. Full time

    Roles & Responsibilities At least 8 years of relevant working experience in software or data engineering with proficiency in Python. • Strong experience in unit and integration testing. • Familiarity with DevOps practices and Agile methodologies. • Strong software engineering, analytical and problem-solving skills. • Experience in AWS and Kubernetes...

  • Senior Data Engineer

    3 weeks ago

    Singapore OX CONSULTANCY PTE. LTD. Full time

    Roles & ResponsibilitiesAt least 8 years of relevant working experience in software or data engineering with proficiency in Python.• Strong experience in unit and integration testing.• Familiarity with DevOps practices and Agile methodologies.• Strong software engineering, analytical and problem-solving skills.• Experience in AWS and Kubernetes...

  • Senior Sales Engineer

    2 weeks ago

    Singapore Snowflake Full time

    Build the future of data. Join the Snowflake team. We are currently seeking a Senior Sales Engineer to join our team, specifically focusing on Singapore's enterprise customers. Your primary responsibility will be to collaborate with Sales Executives in managing enterprise accounts based in Singapore, with a specific focus on Government, Healthcare, and...