Data Engineer(Python + Airflow + Snowflake)
1 day ago
Responsibilities
- Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.
- Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.
- Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.
- Optimize and fine-tune existing data pipelines / Airflow to improve efficiency,scalability, and maintainability.
- Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.
- Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.
- Develop and maintain documentation for data pipelines, processes, and system architecture.
- Stay updated with the latest industry trends and best practices related to data engineering and AWS services.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proficiency in Python, PySpark and SQL for data processing and manipulation.
- Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.
- Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.
- Understanding of Snowflake Data Lake is preferred.
- Experience with optimizing and scaling data pipelines for performance and efficiency.
- Good understanding of data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
- Effective communication skills and the ability to articulate technical concepts to
non-technical stakeholders.
Preferred Qualifications:
- AWS certification(s) related to data engineering or big data.
- Experience working with big data technologies like Snowflake, Spark, Hadoop, or
related frameworks.
- Familiarity with other data orchestration tools in addition to Apache Airflow.
- Knowledge of version control systems like Bitbucket, Git.
Job Type: Contract
Contract length: 12 months
Pay: $8, $9,000.00 per month
Benefits:
- Health insurance
Work Location: In person
-
Senior Data Engineer
2 weeks ago
Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $70,000 - $120,000 per yearSummaryPython + Airflow + Snowflake Data EngineerWe are seeking a highly skilled Python / Airflow Data Engineer with 7 to 8 years of experience, specifically with a strong background in AWS technologies. The ideal candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline...
-
Senior Data Engineer
1 week ago
Yishun, Singapore Rapsys Technologies Full time**Summary** Python + Airflow + Snowflake Data Engineer **Responsibilities** - Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. - Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment. - Implement data integration and...
-
Data Engineer
1 day ago
Ang Mo Kio New Town, Singapore ST Engineering Geo-Insights Pte Ltd Full time $100,000 - $120,000 per yearAbout ST EngineeringST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace, smart...
-
Process Engineer
6 days ago
Yishun New Town, Singapore ASMPT Singapore Pte Ltd Full time $80,000 - $120,000 per yearAs a Process Engineer at ASMPT Singapore, you will play a critical role in optimizing and characterizing semiconductor manufacturing processes. You will work closely with cross-functional teams to enhance product performance, drive automation improvements, and develop next-generation manufacturing solutions.Key Responsibilities:Define process flow,...
-
Vision Engineer
5 days ago
Yishun New Town, Singapore ASMPT Singapore Pte Ltd Full time $120,000 - $180,000 per yearAs a Computer Vision Engineer at ASMPT Singapore, you will develop and implement cutting-edge vision solutions for high-precision semiconductor manufacturing and automation. You will work on advanced image processing, machine learning algorithms, and real-time vision inspection systems to enhance process efficiency, yield, and product quality.Key...
-
DevOps Engineer
1 day ago
Ang Mo Kio New Town, Singapore ST Engineering Cloud & Data Centre Solutions Pte Ltd Full time $80,000 - $120,000 per yearAbout ST EngineeringST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace,...
-
Data Engineer/ Analyst
1 day ago
Yishun New Town, Singapore Abhidi Solution Full time $120,000 - $240,000 per yearMust have skillset with Min 4 to 8 years' experience on· Working knowledge of Azure cloud.· Hands on experience on logic app· Synapse· Strong SQL and spark knowledge· Good Communication and should self-Motivator and decision makerGood to have:· Databricks knowledge or experience.Job Types: Full-time, ContractContract length: 12 monthsPay: Up to...
-
Data Scientist, Blockchain/crypto
2 days ago
Bishan New Town, Singapore Carro Singapore Full timeIndustry - TechnologyCity - Sin Ming LaneProvince - Central SingaporeCountry - SingaporePostal Code - 573953Job Description: Data Scientist (Blockchain/Crypto - SG) Carro is the largest car marketplace in South East Asia - our mission is to be the marketplace of choice for all automotive needs. At Carro, we catalyse change in the automotive ecosystem by...
-
Automation Mechanical Design Engineer
1 week ago
Yishun New Town, Singapore INNOMFG PTE LTD Full time $80,000 - $120,000 per yearRoles and responsibilities:To do conceptual and detailed machine design with regard to mechanical aspects.To prepare solid models, layouts, views and bills of materials for assigned design machine assemblies and sub-assemblies.To prepare final design configurations, layouts, solid models and production/engineering data.To assist in preparation of machine...
-
Bedok New Town, Singapore 1-Net Singapore Pte Ltd Full time $40,000 - $60,000 per yearCOMPANY DESCRIPTION 1-Net manages carrier-neutral Internet Data Centers, along with providing a comprehensive range of integrated services including network connectivity, managed services, work area recovery suite, cloud and media delivery services. 1-Net's Data Center Corridor initiative currently interconnects key data centers in Singapore to facilities...