Data Engineer(Python + Airflow + Snowflake)
4 days ago
**Responsibilities**
- Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.
- Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.
- Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.
- Optimize and fine-tune existing data pipelines / Airflow to improve efficiency,scalability, and maintainability.
- Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and mínimal downtime.
- Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.
- Develop and maintain documentation for data pipelines, processes, and system architecture.
- Stay updated with the latest industry trends and best practices related to data engineering and AWS services.
**Requirements**:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- Proficiency in Python, PySpark and SQL for data processing and manipulation.
- Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.
- Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.
- Understanding of Snowflake Data Lake is preferred.
- Experience with optimizing and scaling data pipelines for performance and efficiency.
- Good understanding of data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
- Effective communication skills and the ability to articulate technical concepts to
non-technical stakeholders.
**Preferred Qualifications**:
- AWS certification(s) related to data engineering or big data.
- Experience working with big data technologies like Snowflake, Spark, Hadoop, or
related frameworks.
- Familiarity with other data orchestration tools in addition to Apache Airflow.
- Knowledge of version control systems like Bitbucket, Git.
**Job Type**: Contract
Contract length: 12 months
Pay: $8,000.00 - $9,000.00 per month
**Benefits**:
- Health insurance
Work Location: In person
-
Data Engineer(Python + Airflow + Snowflake)
2 weeks ago
Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $100,000 - $120,000 per yearResponsibilitiesDesign, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.Implement data integration and transformation processes to ensure optimal performance and...
-
Snowflake Data Engineer
13 hours ago
Yishun, Singapore Blue Ocean Systems Infotech Pvt. Ltd. Full timeHi Immediate Hiring - Design and implement scalable and efficient data pipelines using **Snowflake**. - Develop and maintain **ETL/ELT workflows** to ingest, transform, and deliver clean data. - Optimize Snowflake performance (query tuning, resource management, clustering). - Collaborate with data analysts, data scientists, and business stakeholders to...
-
Senior Data Engineer
4 days ago
Yishun New Town, Singapore Rapsys Technologies Pte. Ltd Full time $70,000 - $120,000 per yearSummaryPython + Airflow + Snowflake Data EngineerWe are seeking a highly skilled Python / Airflow Data Engineer with 7 to 8 years of experience, specifically with a strong background in AWS technologies. The ideal candidate will have a deep understanding of Apache Airflow and its integration within AWS ecosystem, enabling efficient data pipeline...
-
Eti and BI Engineer
1 week ago
Yishun, Singapore amIT Global Solution Full time**Job Overview**: - The ETL Developer will be responsible for designing, developing, and maintaining ETL processes using tools such as Pentaho or Informatica. **Key Responsibilities**: - Design and implement ETL processes to extract, transform, and load data from various sources into data warehouses or data lakes. - Utilize Pentaho and Informatica for...
-
Data Centre Specialist
2 weeks ago
Yishun, Singapore Rapsys Technologies Full timeDC Environment Support Shall monitor the facilities infrastructure in the DC to ensure that they are within the operating levels defined by the Customers. The infrastructure to be monitored shall mínimally include: (a)Power infrastructure (such as Uninterrupted Power Supply and power distribution boards); (b)Cooling Infrastructure (such as computer room...
-
IT Support/data Center Engineer
13 hours ago
Yishun, Singapore Tech Domain Full timeWe are looking for an **ONSITE IT & Network Support Engineer/Technician**to support Casual Freelancing type tasks in your area. Would you be interested in contractual tasks? you will be paid either on an hourly or daily/half-day basis. Tickets will be dispatched with respect to your area & you may support them as per your availability (weekdays,...
-
Data Engineer
6 days ago
Yishun, Singapore Abhidi Solution Full time**Must have skillset with Min 4 to 8 years' experience on** - Working knowledge of Azure cloud. - Hands on experience on logic app - Synapse - Strong SQL and spark knowledge - Good Communication and should self-Motivator and decision maker **Good to have**: - Databricks knowledge or experience. **Job Types**: Full-time, Contract Contract length: 12...
-
Etl Developer
13 hours ago
Yishun, Singapore amIT Global Solution Full time**Job Overview**: - The ETL Developer will be responsible for designing, developing, and maintaining ETL processes using tools such as Pentaho or Informatica. **Key Responsibilities**: - Design and implement ETL processes to extract, transform, and load data from various sources into data warehouses or data lakes. - Utilize Pentaho and Informatica for...
-
Patent Research Engineer
1 week ago
Yishun, Singapore Keysight Technologies, Inc. Full timeOverview: Keysight is on the forefront of technology innovation, delivering breakthroughs and trusted insights in electronic design, simulation, prototyping, test, manufacturing, and optimization. Our ~15,000 employees create world-class solutions in communications, 5G, automotive, energy, quantum, aerospace, defense, and semiconductor markets for customers...
-
Yishun, Singapore AGENSI PEKERJAAN THE SUPREME HR ADVISORY SDN. BHD. Full time**Responsibilities**: **▪ Perform regular inspections and audit of manufacturing processes to ensure compliance with quality standards.** **▪ Establish and maintain quality standards and guidelines for manufacturing processes and output.** **▪ Collect and analyse quality-related data to identify issues and areas for improvement to enhance product...