Data Engineer(Python + Airflow + Snowflake)
4 days ago
**Responsibilities**
- Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.
- Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.
- Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.
- Optimize and fine-tune existing data pipelines / Airflow to improve efficiency,scalability, and maintainability.
- Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and mínimal downtime.
- Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.
- Develop and maintain documentation for data pipelines, processes, and system architecture.
- Stay updated with the latest industry trends and best practices related to data engineering and AWS services.
**Requirements**:
- Bachelor’s degree in Computer Science, Engineering, or a related field.
- Proficiency in Python, PySpark and SQL for data processing and manipulation.
- Min 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.
- Strong knowledge of AWS services, particularly S3, Glue, EMR, Redshift, and AWS Lambda.
- Understanding of Snowflake Data Lake is preferred.
- Experience with optimizing and scaling data pipelines for performance and efficiency.
- Good understanding of data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and ability to work in a fast-paced, collaborative environment.
- Effective communication skills and the ability to articulate technical concepts to
non-technical stakeholders.
**Preferred Qualifications**:
- AWS certification(s) related to data engineering or big data.
- Experience working with big data technologies like Snowflake, Spark, Hadoop, or
related frameworks.
- Familiarity with other data orchestration tools in addition to Apache Airflow.
- Knowledge of version control systems like Bitbucket, Git.
**Job Type**: Contract
Contract length: 12 months
Pay: $8,000.00 - $9,000.00 per month
**Benefits**:
- Health insurance
Work Location: In person
-
Eti and BI Engineer
1 week ago
Yishun, Singapore amIT Global Solution Full time**Job Overview**: - The ETL Developer will be responsible for designing, developing, and maintaining ETL processes using tools such as Pentaho or Informatica. **Key Responsibilities**: - Design and implement ETL processes to extract, transform, and load data from various sources into data warehouses or data lakes. - Utilize Pentaho and Informatica for...
-
Data Centre Specialist
2 weeks ago
Yishun, Singapore Rapsys Technologies Full timeDC Environment Support Shall monitor the facilities infrastructure in the DC to ensure that they are within the operating levels defined by the Customers. The infrastructure to be monitored shall mínimally include: (a)Power infrastructure (such as Uninterrupted Power Supply and power distribution boards); (b)Cooling Infrastructure (such as computer room...
-
Production Control Engineer
2 days ago
Yishun Central, Singapore MTC Consulting Pte. Ltd. Full time $6,000 - $90,000 per yearBrief SummaryBasic up $6000 + Other allowance $2005 days work (Office hours)AWS + VB (Attractive Bonuses)Working Location: North/ West areaWell known MNC- Company ExpandingResponsibilitiesDevelop and manage the Master Production Schedule (MPS) to meet customer delivery commitments and achieve production goals.Coordinate daily with the manufacturing...
-
Test Development Engineer, Principal
2 weeks ago
Yishun, Singapore Broadcom Full timeWe are looking for an experienced Test Development Engineer. **He or she, will**: Be part of a worldwide R&D group, supporting the development of advanced fiber optic testing systems. Actively collaborate to the design of Software and Hardware for extremely high data rate and complex modulation test solutions. Work hands on with software developers,...
-
Automation Project Engineer
1 day ago
Yishun, Singapore INNOMFG PTE. LTD. Full time2 years exp **Roles & Responsibilities** Roles and responsibilities: - The Automation Project Engineer is in charge of developing Application Control Software for the automated machines developed by the company - Produce complete control solution and software design base on customer specification. - PLC based software control design & HMI software...
-
Automation & Mechanical Design Engineer
1 day ago
Yishun, Singapore INNOMFG PTE. LTD. Full time3 years exp **Roles & Responsibilities** Roles and responsibilities: - To do conceptual and detailed machine design with regard to mechanical aspects; - To prepare solid models, layouts, views and bills of materials for assigned design machine assemblies and sub-assemblies; - To prepare final design configurations, layouts, solid models and...
-
Civil Project Engineer 【good Career
3 days ago
Yishun, Singapore STAFFKING PTE LTD Full time**Benefit**: - Career Growth - Allowance, Bonus **Job Scope**: Assist in the planning, design, and implementation of civil engineering projects, including site development, infrastructure construction, and building projects. Conduct site investigations and surveys to assess project feasibility, identify potential challenges, and gather necessary data....
-
Foundation Ip Design Engineer
2 weeks ago
Yishun, Singapore Broadcom Full timePlease Note: Job Description: Foundation IP Design Engineer We are looking for energetic and passionate design engineers to join our Central Engineering Group and be part of an elite team responsible for the development of foundation IP for AI products including memory compilers, logic cells and custom macros of all types on the bleeding edge of process...
-
Foundation Ip Design Engineer
2 weeks ago
Yishun, Singapore Broadcom Full timePlease Note: Job Description: Foundation IP Design Engineer We are looking for energetic and passionate design engineers to join our Central Engineering Group and be part of an elite team responsible for the development of foundation IP for AI products including memory compilers, logic cells and custom macros of all types on the bleeding edge of process...
-
System Analyst
1 week ago
Yishun, Singapore Cognizioni IT Solutions LLC Full timeRole**:System Analyst** Location**:Singapore** Duration**:12 months** Domain**:Global Markets** The bank is looking for an experienced **System Analyst**to join its dynamic derivatives development team to lead in providing solutions as **Enterprise Market Data Services** bank-wide covering Treasury, Risks, Financial, Wholesale and Retail banking across all...