Data Engineer(Python

2 weeks ago

Singapore Unison Consulting Pte Ltd Full time

**Data Architecture and Design**:

- Collaborate with cross-functional teams to understand data requirements and design efficient and scalable data architectures.
- Develop and maintain data models, schema designs, and data flow diagrams.

**ETL Development**:

- Design, develop, and optimize Extract, Transform, Load (ETL) processes using Python and PySpark.
- Implement robust data pipelines for efficient data extraction, transformation, and loading from various sources to data warehouses.

**Data Processing and Transformation**:

- Leverage PySpark for large-scale data processing, ensuring high performance and reliability.
- Implement data transformations, aggregations, and cleansing procedures to maintain data quality.

**Data Integration**:

- Integrate data from various sources, including structured and unstructured data, ensuring consistency and accuracy.
- Work closely with data scientists and analysts to understand their data needs and provide support for data integration into analytical models.

**Performance Optimization**:

- Monitor and optimize data processing and ETL jobs for performance, scalability, and efficiency.
- Troubleshoot and resolve issues related to data pipeline performance.

**Data Quality and Governance**:

- Implement data quality checks and validation processes to ensure the accuracy and reliability of the data.
- Enforce data governance policies and best practices.

**Collaboration and Documentation**:

- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand and address their data requirements.
- Document data engineering processes, ETL workflows, and data architectures.

**Technology Research and Adoption**:

- Stay abreast of the latest trends and advancements in data engineering and recommend the adoption of new technologies and tools to enhance efficiency.

**Requirements**:

- Minimum of 7 years of hands-on experience in data engineering, with a focus on Python and PySpark.
- Proven experience in designing and implementing scalable and efficient ETL processes.
- Strong knowledge of data modeling, data warehousing concepts, and database systems.
- Experience with big data technologies and distributed computing, with a focus on PySpark.
- Proficiency in working with cloud platforms such as AWS, Azure, or Google Cloud.
- Strong problem-solving and troubleshooting skills.

  • Data Engineer

    1 week ago

    Singapore INFOCEPTS PTE. LTD. Full time

    We are hiring aspiring / proven data engineers to join us. **Location**:Tiong Bahru / Redhill **Roles and Responsibilities**: - Design, build, and maintain scalable ETL pipelines using Python. - Extract data from various sources and transform it into clean, usable formats. - Optimize data pipelines for performance and scalability. - Design logical and...

  • Python Data Engineer

    4 weeks ago

    Singapore NEURONES IT ASIA PTE. LTD. Full time

    Roles & ResponsibilitiesAs a Python Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks.Your role is : · Designing and delivering software using an agile and iterative approach based on Scrum or Kanban· Following and contributing to improvement of client's software...

  • Data Engineer

    7 days ago

    Singapore NTT Data Singapore Full time

    **The primary responsibilities**: - Lead architecture design, develop, document, and implement end-to-end data pipelines and data-driven solutions. - Define roadmap to transform data architecture focusing on scalability, performance, and stability for the entire data lifecycle; - Build data flow for data acquisition, aggregation, and modelling, using both...

  • Data Engineer

    2 weeks ago

    Singapore Singapore Technologies Engineering Ltd Full time

    Job ID: 17490 - Location: ST Engineering Jurong East Bui, SG - Description: - **ST Engineering** is a global technology, defence and engineering group with offices across Asia, Europe, the Middle East and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives. Our...

  • Senior Data Engineer

    2 days ago

    Singapore Tangspac Full time

    S - Posted by Subagio Kusuma- Managing Partner Working as data engineer for a large international bank, you will be part of the global data engineering team involved in development and implementing data solutions You will be working as data engineer with a large global bank, you will be playing key role in building complex data solution Job...

  • Python Developer

    2 weeks ago

    Singapore Global bank - Singapore Full time

    A large global investment bank is looking for experienced python data engineer to join their data engineering team. Roles and Responsiblities: - Data analysis and data mapping and modelling - Carry out development of ELT pipeline to migrate data from source systems to data solution - Perform a range of activities including data analysis, data cleansing and...

  • Data Engineer

    7 days ago

    Singapore Seagate Technology Full time

    **About our group**: Join a fast growing and exciting software development team to build and promote the expansion of an enterprise solution in Bigdata platform environment. Responsibilities include partnering with business, engineering and IT teams to quickly understand requirements and develop innovative solutions in Agile fashion. **About the role - you...

  • Data Engineer

    1 week ago

    Singapore ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. Full time

    **Job Title**:Data Engineer - Mid-level/Senior **Location**:Singapore (CBD) **Job Type**:Full-Time Permanent **Key Responsibilities**: - Design and maintain scalable ETL pipelines using **Informatica**. - Optimize **SQL**queries and **stored procedures**for data manipulation. - Lead **data migration**projects ensuring data integrity and quality. -...

  • Data Engineer(Python + Airflow + Snowflake)

    2 weeks ago

    Singapore Rapsys Technologies Pte Ltd. Full time

    Overview We're Hiring: Data Engineer (Python + Airflow + Snowflake) Location: Singapore, Singapore Work Mode: Work From Office Role: Data Engineer (Python + Airflow + Snowflake) What You'll Do Design and develop scalable data pipelines using Python and Airflow Build and optimize data warehouses in Snowflake Implement ETL/ELT processes for data...

  • Data Engineer

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$6,500 - S$10,500 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** ETL Informatica PowerBI Java Python - Evaluate new data visualization products and tools for enterprise use. - Create data pipelines, ensure data quality, and implement ETL processes to migrate and deploy data...