Python Data Engineer

4 weeks ago

Singapore NEURONES IT ASIA PTE. LTD. Full time
Roles & Responsibilities

As a Python Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks.

Your role is :

· Designing and delivering software using an agile and iterative approach based on Scrum or Kanban

· Following and contributing to improvement of client's software engineering practices e.g., source code management, test-driven development, automated testing, CI/CD

· Supporting the Lead Data Engineer in technical architecture and design

· Building an understanding of how data is used within client's commercial activity, and using this knowledge when working with Business Analysts or users to identify system requirements

· Analyzing and estimating IT changes giving input on technical opportunities, constraints, and trade-offs

· Creating documentation and presenting to technical and non-technical audiences · Providing handover to Support teams and providing third-line support for short periods after releases

· Owning their own learning so that they remain a technical subject matter expert.

Technical Skills

· Having solid experience of at least 7 plus years in data modeling, data warehousing, SQL Server database design and development (SQL, relational and dimensional modelling) Analytical querying using technologies such as SSAS cubes or columnar data stores, Python. · ETL tools such as SSIS, Alteryx, Azure Data Factory · Building data pipelines using code e.g. Pyspark · Modern cloud-based data architectures and technologies such as Data Lake, Delta Lake, Data Lakehouse e.g. Azure Data Lake Gen 2, Azure Delta Lake Lakehouse architecture using Databricks, Hudi or Iceberg

· NoSQL using CosmosDB, MongoDB, DynamoDB etc.

· Python scripting, including data engineering and machine learning libraries · Scripting languages such as Powershell, bash, DAX, M-language etc. (Note - We primarily use SQL, Pyspark and Python in our day-to-day work along with some coding around DAX, Bash, Powershell and other scripting languages. )

· Distributed computing mainly around Spark § Big data files formats e.g. Parquet, Avro, ORC etc.

· Big data technologies such as Hadoop, Hive, Map-Reduce, Partition design, performance tuning in Spark etc.

· Reporting and visualisation tools such as SSRS, Tableau, Power BI or Excel

Tell employers what skills you have

Tableau
Machine Learning
MongoDB
Azure
Big Data
Data Modeling
Pipelines
Hadoop
Software Engineering
Scripting
ETL
Data Engineering
SQL
Python
Power BI
Data Warehousing

  • Data Engineer

    2 weeks ago

    Singapore INFOCEPTS PTE. LTD. Full time

    We are hiring aspiring / proven data engineers to join us. **Location**:Tiong Bahru / Redhill **Roles and Responsibilities**: - Design, build, and maintain scalable ETL pipelines using Python. - Extract data from various sources and transform it into clean, usable formats. - Optimize data pipelines for performance and scalability. - Design logical and...

  • Data Engineer

    1 week ago

    Singapore NTT Data Singapore Full time

    **The primary responsibilities**: - Lead architecture design, develop, document, and implement end-to-end data pipelines and data-driven solutions. - Define roadmap to transform data architecture focusing on scalability, performance, and stability for the entire data lifecycle; - Build data flow for data acquisition, aggregation, and modelling, using both...

  • Senior Data Engineer

    5 days ago

    Singapore Tangspac Full time

    S - Posted by Subagio Kusuma- Managing Partner Working as data engineer for a large international bank, you will be part of the global data engineering team involved in development and implementing data solutions You will be working as data engineer with a large global bank, you will be playing key role in building complex data solution Job...

  • Python Developer

    2 weeks ago

    Singapore Global bank - Singapore Full time

    A large global investment bank is looking for experienced python data engineer to join their data engineering team. Roles and Responsiblities: - Data analysis and data mapping and modelling - Carry out development of ELT pipeline to migrate data from source systems to data solution - Perform a range of activities including data analysis, data cleansing and...

  • Data Engineer

    1 week ago

    Singapore Seagate Technology Full time

    **About our group**: Join a fast growing and exciting software development team to build and promote the expansion of an enterprise solution in Bigdata platform environment. Responsibilities include partnering with business, engineering and IT teams to quickly understand requirements and develop innovative solutions in Agile fashion. **About the role - you...

  • Data Engineer

    2 weeks ago

    Singapore ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. Full time

    **Job Title**:Data Engineer - Mid-level/Senior **Location**:Singapore (CBD) **Job Type**:Full-Time Permanent **Key Responsibilities**: - Design and maintain scalable ETL pipelines using **Informatica**. - Optimize **SQL**queries and **stored procedures**for data manipulation. - Lead **data migration**projects ensuring data integrity and quality. -...

  • Data Engineer(Python + Airflow + Snowflake)

    2 weeks ago

    Singapore Rapsys Technologies Pte Ltd. Full time

    Overview We're Hiring: Data Engineer (Python + Airflow + Snowflake) Location: Singapore, Singapore Work Mode: Work From Office Role: Data Engineer (Python + Airflow + Snowflake) What You'll Do Design and develop scalable data pipelines using Python and Airflow Build and optimize data warehouses in Snowflake Implement ETL/ELT processes for data...

  • Data Engineer

    2 weeks ago

    Singapore NTT Data Singapore Full time

    **Competency Required**: - Possess a degree in Computer Science/Information Technology or related fields. - At least 3 years of experience in a role focusing on development and support of data ingestion pipelines. - **Experience with building on data platforms, e.g. Snowflake.**: - **Proficient in SQL and Python.**: - Experience in continuous integration...

  • Data Engineer

    13 hours ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$5,000 - S$9,000 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 3 years **Tech Stacks** Analytics Java SQL Python Astek is proposing an opportunity for **Data Engineer (Python/Java/SQL) **to support our project based in** Singapore.** **Responsibilities** - Work closely with business analyst and...

  • Data Engineer

    5 days ago

    Singapore TECHCOM SOLUTIONS (SINGAPORE) PTE. LTD. Full time

    Using Snowflake for data storage, querying, and developing optimized SQL queries, contributing to efficient data retrieval and reporting. - Leading design and implementation of ETL pipelines using GBI ETL tools, supporting data integration from multiple sources into the data warehouse, and ensuring robust data flows and system enhancements. - Creating...