Data Engineer using DBT

3 weeks ago


Singapur, Singapore Unison Consulting Pte Ltd Full time

Responsibilities:

  • Engage in both exploratory data analysis to identify trends and data patterns
  • Perform data preparation and data cleaning part of data migration project. 
  • Understand Source data and support Source to Target mapping rules
  • Design and implement the ETL framework for the data warehouse
  • Build and maintain DBT pipelines to transform source data into reporting data models for our visualization team
  • Build the Data Ingestion framework and ingest data from source files to load into Staging layer using DBT
  • Code orchestration pipelines leveraging Python to connect data ingestion, DBT, Power BI
  • Perform data profiling and identify any data quality issues and report to source systems 
  • Using best practices build the ETL components as per mapping rules to move data into ODS; SOR layer and Data marts 
  • Ensure the ETL jobs are set up; configured and scheduled  
  • Develop data pipeline observability test scripts in DBT to monitor anomaly and freshness, and detect data quality issues
  • Perform Unit testing on ETL components built and provide unit test results. Support System Integration testing and User Acceptance testing and fix any issues 

 

Requirements

  • Proven experience as a Data Engineer, with a focus on ETL development, Azure Data Factory, and DBT Cloud.
  • Strong proficiency in SQL and experience working with relational databases.
  • In-depth knowledge of data modeling and data warehousing concepts.
  • Experience with cloud-based data platforms, particularly Azure.
  • Certification in Azure Data Factory or DBT.

  • Cloud Data Engineer

    4 weeks ago


    Singapur, Singapore Helius Full time

    At our company, we believe in harnessing artificial intelligence to amplify human potential, creating a powerful synergy that enhances performance and opportunities in the real estate sector, and beyond. Our vision of human-AI symbiosis is about amplifying the potential of real estate agents through the power of AI, enabling them to unlock their data, serve...

  • Data Engineer Intern

    4 weeks ago


    Singapur, Singapore Razer Full time

    Job Responsibilities :Intern will be playing a key role in the revamp of existing data pipelines and building new pipelines that are essential for data analyst/data scientist to perform analytics.By studying and understanding the data and pipelines, the intern will perform data profiling, data processing, automation and deployment to ensure that data is...


  • Singapur, Singapore GECO Asia Full time

    Job Description:Primarily accountable for overseeing the engineering, development and maintenance / continuous improvement of threat detection use-cases scenarios within the SIEM (Splunk ES). Additionally, involved in the engineering and development of custom rules in the EDR (CrowdStrike) to swiftly detect potential threats attacks. Another responsibility...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer - 111170 Description Job Summary We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore NodeFlair Full time

    1. Key Outputs:  Collaborate with both non-technical and technical team members from cross-functional business and IT teams to gather, elicit and document data requirements and translate them into: Data Models and Mapping Specifications Develop and govern data modeling standards, best practices, and guidelines to ensure consistency and maintainability...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore ST Engineering Full time

    Title: Data Engineer Job ID: Location: Aero - Airport Road, SG Description: Job Description Develop web applications using ASP.Net MVC and ASP.Net Web API, following the development standards and technical design provided Understanding of best practices in software development process (SDLC) including coding standards, code reviews,...

  • Azure Data Engineer

    4 weeks ago


    Singapur, Singapore SMRT Corporation Ltd Full time

    Job Purpose We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, you will create and maintain optimal data pipelines architecture in cloud. Overall, you will strive for efficiency by aligning data systems with business goals.  To succeed in this...


  • Singapur, Singapore OnLoop Full time

    We are seeking a motivated and talented Data Engineering Intern with a special focus on prompt engineering to join our growing team. This internship offers an exciting opportunity to gain hands-on experience in the field of data engineering while contributing to the development of cutting-edge natural language processing (NLP) models. The ideal candidate is...


  • Singapur, Singapore TikTok Full time

    About the team The mission of the Data Business Partner (BP) Quality Assurance Team is to build a highly robust(standardized/professional/efficient) data quality system, ensuring the better delivery of data. Our goal is to safeguard the quality of data throughout the entire lifecycle in terms of timeliness, accuracy, and stability, empowering the...


  • Singapur, Singapore TikTok Full time

    About the team The mission of the Data Business Partner (BP) Quality Assurance Team is to build a highly mature (standardized/professional/efficient) data quality system, ensuring the better delivery of data. Our goal is to safeguard the quality of data throughout the entire lifecycle in terms of timeliness, accuracy, and stability, empowering the...


  • Singapur, Singapore StarHub Full time

    Job Description Purpose: ⦁ The Data Platform Team is responsible for conceptualisation, implementation and management of organisation-wide data platform and solutions. ⦁ You will be part of the team responsible for data management to solve business problems, build and maintain data infrastructure to address business requirements and improve...


  • Singapur, Singapore TikTok Full time

    Team Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...


  • Singapur, Singapore TikTok Full time

    Team Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...

  • Azure Data Engineer

    3 weeks ago


    Singapur, Singapore PSA International Full time

    As a member of #TeamPSA, you will be part of a diverse and global team at the forefront of creating cargo solutions to enable more agile, resilient and sustainable supply chains. Experience first-hand and drive supply chain transformation from Singapore and through PSA’s global network of ports, logistics and supply chain solutions. Alongside, we can...


  • Singapur, Singapore NTT Data Singapore Full time

    Position entails all aspects of Microsoft SQL Server Instance, Sybase Data Server administrations, backup and recovery, designing for high availability, disaster recovery and replication, performance monitoring-tuning, scripting, software installation and upgrades, troubleshooting, and proactively maintaining production databases....

  • Data Engineer

    3 weeks ago


    Singapur, Singapore IHiS Full time

    Data EngineerJob Role DescriptionThe Data Engineer supports the implementation of data structure and architecture, master/meta-data management approach and data quality programme to facilitate access to data and information. He/She support the design, implementation and maintenance of data flow channels and data processing systems that support the...


  • Singapur, Singapore GovTech Singapore Full time

    The Government Technology Agency (GovTech) is the lead agency driving Singapore’s Smart Nation initiatives and public sector digital transformation. As the Centre of Excellence for Infocomm Technology and Smart Systems (ICT & SS), GovTech develops the Singapore Government’s capabilities in Data Science & Artificial Intelligence, Application Development,...


  • Singapur, Singapore Aon Full time

    Associate Data Engineer Aon is in the business of better decisions At Aon, we shape decisions for the better to protect and enrich the lives of people around the world. As an organization, we are united through trust as one inclusive, diverse team, and we are passionate about helping our colleagues and clients succeed. About ACIA Aon’s Centers...


  • Singapur, Singapore HUMMINGBIRD BIOSCIENCE PTE. LTD. Full time

    Roles & ResponsibilitiesDevOps & Data Engineer – Hummingbird BioscienceFull-time position – SG based locationABOUT HUMMINGBIRD BIOSCIENCEHummingbird Bioscience is a data-driven precision biotherapeutics team discovering and developing transformative biologic medicines for hard-to-treat diseases. The Hummingbird Bioscience model combines computational and...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...