Data Engineer using DBT
3 weeks ago
Responsibilities:
- Engage in both exploratory data analysis to identify trends and data patterns
- Perform data preparation and data cleaning part of data migration project.
- Understand Source data and support Source to Target mapping rules
- Design and implement the ETL framework for the data warehouse
- Build and maintain DBT pipelines to transform source data into reporting data models for our visualization team
- Build the Data Ingestion framework and ingest data from source files to load into Staging layer using DBT
- Code orchestration pipelines leveraging Python to connect data ingestion, DBT, Power BI
- Perform data profiling and identify any data quality issues and report to source systems
- Using best practices build the ETL components as per mapping rules to move data into ODS; SOR layer and Data marts
- Ensure the ETL jobs are set up; configured and scheduled
- Develop data pipeline observability test scripts in DBT to monitor anomaly and freshness, and detect data quality issues
- Perform Unit testing on ETL components built and provide unit test results. Support System Integration testing and User Acceptance testing and fix any issues
Requirements
- Proven experience as a Data Engineer, with a focus on ETL development, Azure Data Factory, and DBT Cloud.
- Strong proficiency in SQL and experience working with relational databases.
- In-depth knowledge of data modeling and data warehousing concepts.
- Experience with cloud-based data platforms, particularly Azure.
- Certification in Azure Data Factory or DBT.
-
Cloud Data Engineer
4 weeks ago
Singapur, Singapore Helius Full timeAt our company, we believe in harnessing artificial intelligence to amplify human potential, creating a powerful synergy that enhances performance and opportunities in the real estate sector, and beyond. Our vision of human-AI symbiosis is about amplifying the potential of real estate agents through the power of AI, enabling them to unlock their data, serve...
-
Data Engineer Intern
4 weeks ago
Singapur, Singapore Razer Full timeJob Responsibilities :Intern will be playing a key role in the revamp of existing data pipelines and building new pipelines that are essential for data analyst/data scientist to perform analytics.By studying and understanding the data and pipelines, the intern will perform data profiling, data processing, automation and deployment to ensure that data is...
-
SIEM Use-Case Engineer
3 weeks ago
Singapur, Singapore GECO Asia Full timeJob Description:Primarily accountable for overseeing the engineering, development and maintenance / continuous improvement of threat detection use-cases scenarios within the SIEM (Splunk ES). Additionally, involved in the engineering and development of custom rules in the EDR (CrowdStrike) to swiftly detect potential threats attacks. Another responsibility...
-
Big Data Engineer
3 weeks ago
Singapur, Singapore Virtusa Full timeBig Data Engineer - 111170 Description Job Summary We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will...
-
Data Engineer
4 weeks ago
Singapur, Singapore NodeFlair Full time1. Key Outputs: Collaborate with both non-technical and technical team members from cross-functional business and IT teams to gather, elicit and document data requirements and translate them into: Data Models and Mapping Specifications Develop and govern data modeling standards, best practices, and guidelines to ensure consistency and maintainability...
-
Data Engineer
3 weeks ago
Singapur, Singapore ST Engineering Full timeTitle: Data Engineer Job ID: Location: Aero - Airport Road, SG Description: Job Description Develop web applications using ASP.Net MVC and ASP.Net Web API, following the development standards and technical design provided Understanding of best practices in software development process (SDLC) including coding standards, code reviews,...
-
Azure Data Engineer
4 weeks ago
Singapur, Singapore SMRT Corporation Ltd Full timeJob Purpose We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, you will create and maintain optimal data pipelines architecture in cloud. Overall, you will strive for efficiency by aligning data systems with business goals. To succeed in this...
-
Data Engineering Intern
3 weeks ago
Singapur, Singapore OnLoop Full timeWe are seeking a motivated and talented Data Engineering Intern with a special focus on prompt engineering to join our growing team. This internship offers an exciting opportunity to gain hands-on experience in the field of data engineering while contributing to the development of cutting-edge natural language processing (NLP) models. The ideal candidate is...
-
Data Quality Assurance Engineer
3 days ago
Singapur, Singapore TikTok Full timeAbout the team The mission of the Data Business Partner (BP) Quality Assurance Team is to build a highly robust(standardized/professional/efficient) data quality system, ensuring the better delivery of data. Our goal is to safeguard the quality of data throughout the entire lifecycle in terms of timeliness, accuracy, and stability, empowering the...
-
Data Quality Assurance Engineer, Data Platform
4 weeks ago
Singapur, Singapore TikTok Full timeAbout the team The mission of the Data Business Partner (BP) Quality Assurance Team is to build a highly mature (standardized/professional/efficient) data quality system, ensuring the better delivery of data. Our goal is to safeguard the quality of data throughout the entire lifecycle in terms of timeliness, accuracy, and stability, empowering the...
-
Data Engineer/ Senior Engineer
1 week ago
Singapur, Singapore StarHub Full timeJob Description Purpose: ⦁ The Data Platform Team is responsible for conceptualisation, implementation and management of organisation-wide data platform and solutions. ⦁ You will be part of the team responsible for data management to solve business problems, build and maintain data infrastructure to address business requirements and improve...
-
Big Data Software Engineer, Ad Data
4 weeks ago
Singapur, Singapore TikTok Full timeTeam Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...
-
Big Data Software Engineer
3 weeks ago
Singapur, Singapore TikTok Full timeTeam Introduction Ad Data is the cornerstone of every business decision that millions of advertisers make every day on Bytedance. Therefore, we are tasked with innovating on daily basis to create and maintain complex systems at large scale and continue expanding the capacity to better serve advertisers growing at exponential pace. You will share in the...
-
Azure Data Engineer
3 weeks ago
Singapur, Singapore PSA International Full timeAs a member of #TeamPSA, you will be part of a diverse and global team at the forefront of creating cargo solutions to enable more agile, resilient and sustainable supply chains. Experience first-hand and drive supply chain transformation from Singapore and through PSA’s global network of ports, logistics and supply chain solutions. Alongside, we can...
-
Database Administrator
3 weeks ago
Singapur, Singapore NTT Data Singapore Full timePosition entails all aspects of Microsoft SQL Server Instance, Sybase Data Server administrations, backup and recovery, designing for high availability, disaster recovery and replication, performance monitoring-tuning, scripting, software installation and upgrades, troubleshooting, and proactively maintaining production databases....
-
Data Engineer
3 weeks ago
Singapur, Singapore IHiS Full timeData EngineerJob Role DescriptionThe Data Engineer supports the implementation of data structure and architecture, master/meta-data management approach and data quality programme to facilitate access to data and information. He/She support the design, implementation and maintenance of data flow channels and data processing systems that support the...
-
Data Engineer, Quantitative Strategy
3 weeks ago
Singapur, Singapore GovTech Singapore Full timeThe Government Technology Agency (GovTech) is the lead agency driving Singapore’s Smart Nation initiatives and public sector digital transformation. As the Centre of Excellence for Infocomm Technology and Smart Systems (ICT & SS), GovTech develops the Singapore Government’s capabilities in Data Science & Artificial Intelligence, Application Development,...
-
Associate Data Engineer
2 weeks ago
Singapur, Singapore Aon Full timeAssociate Data Engineer Aon is in the business of better decisions At Aon, we shape decisions for the better to protect and enrich the lives of people around the world. As an organization, we are united through trust as one inclusive, diverse team, and we are passionate about helping our colleagues and clients succeed. About ACIA Aon’s Centers...
-
DevOps & Data Engineer
3 weeks ago
Singapur, Singapore HUMMINGBIRD BIOSCIENCE PTE. LTD. Full timeRoles & ResponsibilitiesDevOps & Data Engineer – Hummingbird BioscienceFull-time position – SG based locationABOUT HUMMINGBIRD BIOSCIENCEHummingbird Bioscience is a data-driven precision biotherapeutics team discovering and developing transformative biologic medicines for hard-to-treat diseases. The Hummingbird Bioscience model combines computational and...
-
Data Engineer
3 weeks ago
Singapur, Singapore Unison Consulting Pte Ltd Full timeImplement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...