This is an IT support group | Senior Data Engineer
16 hours ago
Senior Data Engineer
Date:
26 Nov 2024
Location:
Singapore, Singapore
Company:
Singtel Group
Be a Part of Something BIG
We are seeking a Senior Data Engineer who will be responsible for designing and delivering data engineering products, solutions, and running operations. Ensure the efficient and sustainable operation of Singtel Unified Data Platform and Event Streaming Platform, and to build and maintain large-scale, highly available, high-performance distributed systems based on system availability and performance. You will be part of Group IT - Data & Platform Management (DPM) team. In this position, you will work with business, IT, and data professionals to drive Singtel data and insights transformation for our business and customers.
Key Responsibilities:
Develop new data solutions and accelerators that help to deploy our data platform and engineering services at scale.
Manage the full life cycle from requirement gathering and analysis, design of the architecture, and deployment.
Design and build scalable and reliable data pipelines for both batch and real-time streaming data.
Drive optimization, testing, and tooling to improve data quality and high availability.
Maintenance and troubleshooting of data pipelines and services as per SLA.
Review and approve solution design of data pipelines.
Make An Impact By:
Designing, developing, and automating large scale, high-performance distributed data processing systems (batch and/or real-time streaming).
Practicing high quality data engineering/software engineering towards building data platform infrastructure and data pipelines at scale to deliver Big Data Analytics and Data-Science initiatives.
Leading data engineering projects and data pipeline delivery with reliable, efficient, testable, & maintainable artifacts, involving ingesting & processing data from a large number and variety of data sources.
Driving Cloud data engineering practices and Cloud Lake-house re-platform drive to build & scale Modern Data Platform & Infrastructure.
Designing data models for optimal storage across data layers, workload, and presentation retrieval to meet critical business requirements and platform operational efficiency.
Developing data pipelines for both real-time stream processing and batch processing on the Data platform (Cloud and On-Prem) that meet functional/non-functional business requirements.
Building/Optimizing and Contributing to shared Data Engineering Frameworks/Tools, Data Products & standards to improve the productivity and quality of output for Data Engineers across the data platform group.
Designing and building scalable Data APIs to host Operational data and Unified Data Platform assert in Data-Mesh / Data Fabric Architecture.
Skills for Success:
Minimum of 8 years of experience in Data Engineering, Unified Data Platform (Databricks) Infrastructure, Data Warehousing, Data Analytics tools or related, in design and developing of end-to-end scalable data pipelines and data products.
Experience designing, building, and operating robust distributed data systems, and deploying high performance with reliable systems with monitoring and logging practices.
Experience building and managing data products and pipelines using scalable and resilient open-source big data technologies – Spark, Delta-Lake, Kafka, Flink, Airflow, and related distributed data processing.
Experience building scalable and reliable Data pipelines/ETL for large number and variety of data sources ingestion, processing, and data integrations.
Build and deploy performant modern data engineering & automation frameworks using programming languages such as Scala/Python and automate big data workflows such as ingestion, aggregation, ETL processing etc.
Proficiency in programming languages like Scala, Python, Java, Go, Rust or scripting languages like Bash.
Experience in handling large Unified Data Platforms (Databricks) (multiple PBs).
Experience on cloud systems like AWS, Azure, or Google Cloud Platform.
Cloud data engineering experience in at least one cloud (Azure, AWS, GCP).
Fluent with Azure cloud services. Azure Certification is a plus.
Experience with Azure – Databricks (Cloud Data Lakehouse) is a big plus.
Experience on OSS Hadoop stack Hadoop/HDFS, Yarn, Hive, HBase and Cloudera CDP/Hortonworks platform.
Experience on Event Streaming Platform, Message Queues systems like Kafka, Pulsar, Rabbit-MQ, Redis MQ.
Experience on Event Processing systems– Kafka Streaming, KSQL, Spark Streaming, Apache Flink, Apache Beam etc.
Exposure to CDC Tools– Striim, Attunity, Goldengate for Near Real Time Data Ingestion and Low Latency processing.
Experience on NoSQL & Graph databases (KeyValue/Document/Graph) and similar– Cassandra, Hbase, Tiger graph DB, cloud Native NoSQL DB.
Build, deploy and manage big data solutions with solid DevOps functions. Be able to manage CI/CD pipelines using Ansible & Terraform as well as cloud infrastructure automations.
Experience building data solutions on different data formats in distributed data processing and data storage. Data Formats like: Parquet, Avro, Protobuf, ORC, Arrow; Open Table Formats: Delta Lake, Iceberg, Hudi and Hive.
Expertise in SQL interface to tabular, relational datasets on distributed analytic engines like Databricks SQL, Synapse, Presto, Snowflake, BigQuery etc.
Excellent experience in handling relational and transactional databases like – Postgres, MySQL, Oracle and knowledge of Advanced SQL in distributed OLAP environments.
Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
Experience with data profiling and data quality tools like Apache Griffin, Deequ, and Great Expectations.
Good understanding of data engineering/software engineering best practices - including handling and logging errors, monitoring the system, fault-tolerant pipelines, scale up, data quality and ensuring a deterministic pipeline with Data Ops.
Experience in Data APIs, Domain Modeling for RESTful & GraphQL APIs, building using Spring Boot.
Able to drive DevOps and DataOps best practices like CI/CD, containerization, blue-green deployments, secrets management etc in the Data ecosystems. Also running and scaling applications on the cloud infrastructure and containerized services like Kubernetes & Openshift.
Good to have exposure in Telco Data Model and Warehouse and/or Data Lake.
Good to have exposure to Reporting/Visualization Stack – Power BI / Tableau.
Good to have exposure for Machine Learning, MLOps.
Rewards that Go Beyond:
Full suite of health and wellness benefits.
Ongoing training and development programs.
Internal mobility opportunities.
Are you ready to say hello to BIG Possibilities?
Take the leap with Singtel to unlock new opportunities and accelerate your growth. Apply now and start your empowering career
#J-18808-Ljbffr
-
This is an IT support group | Senior Data Engineer
16 hours ago
Singapore This is an IT support group Full timeSecretlab is an international gaming chair brand seating over a million users worldwide, with our key markets in the United States, Europe and Singapore, where we are headquartered. You will be a Data or Senior Data Engineer in our team, responsible for bringing Secretlab’s data infrastructure and analytics. The demand for clean serviceable stream/batch...
-
This is an IT support group | Game Data Engineer
16 hours ago
Singapore This is an IT support group Full timeFounded in 2019 in Singapore, the Recoded Studio is stimulated to develop unique games that embrace beauty and simplicity. We are dedicated to creating long-term value for not only our players, but also our fellow team members. We achieve this by encouraging individuality and teamwork so that each team member can take ownership in their own expertise while...
-
This is an IT support group | Data Engineer
16 hours ago
Singapore This is an IT support group Full timeWe are seeking a seasoned Data Engineer with over 3 years of relevant experience in data engineering, business intelligence, and AI-driven analytics with overall 10 years + of experience. The ideal candidate will have a proven ability to design scalable data pipelines, implement machine learning models, and deliver data-driven solutions that optimize...
-
Singapore This is an IT support group Full timeSenior Risk Manager APAC London Stock Exchange Group (LSEG) are looking for Risk professionals in Singapore!Role summary: Are you curious with a desire to influence 'Safe Growth' in a rapidly changing industry? This new 2nd Line of Defence Group Risk role will cover all aspects of Operational, Financial and Strategic risk, and have exposure to all...
-
This is an IT support group | Network Engineer
16 hours ago
Singapore This is an IT support group Full timeJob ID: 16534Location: ST Engineering Hub, SG About ST Engineering ST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse...
-
Singapore This is an IT support group Full timeJob ID: 13971Location: ST Engineering Hub, SG About ST Engineering ST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse...
-
Singapore This is an IT support group Full timeDo you have a passion for sustainability, a talent for understanding data trends, and an ambition to lead sustainability initiatives in an organization? We are currently looking for a Senior Sustainability Analyst to join our Asia Sustainability team at our state-of-the-art offices in Paya Lebar Quarter, Singapore. This role is suited to someone who enjoys...
-
This is an IT support group | Senior Marine Engineer
16 hours ago
Singapore This is an IT support group Full timeJob Summary: Seeking a Senior Marine Engineer with 8-15 years in marine engineering and automation. Expertise required in electrical control systems, automation design, and vessel performance optimization. Responsibilities include project management, technical troubleshooting, and cross-functional collaboration to meet timelines and project scope. Key...
-
This is an IT support group | Senior
16 hours ago
Singapore This is an IT support group Full timeCOMPANY DESCRIPTION Mediacorp is Singapore's largest content creator and national media network, operating a suite of TV channels, radio stations, and multiple digital platforms. Its mission is to engage, entertain, and enrich audiences by harnessing the power of creativity. We are committed to creating an inclusive and diverse workplace where talent...
-
This is an IT support group | Senior/Lecturer
16 hours ago
Singapore This is an IT support group Full time(What the role is)Senior/Lecturer (Big Data and Analytics) /School of Informatics & IT (What you will be working on)Conduct lessons and carry out all relevant responsibilities related to the teaching role in full-time and part-time course offerings in areas related to Data Analytics, Machine Learning, Data Engineering and Data Management, and other...
-
Singapore This is an IT support group Full timeJob Title Senior Geotechnical Design Engineer Occupation GEOTECHNICAL ENGINEER Job Description & Requirements You will be part of the Geotechnical Team that supports the Director of the department with tender projects and projects in various Gov Agents like LTA, PUB, HDB. Responsibilities Undertake geotechnical engineering design related to MRT Stations,...
-
This is an IT support group | SENIOR SITE ENGINEER
16 hours ago
Singapore This is an IT support group Full timeJob Summary: We are seeking an experienced and dedicated Senior Site Engineer to join our dynamic team in the construction industry. The Senior Site Engineer will be responsible for overseeing construction projects from inception to completion, ensuring they are completed on time, within budget, and to the highest quality standards. This role requires...
-
This is an IT support group | Associate Data Analyst
16 hours ago
Singapore This is an IT support group Full timeJob Description: Rakuten Group, Inc. is the largest e-commerce company in Japan, and the third-largest e-commerce marketplace company worldwide, with over 1.5 billion registered users. The Rakuten brand is recognized globally for its leadership and innovation, providing a variety of consumer and business-focused services including e-commerce, e-reading,...
-
Singapore This is an IT support group Full timeResponsibilities Provide administrative support to the Senior Production Executive and overall department to ensure smooth running of the operation Provide training and guidance for Production Assistants Data entry of daily production input and output including volume produced, new material consumed, material and products on-hold/defect Compile information...
-
Singapore This is an IT support group Full timeResponsibilities SIGE EPI tool daily maintenance, and in-line SPC/defect/Equipment abnormality handling and product disposition New product pilot wafer handling, APC setup and maintenance New tool evaluation and establish process parameters for new EPI tool ensure line stability Collaborate with cross functional teams to drive capability, productivity...
-
This is an IT support group | Senior Engineer
16 hours ago
Singapore This is an IT support group Full timeElement has an opportunity for a Senior Engineer (6-month contract)to join our rapidly expanding team. As a member of the operations team, the Engineer position performs a variety of routine and non-routine testing assignments within the department assigned and ensures timely and accurate test results under minimal direct supervision. Job Duties:...
-
This is an IT support group | ESG Data Lead/Analyst
16 hours ago
Singapore This is an IT support group Full timeAbout Us CapitaLand Group (CapitaLand) is one of Asia’s largest diversified real estate groups. Headquartered in Singapore, CapitaLand’s portfolio focuses on real estate investment management and real estate development, and spans across more than 260 cities in over 40 countries. Job Description As an ESD Data Expert, you will be responsible for...
-
Singapore This is an IT support group Full timeAs a publicly listed company, we are dedicated to integrating sustainability into every aspect of our operations and setting industry standards for environmental and social governance. Your efforts will be crucial in ensuring the organization not only stays resilient but also champions sustainable growth and leadership in the marketplace. This role will...
-
Singapore This is an IT support group Full timeAbout Us Located on Jurong Island, the petrochemical hub of Singapore, SRC operates a refinery that is capable of processing 290,000 barrels of crude oil per day to produce fuel products and chemical feedstocks that are supplied to domestic markets and overseas export markets under the brand labels of its shareholders. Job Description Selected candidate...
-
Singapore This is an IT support group Full timeTHE COMPANY Our Client is a renowned organisation in the Medical Device industry with a strong presence in APAC and Singapore. They are looking for a Principal/Senior R&D Mechanical Engineer to join their team in Singapore, reporting to the R&D Director. THE ROLE The Principal/Senior R&D Mechanical Engineer is responsible for: Mechanical Design of new...