Senior Data Engineer
1 month ago
Experience Required: 5-10 Years
Position Summary:
We are seeking an experienced Senior Data Engineer with 5 to 10 years of hands-on experience in building and managing data pipelines and big data infrastructure. The ideal candidate will have a strong technical background in data engineering tools, big data ecosystems, and programming languages. This role focuses on designing, implementing, and optimizing complex data workflows using Apache Airflow, Apache NiFi, Kafka, PySpark, Spark Scala, and other key technologies. Candidates with proficiency in Python, Java, and shell scripting are highly preferred.
Key Responsibilities:
Data Pipeline Design & Management:
Design, build, and maintain scalable and reliable data pipelines to support both real-time and batch processing requirements.
Workflow Optimization & Monitoring:
Manage, monitor, and optimize workflows in Apache Airflow to ensure data quality, integrity, and system performance.
Data Integration:
Develop and integrate data flows using Apache NiFi to ensure seamless data ingestion and transformation processes.
Real-Time Data Streaming:
Work extensively with Apache Kafka for data streaming and messaging across various data sources.
Data Processing Solutions:
Implement data processing solutions using PySpark and Spark Scala to handle large-scale datasets and complex transformations.
Automation & Scripting:
Write efficient code in Python and Java to automate data workflows and support data engineering needs, utilizing shell scripting for operational tasks.
Cross-functional Collaboration:
Collaborate with cross-functional teams to understand data requirements and provide optimized engineering solutions.
Data Security & Compliance:
Ensure data security, compliance, and performance by following best practices in big data and distributed systems.
Continuous Improvement:
Continuously improve the performance, scalability, and reliability of data processing pipelines.
Required Skills and Experience:
Apache Airflow:
Extensive experience in managing, scheduling, and monitoring data pipelines.
Apache NiFi:
Strong experience in designing data flows for ingestion and transformation.
Apache Kafka:
In-depth knowledge of Kafka for real-time data streaming and messaging systems.
PySpark & Spark Scala:
Proficiency in using PySpark and Spark Scala for large-scale data processing.
Programming Languages:
Strong experience with Python and Java, with additional expertise in shell scripting.
Big Data Knowledge:
Familiarity with big data ecosystems and distributed data processing.
Problem-Solving & Teamwork:
Ability to work independently and collaboratively in a fast-paced environment, solving complex data engineering challenges.
Educational Qualifications:
Required:
Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
Preferred Qualifications:
Cloud Experience:
Experience in cloud environments (AWS, GCP, Azure) with big data components.
Version Control & CI/CD:
Experience with version control tools (e.g., Git) and CI/CD practices in data engineering.
Analytical & Communication Skills:
Strong analytical, problem-solving, and communication skills.
-
Senior Associate Software Engineer
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Engineer, Data Center Infrastructure Management
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Associate Software Development Engineer
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Data Engineer
6 months ago
Singapur, Singapore HKT Full timeThis is a Full-time position with a Hybrid working arrangement (1-2 days in office). Role Overview We are passionate about harnessing the power of data to drive innovation and empower businesses. We are seeking a highly technical and experienced Senior Data Engineer to join our dynamic team. The successful candidate will be a key player in...
-
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Director, Design Management
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Network Engineer
4 weeks ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Manager, Design, APAC
7 months ago
Singapur, Singapore Vantage Data Centers Full timeDescription Vantage is actively seeking an experienced Senior Design Manager for the APAC market to strategically lead and enhance design and engineering efforts in the development delivery process. The role involves holistic ownership of the design life cycle, managing engineering teams, overseeing procurement processes, and serving as the primary...
-
Azure DevOps Engineer
3 months ago
Singapur, Singapore FL DATA Full timeFrontline Data Solutions provides web-based (SaaS) environmental, health, and safety software to some of the world's leading chemical, oil & gas, and manufacturing companies. Frontline's work culture combines the fast-paced, entrepreneurial culture of a startup software company with the stability and professionalism that reflects our 25-year history.The...
-
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Data Engineer
4 weeks ago
Singapur, Singapore Centre for Strategic Infocomm Technologies Full timeRoleAs a senior data engineer in a team that is responsible for mission-critical data analytics systems, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. You will be involved in complex technical projects to collect, parse and process large datasets with the aim to turn them...
-
Senior Manager, Automation Systems, Design
3 months ago
Singapur, Singapore Vantage Data Centers Full timeAbout Vantage Data Centers Vantage is committed to being a workplace of inclusion, equity, respect and acceptance. We celebrate diversity and intentionally seek out opportunities to learn from one another’s experience. Vantage Data Centers powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud...
-
Data Engineer
4 months ago
Singapur, Singapore Centre for Strategic Infocomm Technologies Full timeAbout this roleWe are seeking a Data Engineer to join our Data & Systems Engineering team. In this role, you will be responsible for enhancing and optimizing the data flow between our systems and refining our data pipeline architecture. You will collaborate closely with cross-functional teams to gather requirements, design scalable solutions, and implement...
-
Threat Prevention Engineer
2 months ago
Singapur, Singapore NTT Data Singapore Full timeJob Description & Requirements Primary responsibilities will be to provide technical consulting for Threat Prevention (Web Proxy) solutions, traffic analysis and transforming the business requirement into PAC file, proxy policy and/or firewall/load balancing policy design. In addition, the engineer will be participating those Threat Prevention infrastructure...
-
Senior Data Engineer
6 months ago
Singapur, Singapore SAP Full timeBring out your best SAP innovations help more than four hundred thousand customers worldwide work together more efficiently and use business insight more effectively. Originally known for leadership in enterprise resource planning (ERP) software, SAP has evolved to become a market leader in end-to-end business application software and related services...
-
Senior Data Engineer, DA(MRO), GEC
7 months ago
Singapur, Singapore ST Engineering Full timeTitle: Senior Data Engineer, DA(MRO), GEC Job ID: Location: ST Engineering Hub, SG Description: Job Description The key responsibilities include the design, execution and maintenance of critical data pipelines supporting a myriad of data-related use cases. A senior data engineer is encouraged to take ownership of the pipelines, to...
-
MS Engineer
3 weeks ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Project Manager
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Data Engineer
7 months ago
Singapur, Singapore Helius Full timeResponsibilities Design and develop data pipelines and data infrastructure to support efficient and scalable data processing and analytics. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions. Implement and maintain data integration and ETL (Extract, Transform,...
-
Senior Networking Sales Specialist
2 months ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
