Senior Data Engineer

7 days ago

Singapur, Singapore Rapsys Technologies Pte Ltd. Full time
Senior Data Engineer

Experience Required: 5-10 Years


Position Summary:

We are seeking an experienced Senior Data Engineer with 5 to 10 years of hands-on experience in building and managing data pipelines and big data infrastructure. The ideal candidate will have a strong technical background in data engineering tools, big data ecosystems, and programming languages. This role focuses on designing, implementing, and optimizing complex data workflows using Apache Airflow, Apache NiFi, Kafka, PySpark, Spark Scala, and other key technologies. Candidates with proficiency in Python, Java, and shell scripting are highly preferred.


Key Responsibilities:

Data Pipeline Design & Management:

Design, build, and maintain scalable and reliable data pipelines to support both real-time and batch processing requirements.


Workflow Optimization & Monitoring:

Manage, monitor, and optimize workflows in Apache Airflow to ensure data quality, integrity, and system performance.


Data Integration:

Develop and integrate data flows using Apache NiFi to ensure seamless data ingestion and transformation processes.


Real-Time Data Streaming:

Work extensively with Apache Kafka for data streaming and messaging across various data sources.


Data Processing Solutions:

Implement data processing solutions using PySpark and Spark Scala to handle large-scale datasets and complex transformations.


Automation & Scripting:

Write efficient code in Python and Java to automate data workflows and support data engineering needs, utilizing shell scripting for operational tasks.


Cross-functional Collaboration:

Collaborate with cross-functional teams to understand data requirements and provide optimized engineering solutions.


Data Security & Compliance:

Ensure data security, compliance, and performance by following best practices in big data and distributed systems.


Continuous Improvement:

Continuously improve the performance, scalability, and reliability of data processing pipelines.


Required Skills and Experience:

Apache Airflow:

Extensive experience in managing, scheduling, and monitoring data pipelines.


Apache NiFi:

Strong experience in designing data flows for ingestion and transformation.


Apache Kafka:

In-depth knowledge of Kafka for real-time data streaming and messaging systems.


PySpark & Spark Scala:

Proficiency in using PySpark and Spark Scala for large-scale data processing.


Programming Languages:

Strong experience with Python and Java, with additional expertise in shell scripting.


Big Data Knowledge:

Familiarity with big data ecosystems and distributed data processing.


Problem-Solving & Teamwork:

Ability to work independently and collaboratively in a fast-paced environment, solving complex data engineering challenges.


Educational Qualifications:

Required:

Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.

Preferred Qualifications:

Cloud Experience:

Experience in cloud environments (AWS, GCP, Azure) with big data components.


Version Control & CI/CD:

Experience with version control tools (e.g., Git) and CI/CD practices in data engineering.


Analytical & Communication Skills:

Strong analytical, problem-solving, and communication skills.

  • Data Engineer

    4 weeks ago

    Singapur, Singapore NTT Data Singapore Full time

    Job Title: Data EngineerLocation: SingaporeContract Duration: 12 months renewableAbout the Role:We are seeking a highly skilled Data Engineer to join our team in Singapore. As a Data Engineer, you will be responsible for designing and building robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources...

  • Senior Automation Test Engineer

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Title: Senior Automation Test EngineerJob Summary:We are seeking a highly skilled Senior Automation Test Engineer to join our team at NTT DATA. As a key member of our automation team, you will be responsible for designing and developing automated test cases using Cucumber BDD format and Selenium automation tools.Key Responsibilities:Design and develop...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    5 months ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Data Center Solutions Engineer

    4 weeks ago

    Singapur, Singapore Vantage Data Centers Full time

    About Vantage Data CentersVantage Data Centers is a leader in the data center industry, committed to creating a workplace of inclusion, equity, respect, and acceptance. We celebrate diversity and strive to learn from each other's experiences.Job SummaryThe Customer Solutions Engineer is a key member of our sales solutions and engineering team, responsible...

  • Senior Software Engineer

    4 weeks ago

    Singapur, Singapore NTT Data Singapore Full time

    Job Summary:We are seeking a highly skilled Senior Software Engineer to join our team at NTT Data Singapore. The ideal candidate will have a strong background in system development and a passion for designing and implementing innovative solutions.Key Responsibilities:Design and implement a CI/CD process for an enterprise-wide visualization platform,...

  • Data Engineer

    2 months ago

    Singapur, Singapore NTT Data Singapore Full time

    Work Location: SingaporeContract Duration - 12 months renewable  Responsibilities: Work closely with data stewards, data analysts and business end-users to implement and support data solutions. Design and build robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources using Python. Familiar with SDLC...

  • Senior Data Engineer

    4 weeks ago

    Singapur, Singapore Careers@Gov Full time

    Job Title: Senior / Engineer (AMDA - Data Strategy & System)About the Role:We are seeking a highly skilled Senior / Engineer to join our Asset Management team. As a key member of our team, you will be responsible for developing an integrated Asset Management System to gain a deeper understanding of railway assets, including their performance and...

  • Senior Associate Software Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Data Applications Engineer

    4 weeks ago

    Singapur, Singapore Careers@Gov Full time

    Job Title: Senior Data Applications EngineerAs a Senior Data Applications Engineer, you will play a crucial role in advancing technological innovation, particularly in managing and analyzing data collected through distributed sensors and other channels.Key Responsibilities:Leading industry and internal projects focused on innovative data application...

  • Senior Data Engineer

    5 months ago

    Singapur, Singapore HKT Full time

    This is a Full-time position with a Hybrid working arrangement (1-2 days in office). Role Overview We are passionate about harnessing the power of data to drive innovation and empower businesses. We are seeking a highly technical and experienced Senior Data Engineer to join our dynamic team. The successful candidate will be a key player in...

  • Senior Associate Software Development Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Engineer, Data Center Infrastructure Management

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Enterprise Architect

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Principal Enterprise Architect to join our team at NTT DATA. As a key member of our organization, you will play a pivotal role in shaping the future of our clients' businesses through innovative technology solutions.Key ResponsibilitiesSupport client managers in account planning, opportunity seeding,...

  • Senior Director, Design Management

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Software Development Engineer

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    About the RoleNTT DATA is seeking a talented Software Development Engineer to join our team. As a Software Development Engineer, you will work with senior members of the development team on complex projects that require multiple steps in the software development process.Key ResponsibilitiesWriting and debugging code to ensure that an application works as...

  • Senior Associate IT Data Centre Operations Analyst

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Network Engineer

    9 hours ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Enterprise Architect

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Enterprise Architect to join our team at NTT DATA. As a key member of our organization, you will play a critical role in shaping the future of our clients' businesses through the development of innovative enterprise architecture solutions.Key ResponsibilitiesSupport client managers in account planning,...

  • Senior Manager, Design, APAC

    6 months ago

    Singapur, Singapore Vantage Data Centers Full time

    Description Vantage is actively seeking an experienced Senior Design Manager for the APAC market to strategically lead and enhance design and engineering efforts in the development delivery process. The role involves holistic ownership of the design life cycle, managing engineering teams, overseeing procurement processes, and serving as the primary...

  • Senior Data Engineer

    4 weeks ago

    Singapur, Singapore GIC Full time

    About GICGIC is one of the world's largest sovereign wealth funds, with a global presence and a commitment to long-term investing. As a member of our Data Strategy Group, you will play a key role in shaping our data infrastructure and driving business insights.Job SummaryWe are seeking an experienced Senior Data Engineer to join our Research Data Engineering...