See more Collapse

Data Engineer

3 months ago

Singapur, Singapore Nityo Infotech Full time
Responsibilities:•Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data•Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data•Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch•Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions•Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch•Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards•Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration•Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components•Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovationRequirements:•Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field •Strong experience in developing Spark applications. Experience with Spark Streaming is a plus•Proficiency in Scala programming language and familiarity with functional programming concepts•In-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL•Experience integrating and working with Elasticsearch for data indexing and search applications•Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization•Experience with distributed computing, parallel processing, and working with large datasets•Familiarity with big data technologies such as Hadoop, Hive, and HDFS•Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries•Strong problem-solving and analytical skills with the ability to debug and resolve complex issues•Familiarity with version control systems (, Git) and collaborative development workflows•Excellent communication and teamwork skills with the ability to work effectively in cross-functional teams

Experience Required

2 - 9 Years

Industry Type

IT

Employment Type

Contract

Location

Singapore

We have other current jobs related to this field that you can find below

  • Transmission Network Engineer

    4 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Network and Security Engineer

    1 week ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Associate SRE Engineer

    7 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Associate DevOps Engineer

    7 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior DevOps Engineer

    3 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Network & Security Engineer (L2)

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Network & Security Engineer (L3)

    4 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Want to be a part of our team?A Senior Associate Network Engineer is a developing technical professional responsible for leveraging their networking skills and experience to support, design, and optimise NTT's network infrastructure. This role works closely with network engineers, architects, and cross-functional teams to ensure the...

  • Senior Associate Infrastructure Engineering and Operations Engineer

    3 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Networking Service Delivery Field Support Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Security and Operations Engineers-L3

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Associate DevOps Engineer

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Data Engineer

    2 weeks ago

    Singapur, Singapore Nityo Infotech Full time

    Responsibilities:•Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives•Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions•Ensure data quality and integrity throughout the data processing...

  • Cross Technology Managed Services Engineer

    4 days ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Transmission Network Engineer

    4 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • Transmission Network Engineer

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today’s ‘iNTTerconnected’ world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...

  • TS Implementation Engineer L2

    2 weeks ago

    Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Machine Learning Engineer

    3 weeks ago

    Singapur, Singapore NTT Data Singapore Full time

    Job Description:Conduct reviews for compliance of the ML models in accordance with overall platform governance principles such as versioning, data / model lineage, code best practices and provide feedback to data scientists for potential improvementsDevelop pipelines for continuous operation, feedback and monitoring of ML models leveraging best practices...

  • Data Engineer

    2 hours ago

    Singapur, Singapore Amazon Asia-Pacific Holdings Private Limited - D43 Full time

    Do you want to contribute to a team working on cutting edge technology, solve new problems that didn’t exist before, and have the ability to see the impact of your successes?Amazon is shaping the future of digital video entertainment. We seek Data Engineers who can help us store, process and analyze vast amounts of data as we build products in the video...