Data Engineer(spark,scala And Elastic Search)

Found in: beBee jobs SG - 2 weeks ago

Singapore search index Full time

We are seeking a talented and experienced Spark Scala Developer with expertise in Elasticsearch to join our team. As a Spark Scala Developer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch. You will collaborate with cross-functional teams to build scalable and efficient data processing pipelines and search applications. Knowledge and experience in the Compliance / AML domain will be a plus.

Responsibilities: Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation Requirements Bachelor\'s or Master\'s degree in Computer Science, Software Engineering, or a related field Min 5+ Years experience in developing Spark & Scala applications. Experience with Spark Streaming is a plus. Proficiency in Scala programming language and familiarity with functional programming concepts In-depth understanding of Apache Spark architecture, RDDs, Data Frames, and Spark SQL Experience integrating and working with Elasticsearch for data indexing and search applications. Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization Experience with distributed computing, parallel processing, and working with large datasets Familiarity with big data technologies such as Hadoop, Hive, and HDFS Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries Strong problem-solving and analytical skills with the ability to debug and resolve complex issues. Familiarity with version control systems (e.g., Git) and collaborative development workflows -

Additional Information

Career Level

Not Specified

Qualification

Not Specified

Job Specializations

,

Company Overview

Headquartered in Singapore, Unison Consulting was founded in September 2012 with the core motive to provide cutting-edge solutions and services in a variety of areas. At Unison Consulting, our mission is to empower businesses with our financial and insurance services.

Our consulting services help our clients implement risk management, risk analytics system, systems integration, business intelligence, information management, and custom development of IT solutions. We surpass all standards of quality for financial, insurance institutions and other verticals.

Our team views our clients as our partners and derives its success through the success of our clients. We are highly focused on exceeding our clients\' expectations and offering them the best possible solutions for their businesses.

Unison Consulting has a team of young, energetic, imaginative, and innovative individuals who never hold back from facing their challenges head-on. They push all boundaries to help our clients and exceed their expectations. We help our team to upscale their knowledge as required in the market, thereby to increase their personal and professional capabilities.

Additional Company Information

Registration No.

C

Industry

Computer / Information Technology (Software)

Be careful

This job ad has not been subjected to our hirer verification process. Proceed cautiously and do your own checks before providing any personal information. Search Index

  • Data Engineer

    Found in: Talent SG 2A C2 - 2 weeks ago

    Singapore JONDAVIDSON PTE. LTD. Full time

    Roles & Responsibilities Degree in computer science or a numerate subject (e.g. engineering, sciences, or mathematics) or Bachelor's/Master degree with 6 years of experience Hand-on Development experience with Spark, ScalaSpark and Distributed computing. Development and implementation experience of applications for 4-6 years. 4 to 6 years' experience...

  • Data Engineer

    2 weeks ago

    Singapore NTT Singapore Pte Ltd Full time

    Summary:We are seeking a talented and experienced Spark Scala Developer with expertise in Elasticsearch to join our team. As a Spark Scala Developer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch. Key Responsibilities:Design, develop, and implement Spark Scala...

  • Data Engineer(Spark Developer)

    Found in: Talent SG 2A C2 - 3 days ago

    Singapore UNIZEN TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesThe OpportunityLeveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data.Key Responsibilities Creating complex, enterprise-transforming...

  • Data Engineer

    Found in: Talent SG 2A C2 - 1 day ago

    Singapore LYNEER CORP (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesThe Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key Responsibilities Creating complex,...

  • Data Engineer

    Found in: Talent SG 2A C2 - 2 days ago

    Singapore IDC TECHNOLOGIES (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesThe Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key Responsibilities Creating complex,...

  • Data Engineer( Sparks Developer )

    Found in: Talent SG 2A C2 - 2 weeks ago

    Singapore TECHNOPALS PTE. LTD. Full time

    Roles & ResponsibilitiesMandatory SkillsThe Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key Responsibilities Creating complex, ...

  • Hadoop scala

    Found in: Talent SG 2A C2 - 3 days ago

    Singapore RAPSYS TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesRole:Hadoop scalaJD:Description:1. Candidate should be well versed with Scala and have applied object-oriented Scala concepts day in and day out (Inheritence, Anonymous Classes ,Scala Objects,Abstract Classes etc)2. Candidate should be well versed with Distributed Framework like Spark .3. Candidate should be well versed with Spark RDD...

  • Hadoop scala

    4 days ago

    Singapore Rapsys Technologies Pte. Ltd. Full time

    Role:Hadoop scalaJD:Description:1. Candidate should be well versed with Scala and have applied object-oriented Scala concepts day in and day out (Inheritence, Anonymous Classes ,Scala Objects, Abstract Classes etc) 2. Candidate should be well versed with Distributed Framework like Spark . 3. Candidate should be well versed with Spark RDD , Spark Dataframe ,...

  • Data Engineer( Sparks Developer )

    2 weeks ago

    Singapore Technopals Consultants Pte. Ltd. Full time

    Mandatory SkillsThe Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key ResponsibilitiesCreating complex, enterprise-transforming applications on...

  • Data Engineer(Spark Developer)

    1 day ago

    Singapore Unizen Technologies Pte. Ltd. Full time

    The Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key ResponsibilitiesCreating complex, enterprise-transforming applications on diverse, high...

  • Data Engineer

    1 day ago

    Singapore Idc Technologies (singapore) Pte. Ltd. Full time

    The Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key ResponsibilitiesCreating complex, enterprise-transforming applications on diverse, high...

  • Data Engineer

    18 hours ago

    Singapore Lyneer Corp (singapore) Pte. Ltd. Full time

    The Opportunity Leveraging the latest technology to develop mission critical applications from scratch. design, develop, test and deploy new applications as part of the strategic transformation programme, using Agile, cloud technologies, microservices and Big Data. Key ResponsibilitiesCreating complex, enterprise-transforming applications on diverse, high...

  • Data Engineer – Spark Developer

    Found in: Talent SG 2A C2 - 1 week ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service...

  • Data Engineer

    Found in: Talent SG 2A C2 - 2 weeks ago

    Singapore EVO OUTSOURCING SOLUTIONS PTE. LTD. Full time

    Roles & ResponsibilitiesJob Description:-- Able to drive functional discussion with users and tech teams- Able to work with tech teams on design and solution- Able to perform deep dive data analysis- Able to provide summary view of the deep dive data analysisJob Requirements:-- 8+ years of experience with application development in Big Data Ecosystem (such...

  • Data Engineer

    1 week ago

    Singapore Evo Outsourcing Solutions Pte. Ltd. Full time

    Job Description:-- Able to drive functional discussion with users and tech teams- Able to work with tech teams on design and solution- Able to perform deep dive data analysis- Able to provide summary view of the deep dive data analysisJob Requirements:-- 8+ years of experience with application development in Big Data Ecosystem (such as Cloudera,...

  • Scala Lead #IAC

    1 week ago

    Singapore Recruit Express Pte Ltd Full time

    ResponsibilitiesLead and mentor a team of Scala developers, providing technical guidance, code reviews, and ensuring adherence to best practices.Collaborate with cross-functional teams to gather requirements, architect solutions, and deliver high-quality software products.Design and develop scalable, reliable, and high-performance Scala applications and...

  • Data Engineer

    1 week ago

    Singapore Avn Digital Pte. Ltd. Full time

    Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service a client base across...

  • Data Engineer

    Found in: Talent SG 2A C2 - 1 week ago

    Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesJob Description1. Creating complex, enterprise-transforming applications on diverse, high-energy teams2. Working with the latest tools and techniques3. Hands-on coding, usually in a pair programming environment4. Working in highly collaborative teams and building quality code5. The candidate must exhibit a good understanding of model...

  • Tech Consultant

    Found in: Talent SG 2A C2 - 2 weeks ago

    Singapore RECRUIT EXPRESS PTE LTD Full time

    Roles & ResponsibilitiesResponsibilities Lead and mentor a team of Scala developers, providing technical guidance, code reviews, and ensuring adherence to best practices. Collaborate with cross-functional teams to gather requirements, architect solutions, and deliver high-quality software products. Design and develop scalable, reliable, and...

  • Data Engineer Intern

    2 weeks ago

    Singapore Shopee Full time

    Job Description:Design, build, and maintain the real-time streaming system to support timely and accurate ingestion of various types of data (e.g. User behavior, RDS, NoSQL DB, and others) into storageDesign, build, and maintain data warehouse for Shopee's e-commerce search and recommendationTranslate data requirements into scalable technical data service...