Big Data Engineer with Expertise in Hadoop and Spark

3 weeks ago


Singapore Unison Consulting Pte Ltd Full time

At Unison Consulting Pte Ltd, we are seeking an experienced Big Data Engineer to join our team. The successful candidate will have a strong background in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.

The role involves collaborating with cross-functional teams to build scalable and efficient data processing pipelines and search applications. A comprehensive understanding of Apache Spark architecture, including RDDs, DataFrames, and Spark SQL, is essential for this position.

We require the candidate to have prior experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments. Expertise in the Scala programming language and familiarity with functional programming principles are also necessary.

A solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization techniques is crucial for this role. Additionally, experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration via Kubernetes is highly desirable.

Responsibilities:

  • Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
  • Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
  • Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
  • Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
  • Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch

We offer a competitive salary of SGD 120,000 per annum, commensurate with experience, plus benefits and opportunities for career growth.

Required Skills and Qualifications:

  • More than 5 years of experience as a Data Engineer
  • Bachelor's or Master's degree in Computer Science, Software Engineering, or a related discipline
  • Possession of Quantexa certification as a Data Engineer or Data Architect, with proficiency in the tool
  • Demonstrated experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments
  • Expertise in the Scala programming language and familiarity with functional programming principles

Benefits:

  • Competitive salary of SGD 120,000 per annum
  • Opportunities for career growth and professional development
  • Comprehensive benefits package

Others:

  • We are an equal opportunities employer and welcome applications from diverse candidates
  • Only shortlisted candidates will be contacted for further assessment

  • Data Engineer

    7 days ago


    Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities:•Collaborate with business partners to understand business needs and functional requirements, translating them into Hadoop-based solutions.•Ensure seamless integration and management of large datasets, working with HDFS, Hive, HBase, Apache Kudu, and other Hadoop ecosystem tools to implement efficient data...

  • Data Engineer

    2 days ago


    Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time $8,000 - $9,500

    Responsibilities: Collaborate with business partners to understand business needs and functional requirements, translating them into Hadoop-based solutions. Ensure seamless integration and management of large datasets, working with HDFS, Hive, HBase, Apache Kudu, and other Hadoop ecosystem tools to implement efficient data processing workflows. ...


  • Singapore D L RESOURCES PTE LTD Full time

    Job Title: Big Data Engineer with Advanced Analytics ExpertiseAbout Us: D L RESOURCES PTE LTD is a leading provider of innovative data solutions. We are seeking an experienced Big Data Engineer to join our team.Role Summary: As a Big Data Engineer with Advanced Analytics Expertise, you will be responsible for designing, developing, and implementing...

  • Data Scientist

    3 weeks ago


    Singapore Unison Consulting Pte Ltd Full time

    About This RoleWe are seeking a highly skilled Data Scientist (Hadoop, Spark, Scala) to join our team at Unison Consulting Pte Ltd. The ideal candidate will have extensive experience with big data processing, Hadoop, Spark, and Scala.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines and ETL processes using Hadoop and Spark.Write...


  • Singapore ONE NORTH CONSULTING PTE. LTD. Full time

    At One North Consulting PTE. LTD., we are seeking an experienced Senior Data Engineering Specialist to lead our big data solutions team. With a strong background in data engineering and modelling, you will design and implement scalable data pipelines, ensuring seamless data integration across multiple platforms.The ideal candidate will have at least 8+ years...


  • Singapore KEWMANN PTE. LTD. Full time

    Job OverviewWe are seeking an experienced Senior Big Data Engineer and Scientist to join our team at KEWMANN PTE. LTD.About the RoleThis is a senior-level position that requires strong technical expertise in big data technologies, data engineering, and data science. The ideal candidate will have a solid understanding of ETL technologies, database SQL, and...


  • Singapore KRIS INFOTECH PTE. LTD. Full time

    At KRIS INFOTECH PTE. LTD., we are seeking an experienced Scala developer to join our team.Job SummaryWe are looking for a highly skilled Scala developer with expertise in big data processing using Spark and Hadoop.About the RoleThis is a challenging opportunity to work on large-scale data processing projects using Scala, PySpark, and Hive.We require someone...

  • Big data engineer

    7 days ago


    Singapore SAKSOFT PTE LIMITED Full time

    Roles & ResponsibilitiesExperience: 9+ YearsRole: Big Data EngineerSkills from Primary Skill Cluster Spark, Big Data, Hive, Java, 5+yrs with hands-on on Java is MustDatabase knowledge and should be able to write the queries is Must.Handson on Unix Shell scripting is Must.Knowledge in Spark SQL (Good to have).Understand on Regulatory reporting(Good to...

  • Big Data Developer

    7 days ago


    Singapore KERRY CONSULTING GROUP PTE. LTD. Full time

    Roles & ResponsibilitiesKerry is currently hiring a Big Data Developer / Data Modeller that is proficient with Cloudera Distributed Hadoop (CDH), Big Data, Spark, and Informatica. This role will involve strong experience in testing life cycles as well as excellent verbal and written communications skills to add value to the client-facing experiences.If you...


  • Singapore MC CONSULTING PTE. LTD. Full time

    Estimated Salary: $120,000 - $180,000 per yearAbout MC CONSULTING PTE. LTD.MC Consulting Pte. Ltd. is a leading consulting firm that specializes in providing expert solutions to clients across various industries. Our team of experienced professionals has a proven track record of delivering high-quality results and exceeding client expectations.Job...

  • Big Data Architect

    13 hours ago


    Singapore SAKSOFT PTE LIMITED Full time

    Company OverviewSaksoft PTE LIMITED is a leading technology company that specializes in big data solutions. Our team of experts is dedicated to delivering innovative and scalable data engineering solutions to our clients.SalaryThe estimated salary for this role is $120,000 - $180,000 per annum, depending on experience and location.Job DescriptionWe are...

  • Software Developer

    2 days ago


    Singapore Capgemini Full time

    Role Overview:We are looking for a Software Developer with experience in Hadoop, Spark, and cloud platforms. You will be responsible for building and optimizing data pipelines, working with relational and NoSQL databases, and deploying data applications to cloud environments. The role requires a passion for data engineering and an interest in developing both...


  • Singapore THEE BIRAPANJAA SYSTEMS (PTE. LTD.) Full time

    Thee Birapannjaa Systems Pte. Ltd. - Senior Data Engineer RoleWe are seeking a highly skilled Senior Data Engineer with 10+ years of experience in software development and expertise in Big Data technologies to join our team at Thee Birapannjaa Systems Pte. Ltd.About the Role:We are looking for a seasoned data engineer who can design, develop, and maintain...


  • Singapore Unison Consulting Pte Ltd Full time

    About UsUnison Consulting Pte Ltd is a leading consulting firm that specializes in data engineering solutions.We are seeking a highly skilled Big Data Engineering Specialist to join our team.Job DescriptionJob Title: Big Data Engineering SpecialistEstimated Salary: $120,000 - $180,000 per annumAs a Big Data Engineering Specialist at Unison Consulting Pte...


  • Singapore D L RESOURCES PTE LTD Full time

    About the RoleWe are seeking a highly skilled Big Data Engineer with expertise in Anti-Money Laundering (AML) to join our team at D L RESOURCES PTE LTD. This is an exciting opportunity to work on complex projects and drive business growth.As a key member of our team, you will be responsible for designing, implementing, and maintaining large-scale data...


  • Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time

    Job SummaryWe are seeking an experienced Big Data Solutions Architect to join our team at Neptunez Singapore PTE. LTD.Key Responsibilities:Collaborate with business stakeholders to understand their needs and translate them into scalable Big Data solutions using Hadoop-based technologies.Design, develop, and implement efficient data processing workflows...


  • Singapore KEWMANN PTE. LTD. Full time

    Job SummaryA talented Data Scientist/Engineer is required by KEWMANN PTE. LTD. to spearhead big data initiatives and drive business growth through data-driven insights.About the RoleWe are seeking an experienced professional with a strong background in machine learning, big data technologies, and data engineering. The ideal candidate will have a solid...

  • Data Architect

    3 days ago


    Singapore CAPGEMINI SINGAPORE PTE. LTD. Full time

    Are you passionate about crafting innovative data solutions that drive impactful business decisions? We are seeking a highly skilled Data Architect to join our team in Singapore. In this role, you will play a key part in building cutting-edge platforms that process and analyze vast datasets with efficiency and scalability.This is an exciting opportunity to...


  • Singapore ATOS INFORMATION TECHNOLOGY (SINGAPORE) PTE. LTD. Full time

    Are you a seasoned Hadoop expert looking for a challenging role in Big Data Engineering? Do you have a strong background in Hadoop Hortonworks administration and support, as well as expertise in Cloudera Data Platform admin?We are seeking a talented Senior Hadoop Engineer to join our team at ATOS INFORMATION TECHNOLOGY (SINGAPORE) PTE. LTD. As a key member...


  • Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time

    About the Role">At NEPTUNEZ SINGAPORE PTE. LTD., we are seeking an exceptional Senior Big Data Architect to join our team. This is a high-level technical role that requires expertise in designing and implementing large-scale data processing solutions using the Hadoop ecosystem.">Key Responsibilities">">Collaborate with business stakeholders to understand...