Big Data Engineer with Expertise in Hadoop and Spark
3 weeks ago
At Unison Consulting Pte Ltd, we are seeking an experienced Big Data Engineer to join our team. The successful candidate will have a strong background in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.
The role involves collaborating with cross-functional teams to build scalable and efficient data processing pipelines and search applications. A comprehensive understanding of Apache Spark architecture, including RDDs, DataFrames, and Spark SQL, is essential for this position.
We require the candidate to have prior experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments. Expertise in the Scala programming language and familiarity with functional programming principles are also necessary.
A solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization techniques is crucial for this role. Additionally, experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration via Kubernetes is highly desirable.
Responsibilities:
- Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
- Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
- Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
- Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
- Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch
We offer a competitive salary of SGD 120,000 per annum, commensurate with experience, plus benefits and opportunities for career growth.
Required Skills and Qualifications:
- More than 5 years of experience as a Data Engineer
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related discipline
- Possession of Quantexa certification as a Data Engineer or Data Architect, with proficiency in the tool
- Demonstrated experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments
- Expertise in the Scala programming language and familiarity with functional programming principles
Benefits:
- Competitive salary of SGD 120,000 per annum
- Opportunities for career growth and professional development
- Comprehensive benefits package
Others:
- We are an equal opportunities employer and welcome applications from diverse candidates
- Only shortlisted candidates will be contacted for further assessment
-
Data Engineer
7 days ago
Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full timeRoles & ResponsibilitiesResponsibilities:•Collaborate with business partners to understand business needs and functional requirements, translating them into Hadoop-based solutions.•Ensure seamless integration and management of large datasets, working with HDFS, Hive, HBase, Apache Kudu, and other Hadoop ecosystem tools to implement efficient data...
-
Data Engineer
2 days ago
Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full time $8,000 - $9,500Responsibilities: Collaborate with business partners to understand business needs and functional requirements, translating them into Hadoop-based solutions. Ensure seamless integration and management of large datasets, working with HDFS, Hive, HBase, Apache Kudu, and other Hadoop ecosystem tools to implement efficient data processing workflows. ...
-
Singapore D L RESOURCES PTE LTD Full timeJob Title: Big Data Engineer with Advanced Analytics ExpertiseAbout Us: D L RESOURCES PTE LTD is a leading provider of innovative data solutions. We are seeking an experienced Big Data Engineer to join our team.Role Summary: As a Big Data Engineer with Advanced Analytics Expertise, you will be responsible for designing, developing, and implementing...
-
Data Scientist
3 weeks ago
Singapore Unison Consulting Pte Ltd Full timeAbout This RoleWe are seeking a highly skilled Data Scientist (Hadoop, Spark, Scala) to join our team at Unison Consulting Pte Ltd. The ideal candidate will have extensive experience with big data processing, Hadoop, Spark, and Scala.Key ResponsibilitiesDesign, develop, and maintain scalable data pipelines and ETL processes using Hadoop and Spark.Write...
-
Big Data Solutions Engineer
12 hours ago
Singapore ONE NORTH CONSULTING PTE. LTD. Full timeAt One North Consulting PTE. LTD., we are seeking an experienced Senior Data Engineering Specialist to lead our big data solutions team. With a strong background in data engineering and modelling, you will design and implement scalable data pipelines, ensuring seamless data integration across multiple platforms.The ideal candidate will have at least 8+ years...
-
Senior Big Data Engineer and Scientist
7 days ago
Singapore KEWMANN PTE. LTD. Full timeJob OverviewWe are seeking an experienced Senior Big Data Engineer and Scientist to join our team at KEWMANN PTE. LTD.About the RoleThis is a senior-level position that requires strong technical expertise in big data technologies, data engineering, and data science. The ideal candidate will have a solid understanding of ETL technologies, database SQL, and...
-
Scala Developer with Big Data Expertise
4 weeks ago
Singapore KRIS INFOTECH PTE. LTD. Full timeAt KRIS INFOTECH PTE. LTD., we are seeking an experienced Scala developer to join our team.Job SummaryWe are looking for a highly skilled Scala developer with expertise in big data processing using Spark and Hadoop.About the RoleThis is a challenging opportunity to work on large-scale data processing projects using Scala, PySpark, and Hive.We require someone...
-
Big data engineer
7 days ago
Singapore SAKSOFT PTE LIMITED Full timeRoles & ResponsibilitiesExperience: 9+ YearsRole: Big Data EngineerSkills from Primary Skill Cluster Spark, Big Data, Hive, Java, 5+yrs with hands-on on Java is MustDatabase knowledge and should be able to write the queries is Must.Handson on Unix Shell scripting is Must.Knowledge in Spark SQL (Good to have).Understand on Regulatory reporting(Good to...
-
Big Data Developer
7 days ago
Singapore KERRY CONSULTING GROUP PTE. LTD. Full timeRoles & ResponsibilitiesKerry is currently hiring a Big Data Developer / Data Modeller that is proficient with Cloudera Distributed Hadoop (CDH), Big Data, Spark, and Informatica. This role will involve strong experience in testing life cycles as well as excellent verbal and written communications skills to add value to the client-facing experiences.If you...
-
Singapore MC CONSULTING PTE. LTD. Full timeEstimated Salary: $120,000 - $180,000 per yearAbout MC CONSULTING PTE. LTD.MC Consulting Pte. Ltd. is a leading consulting firm that specializes in providing expert solutions to clients across various industries. Our team of experienced professionals has a proven track record of delivering high-quality results and exceeding client expectations.Job...
-
Big Data Architect
13 hours ago
Singapore SAKSOFT PTE LIMITED Full timeCompany OverviewSaksoft PTE LIMITED is a leading technology company that specializes in big data solutions. Our team of experts is dedicated to delivering innovative and scalable data engineering solutions to our clients.SalaryThe estimated salary for this role is $120,000 - $180,000 per annum, depending on experience and location.Job DescriptionWe are...
-
Software Developer
2 days ago
Singapore Capgemini Full timeRole Overview:We are looking for a Software Developer with experience in Hadoop, Spark, and cloud platforms. You will be responsible for building and optimizing data pipelines, working with relational and NoSQL databases, and deploying data applications to cloud environments. The role requires a passion for data engineering and an interest in developing both...
-
Big Data Engineering Leader
13 hours ago
Singapore THEE BIRAPANJAA SYSTEMS (PTE. LTD.) Full timeThee Birapannjaa Systems Pte. Ltd. - Senior Data Engineer RoleWe are seeking a highly skilled Senior Data Engineer with 10+ years of experience in software development and expertise in Big Data technologies to join our team at Thee Birapannjaa Systems Pte. Ltd.About the Role:We are looking for a seasoned data engineer who can design, develop, and maintain...
-
Big Data Engineering Specialist
3 weeks ago
Singapore Unison Consulting Pte Ltd Full timeAbout UsUnison Consulting Pte Ltd is a leading consulting firm that specializes in data engineering solutions.We are seeking a highly skilled Big Data Engineering Specialist to join our team.Job DescriptionJob Title: Big Data Engineering SpecialistEstimated Salary: $120,000 - $180,000 per annumAs a Big Data Engineering Specialist at Unison Consulting Pte...
-
Big Data Engineer with AML Expertise
3 days ago
Singapore D L RESOURCES PTE LTD Full timeAbout the RoleWe are seeking a highly skilled Big Data Engineer with expertise in Anti-Money Laundering (AML) to join our team at D L RESOURCES PTE LTD. This is an exciting opportunity to work on complex projects and drive business growth.As a key member of our team, you will be responsible for designing, implementing, and maintaining large-scale data...
-
Big Data Solutions Architect
13 hours ago
Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full timeJob SummaryWe are seeking an experienced Big Data Solutions Architect to join our team at Neptunez Singapore PTE. LTD.Key Responsibilities:Collaborate with business stakeholders to understand their needs and translate them into scalable Big Data solutions using Hadoop-based technologies.Design, develop, and implement efficient data processing workflows...
-
Singapore KEWMANN PTE. LTD. Full timeJob SummaryA talented Data Scientist/Engineer is required by KEWMANN PTE. LTD. to spearhead big data initiatives and drive business growth through data-driven insights.About the RoleWe are seeking an experienced professional with a strong background in machine learning, big data technologies, and data engineering. The ideal candidate will have a solid...
-
Data Architect
3 days ago
Singapore CAPGEMINI SINGAPORE PTE. LTD. Full timeAre you passionate about crafting innovative data solutions that drive impactful business decisions? We are seeking a highly skilled Data Architect to join our team in Singapore. In this role, you will play a key part in building cutting-edge platforms that process and analyze vast datasets with efficiency and scalability.This is an exciting opportunity to...
-
Senior Hadoop Engineer
7 days ago
Singapore ATOS INFORMATION TECHNOLOGY (SINGAPORE) PTE. LTD. Full timeAre you a seasoned Hadoop expert looking for a challenging role in Big Data Engineering? Do you have a strong background in Hadoop Hortonworks administration and support, as well as expertise in Cloudera Data Platform admin?We are seeking a talented Senior Hadoop Engineer to join our team at ATOS INFORMATION TECHNOLOGY (SINGAPORE) PTE. LTD. As a key member...
-
Senior Big Data Architect
7 days ago
Singapore NEPTUNEZ SINGAPORE PTE. LTD. Full timeAbout the Role">At NEPTUNEZ SINGAPORE PTE. LTD., we are seeking an exceptional Senior Big Data Architect to join our team. This is a high-level technical role that requires expertise in designing and implementing large-scale data processing solutions using the Hadoop ecosystem.">Key Responsibilities">">Collaborate with business stakeholders to understand...