Big Data Developer

3 weeks ago


Singapore PRIMESTAFF MANAGEMENT SERVICES PTE LTD Full time
Roles & Responsibilities

Our client is looking for an experienced and highly skilled Senior Big Data Developer to join the dynamic team. The ideal candidate will have over 10 years of extensive working knowledge and hands-on expertise in big data technologies, especially in Spark on Scala or PySpark, and Hive. This role will involve the design, development, and performance tuning of Spark applications while working with cutting-edge big data tools and distributed systems.

As a key player, you will work closely with our data engineering team to provide robust, scalable solutions for data processing and analytics.

Responsibilities:

  • Spark Expertise: Design, develop, and optimise Spark applications using Scala, PySpark, and Hive.
  • Big Data Engineering: Work with big data tools and techniques, leveraging the Hadoop ecosystem for processing large datasets.
  • Programming: Utilize strong programming skills in Java, Scala, or Python to build high-performance applications.
  • Distributed Systems: Develop a deep understanding of distributed systems and ensure system efficiency.
  • Database Management: Work with RDMS, Data Warehouses, and Unix Shell scripting for end-to-end data management and optimization.
  • Automation: Implement workflows using Airflow and Ctrl-M for scheduling and monitoring.
  • Collaboration: Work with cross-functional teams to ensure seamless data pipeline integrations.

Qualifications:

  • 10+ years of experience working with Spark on Scala or PySpark and Hive.
  • Strong expertise in Java, Scala, or Python programming.
  • Proficient in the Hadoop ecosystem (HDFS, YARN, MapReduce, HBase, etc.).
  • Solid understanding of distributed systems and their performance considerations.
  • Familiarity with RDMS, Data Warehouses, and Unix Shell scripting.
  • Experience with workflow management tools like Airflow and Ctrl-M.
  • Strong analytical and problem-solving skills.
  • Experience in performance tuning for Spark applications.
  • Familiarity with cloud-based data platforms.
  • Experience in a fast-paced, agile development environment.

EA REG:R1108137

EA LIC:95C5411


Tell employers what skills you have

PySpark
Airflow
Scala
Big Data
Hadoop
Workflow Management
Data Engineering
Spark
Distributed Systems
Python
Performance Tuning
Unix Shell Scripting
Java
Python Programming

  • Singapore NTT Data Singapore Full time

    Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...


  • Singapore NTT Data Singapore Full time

    Job Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...


  • Singapore Helius Full time

    Talend Big Data Developer RoleAt Helius, we are seeking a skilled Big Data Developer to join our team. The ideal candidate will have experience with Talend Big Data version 7+ and Hadoop, with a strong understanding of data models and data flows.Key Responsibilities:Design and develop production-grade Talend Big Data jobs with consideration for IT...


  • Singapore TikTok Full time

    About the teamByteDance/TikTok's Tea analytics platform is a leading in-house data platform used by over 10,000 employees globally. As a key member of our global engineering team, you will be responsible for building and maintaining the platform that supports exabyte-level data, trillion-level events, and millisecond-level response...


  • Singapore Nityo Infotech Full time

    As a key member of our data engineering team at Nityo Infotech, you will play a crucial role in designing, building, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.The ideal candidate will have a solid understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL, as well as experience integrating and working...

  • Big Data Engineer

    7 days ago


    Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 6 - 8 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...


  • Singapore KRIS INFOTECH PTE. LTD. Full time

    At KRIS INFOTECH PTE. LTD., we are seeking an experienced Scala developer to join our team.Job SummaryWe are looking for a highly skilled Scala developer with expertise in big data processing using Spark and Hadoop.About the RoleThis is a challenging opportunity to work on large-scale data processing projects using Scala, PySpark, and Hive.We require someone...


  • Singapore Citi Full time

    Citi is seeking a highly experienced Senior Big Data Solutions Lead to join our Applications Development team. As a key member of our technology organization, you will play a critical role in driving the development and implementation of innovative big data solutions.About the RoleThis position is responsible for leading the design, development, and...


  • Singapore ITCAN Full time

    About ITCANWe are a leading technology company that delivers innovative solutions to our clients.Job Title: Senior Big Data EngineerSalary Range:$170,000 - $220,000 per yearAbout the Role:We are seeking an experienced Senior Big Data Engineer to join our team. The ideal candidate will have extensive experience in designing and implementing large-scale big...


  • Singapore TANGSPAC CONSULTING PTE LTD Full time

    Roles & ResponsibilitiesAs a Data Framework Engineer at TANGSPAC CONSULTING PTE LTD, you will be responsible for creating frameworks and technical features that enable faster operationalisation of data models, analytical models, and user-generated contents. This includes partnering with data scientists to enable faster adoption of AL/ML model-based systems,...


  • Singapore Virtusa Full time

    We are seeking an experienced Big Data Engineer to join our team at Virtusa in Moulmein-Kallang, Central Singapore. The successful candidate will have a strong background in big data engineering and experience working with Python, PySpark, Linux, and ETL tools like Informatica.ResponsibilitiesDesign and implement scalable big data solutions using Hive,...


  • Singapore Unison Consulting Pte Ltd Full time

    About UsAt Unison Consulting Pte Ltd, we are a forward-thinking consulting firm that provides expert solutions to businesses in the digital age.Job Title: Senior Big Data ArchitectJob DescriptionWe are seeking an experienced Senior Big Data Architect to join our team. The ideal candidate will have a strong background in big data engineering and architecture,...


  • Singapore Helius Full time

    Job DescriptionAbout the RoleWe are seeking a highly skilled Big Data Integration Specialist to join our team at Helius. As a key member of our data integration team, you will be responsible for designing and implementing production-grade Talend Big Data jobs that meet our IT organization's architecture standards.Key ResponsibilitiesDevelop and maintain...


  • Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full time

    Senior ETL Developer PositionWe are seeking a highly skilled Senior ETL Developer to join our team at ELLIOTT MOSS CONSULTING PTE. LTD. The ideal candidate will possess strong proficiency in ETL solutions such as Informatica BDM, experience with big data technologies like Cloudera, and advanced knowledge of programming languages such as Python, Scala, and...


  • Singapore Unison Consulting Pte Ltd Full time

    About UsUnison Consulting Pte Ltd is a leading consulting firm that specializes in data engineering solutions.We are seeking a highly skilled Big Data Engineering Specialist to join our team.Job DescriptionJob Title: Big Data Engineering SpecialistEstimated Salary: $120,000 - $180,000 per annumAs a Big Data Engineering Specialist at Unison Consulting Pte...


  • Singapore Tencent Singapore Full time

    Responsibilities: Cloud & Smart Industries Group (CSIG) is responsible for promoting the company's cloud and industry Internet strategy. CSIG explores the interactions between users and industries to create innovative solutions for smart industries via technological advancements such as cloud, AI, and network security. While driving the digitalization of...


  • Singapore WEB SYNERGIES (S) PTE LTD Full time

    About WEB SYNERGIES (S) PTE LTDWe are a dynamic company seeking an experienced Big Data Architect to join our team.Job Summary:As a seasoned professional, you will be responsible for designing and implementing large-scale data architectures using Big Data technologies such as Hadoop and Spark. You will work closely with cross-functional teams to gather,...


  • Singapore Unison Consulting Pte Ltd Full time

    Unison Consulting Pte Ltd is seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have 5+ years of experience in designing, developing, and implementing Spark Scala applications and data processing pipelines.About the RoleThis is a key role that requires expertise in integrating Elasticsearch with Spark to enable efficient...


  • Singapore Tencent Singapore Full time

    Job Responsibilities:As a Senior Big Data Solution Architect at Tencent Singapore, you will be responsible for designing and implementing Big Data solutions tailored to customer business scenarios and technical requirements. You will need to deeply understand customers' needs and pain points regarding Big Data products and solutions.Recommend the core...


  • Singapore U3 Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team at U3 Infotech. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines and data services modules with robustness in mind to support the growing demands from business users.Key ResponsibilitiesDesign and implement...