Senior Big Data Engineer

3 weeks ago

Singapore PALO IT SINGAPORE PTE. LTD. Full time
Roles & Responsibilities

WHO WE ARE:

Build. Scale. Sustain.

PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

  • We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities)
  • We are robust and resilient (100% independent and 0 debt)
  • We are entrepreneurs and passionate experts: We invest in what we believe genuinely and work as a collective intelligence
  • We are positive, courageous, caring, doers and committed to excellence

YOUR ROLE:

The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured / unstructured data. You will bring a vision of the big data solutions landscape for choosing optimal architecture components to use for these purposes as well as a set of data engineering skills for implementing enterprise data foundations such as data lake. Finally, you will work closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them.

YOUR RESPONSIBILITIES:

  • Design, implement, and maintain robust and scalable data pipelines to ingest, transform, and process structured and unstructured data from various sources.
  • Build and manage data warehouses and data lakes, implementing efficient data storage and retrieval mechanisms. Design data models that support business requirements and analytics use cases.
  • Leverage cloud platforms (e.g., AWS, Azure, GCP) to design and deploy scalable data infrastructure, optimizing for performance, cost-effectiveness, and reliability.
  • Implement data quality checks, validation processes, and data governance frameworks to ensure data accuracy, integrity, and compliance with security standards.
  • Continuously monitor and optimize data pipelines and infrastructure to improve processing speed, reduce latency, and enhance overall system performance.
  • Work collaboratively with data scientists, analysts, and other stakeholders to understand their data needs, provide technical expertise, and deliver solutions that meet business objectives.
  • Support business users by implementing data presentation layer including data visualisation using tools like Tableau, PowerBI etc.
  • Stay up-to-date with emerging technologies and industry trends in the big data and data engineering space, evaluating and implementing innovative solutions to improve data processing capabilities.
  • Support the definition and optimization of underlying data infrastructure

WHO YOU ARE:

  • You hold a Bachelor's, Master's, or Ph.D. degree in IT, Information Management, and/or Computer Science with at least 7 years of experience.
  • Good knowledge of the big data technology landscape and concepts related to distributed storage/computing
  • Experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR)
  • Experience with batch & ETL jobs to ingest and process data from multiple data sources
  • Experience with NoSQL databases (e.g. Cassandra, MongoDB, Neo4J, ElasticSearch)
  • Experience with querying tools (e.g Hive, Spark SQL, Impala)
  • Experience with Power BI
  • Experience or willingness to go in real-time stream processing, using solutions such as Kafka, AWS Kinesis, Flume, and/or Spark Streaming
  • Experience or willingness to learn about DevOps and DataOps principles (e.g Infrastructure as Code, automating different parts of the data pipeline)
  • Experience in Pre-sales and Consulting environement
  • Technically Strong and still hands on coding
  • High-level understanding of Data Science concepts and methodologies (how models are built, trained, and deployed)
  • You are passionate about technology and continuous learning comes naturally to you

We will be prioritizing applicants who have a current right to work in Singapore, and do not require sponsorship of a visa.

MORE ABOUT PALO IT:

We're eager to adapt to change, learn from our experiences and move to meet our planet's urgent needs. We are continuously taking action to:

  • Become a climate net-zero company
  • Attain 50% of revenue from projects with a positive impact
  • Train 100% of our workforce on impact
  • Achieve B Corp certification among all our offices across the globe
  • Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

What We Offer

  • Stimulating working environments
  • Unique career path
  • International mobility
  • Internal R&D projects
  • Knowledge sharing
  • Personalized training

For more on our team culture and benefits, check out our careers page.

PALO IT Singapore is an equal opportunity employer. Employment decisions will be based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.

Protecting your privacy and the security of your data are longstanding top priorities for Palo-IT.

Your personal data will be processed for the purposes of managing Palo-IT's recruitment related activities, which include setting up and conducting interviews and tests for applicants, evaluating and assessing the results, and as is otherwise needed in the recruitment and hiring processes.

Please consult our Privacy Notice to know more about how we collect, use, and transfer the personal data of our candidates. Here you can find how you can request for access, correction and/or withdrawal of your Personal Data.

Tell employers what skills you have

MongoDB
Azure
Big Data
Pipelines
Hadoop
Data Management
Presales
ETL
Data Quality
Data Governance
Data Engineering
Impala
SQL
Data Architecture
Hive
Microsoft Power BI
Consulting
Power BI
Data Warehousing
Databases

  • Big Data Engineer

    3 weeks ago

    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesAre you passionate about big data and love working with cutting-edge technologies? Join our team as a Big Data Engineer and help us build scalable data solutions What You'll Do: Design, develop, and optimize big data pipelines Work with Apache Spark, Scala, Hadoop, Hive, Kafka, and more Manage and process large datasets efficiently...

  • Senior Big Data Engineer

    4 days ago

    Singapore TANGSPAC CONSULTING PTE LTD Full time

    Roles & ResponsibilitiesJob Description & RequirementsWorking in a large Big data and datawarehouse team of a bank, you will be responsible for carrying out the followingUnderstand functional requirements Carrying out data analysis, data mapping development of data pipeline, back end services performing data management and data transformation Ensuring...

  • Big Data Engineer

    3 weeks ago

    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesAre you passionate about big data and love working with cutting-edge technologies? Join our team as a Big Data Engineer and help us build scalable data solutions! What You’ll Do:✔ Design, develop, and optimize big data pipelines✔ Work with Apache Spark, Scala, Hadoop, Hive, Kafka, and more✔ Manage and process large datasets...

  • Senior Big Data Engineer

    13 hours ago

    Singapore PALO IT SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesWHO WE ARE:Build. Scale. Sustain.PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World...

  • Big Data Engineer

    3 weeks ago

    Singapore INFINITE COMPUTER SOLUTIONS PTE LTD Full time

    Roles & ResponsibilitiesWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 9 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, building, maintaining and supporting big...

  • Big Data Engineer

    4 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor's or master's degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    2 weeks ago

    Singapore ITCAN PTE. LIMITED Full time

    Roles & Responsibilities· Bachelor's or master's degree in computer science, Engineering, or a related field.· Proven experience as a Big Data Engineer or similar role.· Strong proficiency in Apache Spark and Scala programming.· Experience with big data technologies such as Hadoop, Hive, Kafka, etc.· Familiarity with data warehousing concepts and...

  • Big Data Engineer

    3 weeks ago

    Singapore INFINITE COMPUTER SOLUTIONS PTE LTD Full time

    Roles & ResponsibilitiesWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 9 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, building, maintaining and supporting big...

  • Big Data Engineer

    3 weeks ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesMy ClientIt is a global cryptocurrency exchange dedicated to providing secure, efficient, and innovative digital asset trading services. The platform offers a wide range of services, including spot trading, derivatives trading, and wealth management. They are looking for experienced and motivated Mobile app developer to join the...

  • Big Data Engineer

    3 weeks ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesMy ClientIt is a global cryptocurrency exchange dedicated to providing secure, efficient, and innovative digital asset trading services. The platform offers a wide range of services, including spot trading, derivatives trading, and wealth management. They are looking for experienced and motivated Mobile app developer to join the...

  • Big Data Engineer

    4 weeks ago

    Singapore MICHAEL PAGE INTERNATIONAL PTE LTD Full time

    Roles & Responsibilities Organisation is a global leader in the technology and telecoms industry. Fast-paced environment, rapidly growing organisation.About Our ClientThe organisation is a global leader in the technology and telecoms industry. They are well-known for their innovative approach to tackling complex technical challenges and their commitment to...

  • Big Data Engineer

    3 weeks ago

    Singapore MICHAEL PAGE INTERNATIONAL PTE LTD Full time

    Roles & ResponsibilitiesOrganisation is a global leader in the technology and telecoms industry. Fast-paced environment, rapidly growing organisation.About Our ClientThe organisation is a global leader in the technology and telecoms industry. They are well-known for their innovative approach to tackling complex technical challenges and their commitment to...

  • Big Data

    2 weeks ago

    Singapore EVO OUTSOURCING SOLUTIONS PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities:Collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient big data platform solutions. Design and architect robust big data platforms, ensuring high performance, reliability, and scalability. Implement end-to-end data processing workflows, from data...

  • Big Data Engineer

    4 weeks ago

    Singapore MINDMAP TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities· Perform the role of a Data engineer on the Enterprise Data Architecture and Governance (EDAG) Programs, to ensure the business requirements and deliverables are aligned to business priorities· Design develop data pipelines on EDAG platform to support Wholesale bank data model.· Perform assessments and analysis on...

  • Big Data Engineer

    3 weeks ago

    Singapore MINDMAP TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesResponsibilities· Perform the role of a Data engineer on the Enterprise Data Architecture and Governance (EDAG) Programs, to ensure the business requirements and deliverables are aligned to business priorities· Design develop data pipelines on EDAG platform to support Wholesale bank data model.· Perform assessments and analysis on...

  • Senior Big data Engineer

    4 weeks ago

    Singapore PERSOLKELLY SINGAPORE PTE. LTD. Full time

    Roles & ResponsibilitiesAbout the Client:Our client is a reputed IT Services CompanyResponsibilities:Design, construct and maintain large-scale data processing systems that collect data from various structured and unstructured data sources. Responsible to Store data in a data warehouse or data lake repository. Responsible for building, maintaining, and...

  • Cloudera & Big Data Engineer

    2 weeks ago

    Singapore MANPOWER STAFFING SERVICES (SINGAPORE) PTE LTD Full time

    Roles & ResponsibilitiesJob scopes· Design, build, deployment, of ETL/ELT data pipelines within big data ecosystems ( Impala, Hue, Spark, Talend and Cloudera)· Develop technical design, and ETL specification documents· Manage, monitor and fine tune Hadoop cluster architecture (HDFS) jobs for performance, security and resource management.· Create detailed...

  • Big Data Engineer Lead

    3 days ago

    Singapore Tangspac Full time

    Job DescriptionWe are seeking an experienced Senior Data Developer to join our team in developing and maintaining our Big Data application platform. The successful candidate will have a strong background in distributed systems, data lakes, and streaming technologies, as well as experience with Hadoop ecosystems, Kubernetes, and Spark.The ideal candidate will...

  • Big Data Engineer

    2 weeks ago

    Singapore DADACONSULTANTS PTE. LTD. Full time

    Roles & ResponsibilitiesKey Responsibilities: Ensure seamless deployment, configuration, and operation of applications on cloud servers and container platforms. Collaborate with development teams to design scalable and highly available system architectures. Contribute to the development and maintenance of internal platforms and tools. Perform daily...

  • Big Data Engineer with Kubernetes Expertise

    4 days ago

    Singapore Tangspac Full time

    Company Overview:Tangspac is a leading provider of innovative solutions for the financial industry.We are seeking an experienced Big Data Engineer to join our team and contribute to the development of our cutting-edge big data applications.