Senior Big Data Engineer

3 weeks ago

Singapore PALO IT SINGAPORE PTE. LTD. Full time
Roles & Responsibilities

WHO WE ARE:

Build. Scale. Sustain.

PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

  • We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities)
  • We are robust and resilient (100% independent and 0 debt)
  • We are entrepreneurs and passionate experts: We invest in what we believe genuinely and work as a collective intelligence
  • We are positive, courageous, caring, doers and committed to excellence

YOUR ROLE:

The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured / unstructured data. You will bring a vision of the big data solutions landscape for choosing optimal architecture components to use for these purposes as well as a set of data engineering skills for implementing enterprise data foundations such as data lake. Finally, you will work closely with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact to them.

YOUR RESPONSIBILITIES:

  • Design, implement, and maintain robust and scalable data pipelines to ingest, transform, and process structured and unstructured data from various sources.
  • Build and manage data warehouses and data lakes, implementing efficient data storage and retrieval mechanisms. Design data models that support business requirements and analytics use cases.
  • Leverage cloud platforms (e.g., AWS, Azure, GCP) to design and deploy scalable data infrastructure, optimizing for performance, cost-effectiveness, and reliability.
  • Implement data quality checks, validation processes, and data governance frameworks to ensure data accuracy, integrity, and compliance with security standards.
  • Continuously monitor and optimize data pipelines and infrastructure to improve processing speed, reduce latency, and enhance overall system performance.
  • Work collaboratively with data scientists, analysts, and other stakeholders to understand their data needs, provide technical expertise, and deliver solutions that meet business objectives.
  • Support business users by implementing data presentation layer including data visualisation using tools like Tableau, PowerBI etc.
  • Stay up-to-date with emerging technologies and industry trends in the big data and data engineering space, evaluating and implementing innovative solutions to improve data processing capabilities.
  • Support the definition and optimization of underlying data infrastructure

WHO YOU ARE:

  • You hold a Bachelor's, Master's, or Ph.D. degree in IT, Information Management, and/or Computer Science with at least 6 years of experience.
  • Good knowledge of the big data technology landscape and concepts related to distributed storage/computing
  • Experience with big data frameworks (e.g. Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR)
  • Experience with batch & ETL jobs to ingest and process data from multiple data sources
  • Experience with NoSQL databases (e.g. Cassandra, MongoDB, Neo4J, ElasticSearch)
  • Experience with querying tools (e.g Hive, Spark SQL, Impala)
  • Experience with Power BI
  • Experience or willingness to go in real-time stream processing, using solutions such as Kafka, AWS Kinesis, Flume, and/or Spark Streaming
  • Experience or willingness to learn about DevOps and DataOps principles (e.g Infrastructure as Code, automating different parts of the data pipeline)
  • High-level understanding of Data Science concepts and methodologies (how models are built, trained, and deployed)
  • You are passionate about technology and continuous learning comes naturally to you

We will be prioritizing applicants who have a current right to work in Singapore, and do not require sponsorship of a visa.

MORE ABOUT PALO IT:

We're eager to adapt to change, learn from our experiences and move to meet our planet's urgent needs. We are continuously taking action to:

  • Become a climate net-zero company
  • Attain 50% of revenue from projects with a positive impact
  • Train 100% of our workforce on impact
  • Achieve B Corp certification among all our offices across the globe
  • Continuously measure & improve employee happiness

Our clients are amongst the world's most successful companies. We innovate with both established Fortune 1000s, SMEs and start-ups who aim to make an impact, become global leaders and address the world's most complex challenges.

What We Offer

  • Stimulating working environments
  • Unique career path
  • International mobility
  • Internal R&D projects
  • Knowledge sharing
  • Personalized training

For more on our team culture and benefits, check out our careers page.

PALO IT Singapore is an equal opportunity employer. Employment decisions will be based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.

Protecting your privacy and the security of your data are longstanding top priorities for Palo-IT.

Your personal data will be processed for the purposes of managing Palo-IT's recruitment related activities, which include setting up and conducting interviews and tests for applicants, evaluating and assessing the results, and as is otherwise needed in the recruitment and hiring processes.

Please consult our Privacy Notice to know more about how we collect, use, and transfer the personal data of our candidates. Here you can find how you can request for access, correction and/or withdrawal of your Personal Data.

Tell employers what skills you have

MongoDB
Azure
Big Data
Pipelines
Hadoop
Data Management
UX Design
ETL
Data Quality
Cassandra
Data Engineering
Impala
SQL
Data Architecture
Hive
Data Science
Microsoft Power BI
Power BI
Data Warehousing
Databases

  • Senior Big Data Engineer

    1 week ago

    Singapore Manpower Singapore Full time

    Job Overview:We are seeking a skilled Senior Big Data Engineer to join our team at Manpower Singapore. In this role, you will be responsible for designing, developing, and maintaining large-scale data platforms.Key Responsibilities:Perform operation, installation, and monitoring of big data platformsEvaluate infrastructure requirements for deployments and...

  • Senior Big Data Professional

    1 week ago

    Singapore Tata Consultancy Services Full time

    Tata Consultancy Services is hiring a Senior Big Data Professional to drive innovation in our big data space. As a senior member of our team, you will be responsible for developing and implementing large-scale big data solutions.Key Responsibilities:Lead the design and development of big data pipelines and architectures using Hadoop, Spark, Java, Scala, and...

  • Data Engineer

    22 hours ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$5,000 - S$9,000 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** Snowflake yarn Oracle Shell Script CI Shell UNIX Presto Java MSSQL Airflow Spark Linux Python Astek is proposing an opportunity for **Data Engineer** (Big Data) to support our project based in**...

  • Senior Site Reliability Engineer

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$8,000 - S$13,000 / Monthly **Job Type** **Seniority** Senior **Years of Experience** At least 5 years **Tech Stacks** HBase Spark Linux kafka Flink Hadoop **Job** **Description**: - Design and develop software solutions to automate the technical operations of large-scale systems - Work closely with cross-functional teams...

  • Senior Big Data Engineer

    2 weeks ago

    Singapore Binance Full time

    Binance is a leading global blockchain ecosystem behind the world’s largest cryptocurrency exchange by trading volume and registered users. We are trusted by over 230 million people in 100+ countries for our industry-leading security, user fund transparency, trading engine speed, deep liquidity, and an unmatched portfolio of digital-asset products. Binance...

  • Senior Big Data Architect

    1 week ago

    Singapore Nityo Infotech Full time

    Nityo Infotech is looking for a highly skilled Senior Big Data Architect to lead the design and implementation of our big data infrastructure. As a senior member of our architecture team, you will work closely with cross-functional teams to ensure that our big data solutions meet the needs of our business.The successful candidate will have extensive...

  • Senior Big Data Engineer

    4 weeks ago

    Singapore TANGSPAC CONSULTING PTE LTD Full time

    Roles & ResponsibilitiesJob Description & RequirementsWorking in a large Big data and datawarehouse team of a bank, you will be responsible for carrying out the followingUnderstand functional requirements Carrying out data analysis, data mapping development of data pipeline, back end services performing data management and data transformation Ensuring...

  • Big Data Engineer Lead

    1 week ago

    Singapore Trulyyy Full time

    We're seeking an experienced Senior Data Engineer to join our team at Trulyyy. In this role, you'll play a critical part in shaping our data strategy and driving business outcomes.About the RoleThe successful candidate will have extensive experience in designing, developing, and maintaining scalable data pipelines for processing and analyzing data from...

  • Data Engineer

    22 hours ago

    Singapore TALENTSIS PTE. LTD. Full time

    We are looking for a skilled **Data Engineer**with a focus on **Big Data**to design, develop, and manage large-scale data processing systems. If you're passionate about working with big data technologies and driving impactful solutions, we want to hear from you. **Key Responsibilities**: - Build and maintain big data pipelines using technologies like...

  • Big Data Engineer

    2 days ago

    Singapore SAKSOFT PTE LIMITED Full time

    Experience: 9+ Years Role: Big Data Engineer Skills from Primary Skill Cluster Spark, Big Data, Hive, Java, 5+yrs with hands-on on Java is Must Database knowledge and should be able to write the queries is Must. Handson on Unix Shell scripting is Must. Knowledge in Spark SQL (Good to have). Understand on Regulatory reporting(Good to have). Design...

  • Data Engineer, Big Data

    6 days ago

    Singapore Grab Full time

    **Job Description**: **Life at Grab**: At Grab, every Grabber is guided by The Grab Way, which spells out our mission, how we believe we can achieve it, and our operating principles - the 4Hs: Heart, Hunger, Honour and Humility. These principles guide and help us make decisions as we work to create economic empowerment for the people of Southeast...

  • Big Data Engineer

    1 week ago

    Singapore This is an IT support group Full time

    Responsibilities The Big Data Engineer is responsible for analyzing big data and optimizing the performance of our online advertising delivery system. They are also responsible for the research and development of the advertising system's offline and real-time data streams. Requirements: Bachelor's degree or above, majoring in Computer Science, Statistics,...

  • Big Data Engineer

    1 week ago

    Singapore This is an IT support group Full time

    We are seeking a highly motivated Big Data Engineer to join our team in Singapore and contribute to the development of our cutting-edge data platforms.The ideal candidate will have a solid foundation in big data technologies, including Hadoop, Spark, and NoSQL databases, as well as expertise in cloud computing platforms such as AWS or Azure.This is an...

  • Big Data Engineer Manager

    1 week ago

    Singapore Assurity Trusted Solutions Full time

    As a Big Data Engineer Manager, you will be responsible for leading a team of engineers to design, build, and maintain large-scale big data systems. You will have experience in working with cloud platforms (i.e., AWS, Azure, Google Cloud) and familiarity with Docker containers.Our company offers a range of benefits, including a competitive salary,...

  • Big Data Platform Engineer

    3 days ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    **SINGAPORE, SINGAPORE /** **SOFTWARE ENGINEERING - DATA PLATFORMS /** **FULL-TIME** - Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of...

  • Big Data Engineering Lead

    1 week ago

    Singapore Adecco Personnel Pte Ltd. Full time

    About the RoleWe are looking for a Senior Data Engineer to join our team at Adecco Personnel Pte Ltd. As a key member of our data engineering team, you will be responsible for designing, implementing and maintaining scalable data pipelines for structured and unstructured data. You will work closely with our data scientists and analysts to deliver...

  • Senior Site Reliability Engineer

    2 weeks ago

    Singapore Shopee Full time

    DepartmentEngineering and Technology- LevelExperienced (Individual Contributor)- LocationSingaporeThe Engineering and Technology team is at the core of the Shopee platform development. The team is made up of a group of passionate engineers from all over the world, striving to build the best systems with the most suitable technologies. Our engineers do not...

  • Big Data Architect

    1 week ago

    Singapore Helius Technologies Full time

    Job Description:Helius Technologies is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and ETL processes using Databricks expertise.Required Skills and Qualifications:Strong working experience in end-to-end development of data...

  • Big Data Solutions Architect

    1 week ago

    Singapore CHARLES & KEITH GROUP Full time

    About the Role:The Senior/Lead Data Engineer will be responsible for designing and developing scalable data solutions on Alibaba Cloud's big data platforms, including MaxCompute and Hologres. The ideal candidate has deep experience with Alibaba Cloud's big data ecosystem and is proficient in leading technical efforts in complex data warehousing and...

  • Big Data Engineering Manager

    1 week ago

    Singapore FPT ASIA PACIFIC PTE. LTD. Full time

    Big Data Engineering ManagerFPT Asia Pacific PTE. LTD. is seeking a Big Data Engineering Manager to join our team. The successful candidate will be responsible for leading a team of data engineers to design, implement, and maintain large-scale big data systems that support business growth and innovation.Key Responsibilities:Lead a team of data engineers to...