Senior AWS Data Engineer

Overview

Join to apply for the Senior AWS Data Engineer role at PALO IT .

PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We are World Economic Forum New Champion and a B Corp-certified company.

• We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities)
• We are robust and resilient (100% independent and 0 debt)
• We are entrepreneurs and passionate experts: We invest in what we believe genuinely and work as a collective intelligence
• We are positive, courageous, caring, doers and committed to excellence

Your Role

The Senior Data Engineer will work on acquiring, storing, governing and processing huge sets of structured and unstructured data. You will bring a vision of the big data solutions landscape for choosing optimal architecture components and a set of data engineering skills for implementing enterprise data foundations such as data lakes. You will collaborate with other PALO IT experts in Data Intelligence, Research, UX Design, Digital Technology and Agile teams to enable intelligence in the digital mesh of our clients and deliver big impact.

Responsibilities

• The project is executing a data pipeline, handling data download, decryption, quality checking, cleaning, processing, and prediction, including comprehensive error handling, error reporting, and self-recovery mechanisms. The pipeline is currently deployed on an on-premises server; the project team will monitor operations and performance and align on enhancements using industry-standard tools.
• Experience with migrating data pipelines to AWS Cloud using AWS managed components
• Proficient in Python, SQL, and Shell scripts (Linux)
• Proficient in Airflow or similar tooling to create pipelines
• Proficient in AWS services such as S3, Glue, EMR, Lambda, Step Functions, Fargate, and Batch
• Familiar with tools like AWS Batch, Athena, Parquet formats; NoSQL knowledge and tools like AWS OpenSearch

Who You Are / Qualifications

• Bachelor’s, Master’s, or Ph.D. in IT, Information Management, Computer Science with at least 7 years of experience
• Knowledge of big data technologies and distributed storage/computing concepts
• Experience with big data frameworks (e.g., Hadoop, Spark) and distributions (Cloudera, Hortonworks, MapR)
• Experience with batch ETL jobs to ingest and process data from multiple sources
• Experience with NoSQL databases (e.g., Cassandra, MongoDB, Neo4J, ElasticSearch)
• Experience with querying tools (e.g., Hive, Spark SQL, Impala)
• Experience with Power BI
• Experience or willingness to explore real-time streaming using Kafka, AWS Kinesis, Flume, and/or Spark Streaming
• Interest in DevOps and DataOps principles (Infrastructure as Code, automating data pipelines)
• Understanding of Data Science concepts and methodologies
• Passionate about technology with a drive for continuous learning

We will be prioritizing applicants who have current right to work in Singapore and do not require visa sponsorship.

More About Palo IT

We’re eager to adapt to change, learn from our experiences and move to meet our planet’s urgent needs. Our action areas include: climate net-zero, target revenue from impact projects, workforce training on impact, achieving B Corp certification, and measuring employee happiness.

What We Offer

• Stimulating working environments
• Unique career path
• International mobility
• Internal R&D projects
• Knowledge sharing
• Personalized training

For more on our team culture and benefits, check out our careers page.

PALO IT Singapore is an equal opportunity employer. Employment decisions are based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.

Your personal data will be processed for recruitment purposes, including interviews, tests, evaluation and related activities. Please consult our Privacy Notice to learn how we collect, use and transfer personal data.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Industries

• IT Services and IT Consulting

Referrals increase your chances of interviewing at PALO IT. Get notified about new Senior Data Engineer jobs in Singapore.

#J-18808-Ljbffr

---