Big Data Platform Engineer

4 weeks ago

Singapore COGNIZANT TECHNOLOGY SOLUTIONS ASIA PACIFIC PTE. LTD. Full time
Roles & Responsibilities

Role & Responsibilities

  • Operating Global Data Platform components (VM Servers, Kubernetes, Kafka) and applications (Apache stack, Collibra, Dataiku and similar).
  • Implement automation of infrastructure, security components, and Continuous Integration & Continuous Delivery for optimal execution of data pipelines (ELT/ETL).
  • Develop solutions to build resiliency in data pipelines with platform health checks, monitoring, and alerting mechanisms, quality, timeliness, recency, and accuracy of data delivery are improved
  • Apply DevSecOps & Agile approaches to deliver the holistic and integrated solution in iterative increments.
  • Liaison and collaborate with enterprise security, digital engineering, and cloud operations to gain consensus on architecture solution frameworks.
  • Review system issues, incidents, and alerts to identify root causes and continuously implement features to improve platform performance.
  • Be current on the latest industry developments and technology trends to effectively lead and design new features/capabilities.

Requirements

  • Bachelor's degree in Computer Science, Information Technology, or related field
  • At least 5 years of experience in big data engineering, with a focus on cloud-based infrastructure
  • Strong proficiency in: DevOps tools like Jenkins, GitLab CI/CD, or similar
  • Containerization using Docker and Kubernetes
  • Apache Spark and big data processing
  • Real-time data integration using Apache Kafka
  • Distributed SQL engines like Presto and Trino
  • Data science platforms like Dataiku
  • Cloud-based object storage solutions like Amazon S3
  • Ansible Playbooks to automate infrastructure provisioning, configuration, and deployment
  • Terraform to manage and provision cloud infrastructure resources
  • Experience with programming languages like Python
  • Experience with cloud migration, including assessment, planning, execution, and validation
  • Experience with disaster recovery setup, including backup and restore processes, replication, and failover mechanisms
Tell employers what skills you have

Apache Spark
Kubernetes
Big Data
Pipelines
Agile
Data Integration
Data Engineering
SQL
Python
Apache Kafka
Continuous Integration
Docker
Data Science
Ansible
S3
Disaster Recovery

  • Big Data Platform Engineer

    3 days ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    **SINGAPORE, SINGAPORE /** **SOFTWARE ENGINEERING - DATA PLATFORMS /** **FULL-TIME** - Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of...

  • Big Data Engineer, Data Platform

    1 week ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$6,500 - S$13,000 / Monthly **Job Type** **Seniority** Junior **Years of Experience** At least 1 year **Tech Stacks** ETL ClickHouse Metastore Analytics R Flume Presto Hive Spark kafka Flink SQL **Responsibilities** TikTok will be prioritizing applicants who have a current right to work in Singapore, and do not require...

  • Big Data Engineer, Data Platform

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$6,500 - S$13,000 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** ETL ClickHouse Metastore Analytics R Flume Presto Hive Spark kafka Flink SQL **Responsibilities** TikTok will be prioritizing applicants who have a current right to work in Singapore, and do not require...

  • Big Data Platform Engineer

    2 weeks ago

    Singapore Cognizant Full time

    **The must have skills sets**: - DevOps tools like Jenkins, GitLab CI/CD, or similar - Containerization using Docker and Kubernetes - Real-time data integration using Apache Kafka - Distributed SQL engines like Presto and Trino - Cloud-based object storage solutions like Amazon S3 - Experience with cloud migration, including assessment, planning, execution,...

  • Senior Big Data Platform SRE Engineer

    2 weeks ago

    Singapore Tencent Full time

    Technology Engineering Group (TEG) is responsible for supporting the company and its business groups on technology and operational platforms, as well as the construction and operation of R&D management and data centers, TEG provides users with a full range of customer services. As the operator of the largest networking, devices, and data center in Asia,TEG...

  • Big Data Global Manager

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$9,187 - S$18,375 / Monthly EST **Job Type** Permanent **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** Strategy Analytics Java SQL Scala Python - What you will be doing: - We are looking for a Big Data Global Manager to join the team. As the Big Data Global Manager, you will provide big data power...

  • Big Data Global Manager

    2 weeks ago

    Singapore TikTok Full time

    Responsibilities About TikTok TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul and Tokyo. Why Join Us Creation is the core of TikTok's purpose. Our platform is built to help...

  • Big Data Development Engineer

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$11,250 - S$22,500 / Monthly **Job Type** **Seniority** Junior **Years of Experience** At least 1 year **Tech Stacks** ClickHouse Analytics Java Spark Scala Hadoop **About Tiktok** TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok has global offices...

  • Big Data Development Engineer

    1 week ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$11,250 - S$22,500 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** ClickHouse Analytics Java Spark Scala Hadoop **Responsibilities** **About TikTok** TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok...

  • Big Data Development Engineer

    2 weeks ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$6,500 - S$13,000 / Monthly **Job Type** **Seniority** Junior **Years of Experience** At least 1 year **Tech Stacks** ClickHouse Analytics Java Spark Scala Hadoop **Responsibilities** **About TikTok** TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok...