Big Data Platform Engineer

3 days ago

Singapore COGNIZANT TECHNOLOGY SOLUTIONS ASIA PACIFIC PTE. LTD. Full time
Roles & Responsibilities

Role & Responsibilities

  • Operating Global Data Platform components (VM Servers, Kubernetes, Kafka) and applications (Apache stack, Collibra, Dataiku and similar).
  • Implement automation of infrastructure, security components, and Continuous Integration & Continuous Delivery for optimal execution of data pipelines (ELT/ETL).
  • Develop solutions to build resiliency in data pipelines with platform health checks, monitoring, and alerting mechanisms, quality, timeliness, recency, and accuracy of data delivery are improved
  • Apply DevSecOps & Agile approaches to deliver the holistic and integrated solution in iterative increments.
  • Liaison and collaborate with enterprise security, digital engineering, and cloud operations to gain consensus on architecture solution frameworks.
  • Review system issues, incidents, and alerts to identify root causes and continuously implement features to improve platform performance.
  • Be current on the latest industry developments and technology trends to effectively lead and design new features/capabilities.

Requirements

  • Bachelor's degree in Computer Science, Information Technology, or related field
  • At least 5 years of experience in big data engineering, with a focus on cloud-based infrastructure
  • Strong proficiency in: DevOps tools like Jenkins, GitLab CI/CD, or similar
  • Containerization using Docker and Kubernetes
  • Apache Spark and big data processing
  • Real-time data integration using Apache Kafka
  • Distributed SQL engines like Presto and Trino
  • Data science platforms like Dataiku
  • Cloud-based object storage solutions like Amazon S3
  • Ansible Playbooks to automate infrastructure provisioning, configuration, and deployment
  • Terraform to manage and provision cloud infrastructure resources
  • Experience with programming languages like Python
  • Experience with cloud migration, including assessment, planning, execution, and validation
  • Experience with disaster recovery setup, including backup and restore processes, replication, and failover mechanisms
Tell employers what skills you have
Apache Spark
Kubernetes
Big Data
Pipelines
Agile
Data Integration
Data Engineering
SQL
Python
Apache Kafka
Continuous Integration
Docker
Data Science
Ansible
S3
Disaster Recovery

  • Big Data Platform Engineer

    2 weeks ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    **SINGAPORE, SINGAPORE /** **SOFTWARE ENGINEERING - DATA PLATFORMS /** **FULL-TIME** - Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of...

  • Big Data Platform Engineer

    4 days ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of technology to ensure that the whole big data pipeline runs optimally and smoothly from ingestion...

  • Engineering Manager

    7 days ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    As an Engineering Manager of Big Data Platform, you lead engineering teams to deliver Big Data platforms for running mission-critical workflows. You set the technical direction, architect, lead and manage the development Big Data Platforms. You push the limits of technology to ensure that the whole big data pipeline runs optimally and smoothly from ingestion...

  • Senior Big Data Platform Engineer

    4 days ago

    Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time

    Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of technology to ensure that the whole big data pipeline runs optimally and smoothly from ingestion...

  • Big Data Platform Architect

    7 days ago

    Singapore beBeeBigData Full time $90,000 - $120,000

    Job Title: Big Data Platform ArchitectWe are seeking an experienced professional to design, build, and maintain scalable big data platforms. This is a challenging role that requires strong technical skills and excellent problem-solving abilities.About the RoleThis senior position involves leading the development of big data ecosystems, ensuring high...

  • Senior Big Data Platform Architect

    5 days ago

    Singapore beBeeBigData Full time $120,000 - $180,000

    Job DescriptionAs a Senior Big Data Platform Staff Engineer, you will be responsible for designing, implementing, and maintaining the Big Data infrastructure platform. This includes managing and optimizing large-scale data processing and analytics environments.Key Responsibilities:Design, implement, and manage the Big Data infrastructure platform, including...

  • Big Data Platform Engineer

    4 days ago

    Singapore Cognizant Full time

    Qualifications DevOps tools like Jenkins, GitLab CI/CD, or similar Containerization using Docker and Kubernetes Real-time data integration using Apache Kafka Distributed SQL engines like Presto and Trino Cloud-based object storage solutions like Amazon S3 Experience with cloud migration, including assessment, planning, execution, and validation Experience...

  • Big Data Platform Engineer

    3 days ago

    Singapore Cognizant Technology Solutions Full time $90,000 - $120,000 per year

    The must have skills sets :DevOps tools like Jenkins, GitLab CI/CD, or similarContainerization using Docker and KubernetesReal-time data integration using Apache KafkaDistributed SQL engines like Presto and TrinoCloud-based object storage solutions like Amazon S3Experience with cloud migration, including assessment, planning, execution, and...

  • Big Data Engineer

    1 hour ago

    Singapore ByteDance Full time

    **Big Data Engineer (Libra) - Data Platform** - Singapore Regular - R&D Job ID: A220563 **Responsibilities** About the team Libra is a large-scale online one-stop A/B testing platform developed by Data Platform. Some of its features include: - Provides experimental evaluation services for all product lines within the company, covering solutions for...

  • Big Data Platform Architect

    6 days ago

    Singapore BigTapp Full time

    **Job Title**: Big Data Platform Architect **Job Location**: Singapore **Job Purpose** The Big Data Platform Architect is responsible for the technical design and implementation of big data platform, systems integration, processes and governance to operationalize the big data platform. The Big Data Platform Architect is required to work closely with...