Cloud Data Engineer

2 weeks ago

Singapore Cybervault Full time

Responsibilities Design and architect data storage solutions, including databases, data lakes, and warehouses, using AWS services such as Amazon S3, Amazon RDS, Amazon Redshift, and Amazon DynamoDB, along with Databricks' Delta Lake. Integrate Informatica IDMC for metadata management and data cataloging. Create, manage, and optimize data pipelines for ingesting, processing, and transforming data using AWS services like AWS Glue, AWS Data Pipeline, and AWS Lambda, Databricks for advanced data processing, and Informatica IDMC for data integration and quality. Integrate data from various sources, both internal and external, into AWS and Databricks environments, ensuring data consistency and quality, while leveraging Informatica IDMC for data integration, transformation, and governance. Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and enrich data, making it suitable for analytical purposes using Databricks' Spark capabilities and Informatica IDMC for data transformation and quality. Monitor and optimize data processing and query performance in both AWS and Databricks environments, making necessary adjustments to meet performance and scalability requirements. Utilize Informatica IDMC for optimizing data workflows. Implement security best practices and data encryption methods to protect sensitive data in both AWS and Databricks, while ensuring 1 | P a g e compliance with data privacy regulations. Employ Informatica IDMC for data governance and compliance. Implement automation for routine tasks, such as data ingestion, transformation, and monitoring, using AWS services like AWS Step Functions, AWS Lambda, Databricks Jobs, and Informatica IDMC for workflow automation. Maintain clear and comprehensive documentation of data infrastructure, pipelines, and configurations in both AWS and Databricks environments, with metadata management facilitated by Informatica IDMC. Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver appropriate solutions across AWS, Databricks, and Informatica IDMC. Identify and resolve data-related issues and provide support to ensure data availability and integrity in both AWS, Databricks, and Informatica IDMC environments. Optimize AWS, Databricks, and Informatica resource usage to control costs while meeting performance and scalability requirements. Stay up-to-date with AWS, Databricks, Informatica IDMC services, and data engineering best practices to recommend and implement new technologies and techniques. Qualifications Bachelor’s or master’s degree in computer science, data engineering, or a related field. Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC. Proficiency in programming languages such as Python, Java, or Scala for building data pipelines. Evaluate potential technical solutions and make recommendations to resolve data issues especially on performance assessment for complex data transformations and long running data processes. Strong knowledge of SQL and NoSQL databases. Familiarity with data modeling and schema design. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. AWS certifications (e.g., AWS Certified Data Analytics - Specialty), Databricks certifications, and Informatica certifications are a plus. Good to have skills: Informatica Cloud, Databricks and AWS. #J-18808-Ljbffr

  • Cloud System Engineer

    2 weeks ago

    Singapore ST ENGINEERING CLOUD AND DATA CENTRE SOLUTIONS PTE. LTD. Full time

    About ST Engineering ST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace,...

  • Cloud Engineer

    3 days ago

    Singapore Metaverse Cloud Solution Pte. Limited Full time

    Join hands to create a bright future for the informatization development in the field of engineering construction. Responsibilities: Responsible for Windows and Linux based IT infrastructure maintenance and implementation Provide server and network support and troubleshooting Provide technical support and system administration on cloud platform to meet...

  • Data Engineer

    1 week ago

    Singapore CLOUD KINETICS CONSULTING PTE. LTD. Full time

    Job Title: Data EngineerJob Overview: We are seeking an experienced Data Engineer to join our Data team in Singapore. The ideal candidate will have a proven track record of hands-on data engineering experience, particularly within the AWS and Azure. As a Data Engineer, you will be responsible for developing and maintaining data pipelines, ensuring the...

  • Data Engineer

    1 week ago

    Singapore CLOUD KINETICS CONSULTING PTE. LTD. Full time

    Overview We are seeking an experienced Data Engineer to join our Data team in Singapore. The ideal candidate will have a proven track record of hands-on data engineering experience, particularly within the AWS and Azure. As a Data Engineer, you will be responsible for developing and maintaining data pipelines, ensuring the reliability, efficiency, and...

  • Cloud Infrastructure Lead

    5 days ago

    Singapore ST Engineering Cloud & Data Centre Solutions Pte Ltd Full time

    About ST Engineering ST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace,...

  • Data Center Technical Engineer

    2 weeks ago

    Singapore Alibaba Cloud Full time

    Overview We are looking for an entry-level professional to join Alibaba Cloud's IDC team and explore the large-scale data centers. You will contribute to solution design, research and operational optimization for our Data Center. Responsibilities Responsible for exploring cutting-edge technologies (such as energy, power, HVAC, monitoring, etc.) in Alibaba...

  • Data Engineer

    2 weeks ago

    Singapore CLOUD KINETICS CONSULTING PTE. LTD. Full time $80,000 - $120,000 per year

    Job Title: Data EngineerJob Overview:We are seeking an experienced Data Engineer to join our Data team in Singapore. The ideal candidate will have a proven track record of hands-on data engineering experience, particularly within the AWS and Azure. As a Data Engineer, you will be responsible for developing and maintaining data pipelines, ensuring the...

  • Data Centre Engineer, Field Operations

    1 week ago

    Singapore SMC Cloud Full time

    Overview Data Centre Engineer, Field Operations role at SMC Cloud — SMC Cloud is seeking a skilled Data Centre Engineer to join the Operations team, supporting the daily operations and maintenance of AI-accelerated high-performance computing (HPC) infrastructure. This role will work closely with Field Service Engineers, HPC and Network Engineering teams,...

  • Facility Operation Manager

    2 weeks ago

    Singapore Alibaba Cloud Full time

    Facility Operation Manager (Data Center)Alibaba Cloud Data Center is a global cloud computing infrastructure offering high-performance, available, and secure cloud services across 30 cloud regions and 89 availability zones through advanced technology and global deployment. We are looking for a Facility Operations Manager for our Singapore location. The ideal...

  • Facility Operation Manager

    2 weeks ago

    Singapore Alibaba Cloud Full time

    Facility Operation Manager (Data Center)Alibaba Cloud Data Center is a global cloud computing infrastructure offering high-performance, available, and secure cloud services across 30 cloud regions and 89 availability zones through advanced technology and global deployment. We are looking for a Facility Operations Manager for our Singapore location. The...