Cloud Data Engineer

2 weeks ago

Singapore Cybervault Full time

Responsibilities Design and architect data storage solutions, including databases, data lakes, and warehouses, using AWS services such as Amazon S3, Amazon RDS, Amazon Redshift, and Amazon DynamoDB, along with Databricks' Delta Lake. Integrate Informatica IDMC for metadata management and data cataloging. Create, manage, and optimize data pipelines for ingesting, processing, and transforming data using AWS services like AWS Glue, AWS Data Pipeline, and AWS Lambda, Databricks for advanced data processing, and Informatica IDMC for data integration and quality. Integrate data from various sources, both internal and external, into AWS and Databricks environments, ensuring data consistency and quality, while leveraging Informatica IDMC for data integration, transformation, and governance. Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and enrich data, making it suitable for analytical purposes using Databricks' Spark capabilities and Informatica IDMC for data transformation and quality. Monitor and optimize data processing and query performance in both AWS and Databricks environments, making necessary adjustments to meet performance and scalability requirements. Utilize Informatica IDMC for optimizing data workflows. Implement security best practices and data encryption methods to protect sensitive data in both AWS and Databricks, while ensuring 1 | P a g e compliance with data privacy regulations. Employ Informatica IDMC for data governance and compliance. Implement automation for routine tasks, such as data ingestion, transformation, and monitoring, using AWS services like AWS Step Functions, AWS Lambda, Databricks Jobs, and Informatica IDMC for workflow automation. Maintain clear and comprehensive documentation of data infrastructure, pipelines, and configurations in both AWS and Databricks environments, with metadata management facilitated by Informatica IDMC. Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver appropriate solutions across AWS, Databricks, and Informatica IDMC. Identify and resolve data-related issues and provide support to ensure data availability and integrity in both AWS, Databricks, and Informatica IDMC environments. Optimize AWS, Databricks, and Informatica resource usage to control costs while meeting performance and scalability requirements. Stay up-to-date with AWS, Databricks, Informatica IDMC services, and data engineering best practices to recommend and implement new technologies and techniques. Qualifications Bachelor's or master's degree in computer science, data engineering, or a related field. Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC. Proficiency in programming languages such as Python, Java, or Scala for building data pipelines. Evaluate potential technical solutions and make recommendations to resolve data issues especially on performance assessment for complex data transformations and long running data processes. Strong knowledge of SQL and NoSQL databases. Familiarity with data modeling and schema design. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. AWS certifications (e.g., AWS Certified Data Analytics - Specialty), Databricks certifications, and Informatica certifications are a plus. Good to have skills: Informatica Cloud, Databricks and AWS. #J-18808-Ljbffr

  • Senior/ Manager, Sales

    1 day ago

    Singapore ST ENGINEERING CLOUD AND DATA CENTRE SOLUTIONS PTE. LTD. Full time

    **About ST Engineering** ST Engineering is a global technology, defence, and engineering group with offices across Asia, Europe, the Middle East, and the U.S., serving customers in more than 100 countries. The Group uses technology and innovation to solve real-world problems and improve lives through its diverse portfolio of businesses across the aerospace,...

  • Cloud Engineer

    1 week ago

    Singapore NANO CLOUD PRIVATE LIMITED Full time

    Job Description & Requirements Mandatory skill : 1) Cloud Computing (Azure) Certifications. 2)Azure Solutions Architect Expert is advantageous. Job Responsibilities: To lead and manage, troubleshooting, support across various domain teams. Day1 implementation and existing Day2 operations of Cloud Operations. - At least ten (10) years’ experience in...

  • Data Engineer

    2 weeks ago

    Singapore CLOUD KINETICS CONSULTING PTE. LTD. Full time $80,000 - $120,000 per year

    Job Title: Data EngineerJob Overview:We are seeking an experienced Data Engineer to join our Data team in Singapore. The ideal candidate will have a proven track record of hands-on data engineering experience, particularly within the AWS and Azure. As a Data Engineer, you will be responsible for developing and maintaining data pipelines, ensuring the...

  • Cloud Data Engineer

    6 days ago

    Singapore InfoCepts Full time

    GovTech Singapore has awarded InfoCepts the provision of Data Science software and services for Government Ministries/Departments, statutory boards, organs of the state and participating entities. **About InfoCepts** Presently we are expanding our Singapore business unit and thus seeking out aspiring / proven Data Engineers in cloud environments to join...

  • Facility Operation Manager

    2 weeks ago

    Singapore Alibaba Cloud Full time

    Facility Operation Manager (Data Center)Alibaba Cloud Data Center is a global cloud computing infrastructure offering high-performance, available, and secure cloud services across 30 cloud regions and 89 availability zones through advanced technology and global deployment. We are looking for a Facility Operations Manager for our Singapore location. The ideal...

  • Cloud Engineer

    2 weeks ago

    Singapore CLOUD KINETICS CONSULTING PTE. LTD. Full time

    Responsibilities & Duties - Responsible for day-to-day customer support and administration of cloud services. - Coordinate with vendors where cloud & virtualization specific issues need to be escalated. - Able to operate cloud security technologies / services provided natively in AWS such as Security Hub, Guarduty, Cloudwatch, AWS Config, Control Tower,...

  • Azure Data engineer

    7 days ago

    Singapore NTT Data Singapore Full time $80,000 - $120,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations...

  • Datacenter/cloud Solution Presales Engineer

    2 weeks ago

    Singapore QUANTA CLOUD TECHNOLOGY SINGAPORE PTE. LTD. Full time

    **Job Summary: Are you eager to ride the wave of the software-defined revolution and be part of making a difference in the world? Do you have a passion for combining next-generation software and hardware to develop leading-edge, creative solutions to meet your customer’s challenges? QCT’s portfolio of leadership software-defined datacenter products with...

  • Senior/ Cloud Infra Engineer

    1 week ago

    Singapore ST ENGINEERING CLOUD AND DATA CENTRE SOLUTIONS PTE. LTD. Full time

    **Responsibilities: - Define, deploy and manage cloud-based workloads, security and compliance controls. - Implementing and maintaining cloud security measures, such as firewalls, encryption, and access controls - Managing cloud resources and provisioning capacity as needed - Troubleshooting issues and providing technical support to end-users - Regularly...

  • AWS Cloud Data Engineer

    2 weeks ago

    Singapore EPS Consultants Full time

    Job Title: AWS ETL Cloud Data Engineer Job Overview: The AWS Cloud Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines and data infrastructure in the AWS cloud environment. This role requires expertise in AWS services, data modeling, ETL processes, and a keen understanding of best practices for data management...