Data/ Databricks Engineer

6 days ago

Singapore AVN DIGITAL PTE. LTD. Full time
Roles & Responsibilities

AVN-Digital is an expert in providing technology-enabled business solutions and services. Since inception, AVN-Digital has helped clients Cyber-security, Data Analytics and Robotic Process Automation to improve their operations and profitability, focus on core competencies and achieve business results such as increased agility, innovation and profitable growth. Our in-depth technical knowledge, industry experience, and unique methodologies enable us to successfully deliver and meet our customer’s expectations.

We are currently looking to hire a Databricks Engineer - IDMC. This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.

JOB TYPE

Long Term Contract

ROLES AND REPONSIBILITIES

  • Design and architect data storage solutions, including databases, data lakes, and warehouses, using AWS services such as Amazon S3, Amazon RDS, Amazon Redshift, and Amazon DynamoDB, along with Databricks' Delta Lake.
  • Integrate Informatica IDMC for metadata management and data cataloging.
  • Create, manage, and optimize data pipelines for ingesting, processing, and transforming data using AWS services like AWS Glue, AWS Data Pipeline, and AWS Lambda, Databricks for advanced data processing, and Informatica IDMC for data integration and quality.
  • Integrate data from various sources, both internal and external, into AWS and Databricks environments, ensuring data consistency and quality, while leveraging Informatica IDMC for data integration, transformation, and governance.
  • Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and enrich data, making it suitable for analytical purposes using Databricks' Spark capabilities and Informatica IDMC for data transformation and quality.
  • Monitor and optimize data processing and query performance in both AWS and Databricks environments, making necessary adjustments to meet performance and scalability requirements. Utilize Informatica IDMC for optimizing data workflows.
  • Implement security best practices and data encryption methods to protect sensitive data in both AWS and Databricks, while ensuring compliance with data privacy regulations. Employ Informatica IDMC for data governance and compliance.
  • Implement automation for routine tasks, such as data ingestion, transformation, and monitoring, using AWS services like AWS Step Functions, AWS Lambda, Databricks Jobs, and Informatica IDMC for workflow automation.
  • Maintain clear and comprehensive documentation of data infrastructure, pipelines, and configurations in both AWS and Databricks environments, with metadata management facilitated by Informatica IDMC.
  • Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver appropriate solutions across AWS, Databricks, and Informatica IDMC.
  • Identify and resolve data-related issues and provide support to ensure data availability and integrity in both AWS, Databricks, and Informatica IDMC environments.
  • Optimize AWS, Databricks, and Informatica resource usage to control costs while meeting performance and scalability requirements.
  • Stay up-to-date with AWS, Databricks, Informatica IDMC services, and data engineering best practices to recommend and implement new technologies and techniques.

REQUIREMENTS

  • Bachelor’s or master’s degree in computer science, data engineering, or a related field.
  • Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC.
  • Proficiency in programming languages such as Python, Java, or Scala for building data pipelines.
  • Evaluate potential technical solutions and make recommendations to resolve data issues especially on performance assessment for complex data transformations and long running data processes.
  • Strong knowledge of SQL and NoSQL databases.
  • Familiarity with data modeling and schema design.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • AWS certifications (e.g., AWS Certified Data Analytics - Specialty, AWS Certified Data Analytics - Specialty), Databricks certifications, and Informatica certifications are a plus

CONSULTANT DETAILS

Consultant Name: Keerthana Ramakrishnan


Tell employers what skills you have

RDS
Amazon RDS
Data Modeling
Amazon Redshift
Informatica
Data Transformation
Data Integration
Data Governance
Data Engineering
Robotic Process Automation
Metadata
Cataloging
Encryption
AWS Lambda
S3

  • Data/ Databricks Engineer

    6 days ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service...

  • Data Engineer

    1 month ago

    Singapore INFOCEPTS PTE. LTD. Full time

    Roles & ResponsibilitiesInfoCepts is a global leader of end-to-end data & analytics solutions with nearly 20 years of experience, also named as Gartner’s 2020, 2021 and 2022 customers’ choice for Data & Analytics providers. We continue to grow rapidly year over year, now employing more than 1,200 people in offices across the globe. As we have grown, we...

  • Cloud Data Engineering Leader

    3 weeks ago

    Singapore Databricks Full time

    Estimated Salary: $150,000 - $200,000 per yearDatabricks is seeking a highly skilled Cloud Data Engineering Leader to guide the technical evaluation phase in a hands-on environment throughout the sales process. As a key member of our pre-sales team, you will work closely with customers to help them achieve tangible data-driven outcomes through the use of our...

  • Databricks Data Architect

    1 day ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    We are seeking an experienced Databricks Data Architect to join our team at ZENITH INFOTECH (S) PTE LTD.Job Description:A Databricks Data Architect is responsible for designing and implementing data architecture on Databricks, ensuring high-quality data management, security, and compliance. The role involves migrating from as-is on-prem data warehouse to...

  • Databricks Engineer

    6 days ago

    Singapore KRIS INFOTECH PTE. LTD. Full time

    Roles & ResponsibilitiesJob Description: Responsible for migrating from as-is on-perm data warehouse to Databricks and sound knowledge of Databricks. Requires good understanding of data architecture, cloud solutions, building and maintaining a robust, integrated, and governed data infrastructure. The role involves extracting valuable insights from data...

  • Databricks Engineer

    6 days ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesJob DescriptionAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in...

  • Databricks Data Engineering Professional

    6 days ago

    Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full time

    We are seeking a skilled Data Engineer with 3 to 5+ years of experience in Databricks and Python to join Elliott Moss Consulting PTE. LTD.Job OverviewThis role is ideal for individuals who have expertise in designing, developing, and optimizing data pipelines and analytics solutions on Databricks. Experience with AWS Cloud is highly preferred.The successful...

  • Databricks Engineer with Data Migration Expertise

    6 days ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    Unlock Your Potential as a Databricks EngineerAbout the Role:We are seeking an experienced Databricks Engineer to lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime. This is a fantastic opportunity for someone who is passionate about working with big data technologies and has expertise in AWS or Azure...

  • Data Engineer

    1 month ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    Roles & Responsibilities More than 3 to 5 years of experience in Databricks and Python. Candidates must be proficient on data engineer using Data bricks and Python Need to have Python and Databricks skills. Working exp on AWS cloud is preferrable Databricks, Python and AWS Must have experience in data engineering role Candidates must be open for...

  • Data Architect with Databricks Expertise

    6 days ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    We are seeking a highly skilled Data Architect to join our team at AVENSYS CONSULTING PTE. LTD. This is an exciting opportunity to work on challenging projects and expand your skill set as a Databricks Engineer.The ideal candidate will have a strong background in data architecture, cloud solutions, and data governance. They will be responsible for designing...

  • Data Solutions Consultant

    3 weeks ago

    Singapore Databricks Full time

    As a Data Solutions Consultant at Databricks, you will play a vital role in driving the adoption and growth of our Data Intelligence Platform. With 7+ years of experience in technical project delivery, you will collaborate with our sales and field engineering teams to accelerate customer success.This is a hybrid technical and commercial role that requires...

  • Databricks Engineer

    6 days ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    Roles & ResponsibilitiesZenith Infotech (S), a tech recruitment firm with 27 years of presence in Singapore, is assisting one of our reputable clients to lookout for Databricks Engineer with 5 years of experience. Job Scope:• Lead end to end data migration project from on-premises environments to Databricks with minimal downtime.• Work with architects...

  • Databricks Engineer Lead

    4 days ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    At Avensys Consulting Pte Ltd, we are seeking an experienced Databricks Engineer to join our team. This exciting opportunity will challenge you to lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime.About the RoleLead solution design and implementation of Databricks solutions on AWS.Design and implement...

  • Databricks Engineer

    6 days ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    Roles & ResponsibilitiesA Databricks Engineer is responsible for migrating from as-is on-perm data warehouse to Databricks and sound knowledge of Databricks. Requires good understanding of data architecture, cloud solutions, building and maintainirobust, integrated, and governed data infrastructure. The role involves extracting valuable insights from...

  • Data Solutions Architect

    3 weeks ago

    Singapore Databricks Full time

    Job OverviewWe are seeking a highly skilled Data Solutions Architect to join our team in ASEAN. As part of our accelerating growth, this role will play a critical part in driving revenue and expansion across the region.Job DescriptionIn this exciting new logo Sales role, you will be responsible for acquiring and growing some of our focus customers in...

  • Cloud Data Engineer

    6 days ago

    Singapore HELIUS TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesDo you love healthtech and Singapore put together? Then you just might love this companyThe company is a multiple award-winning Healthcare IT Leader that digitises, connects, and analyses Singapore's health ecosystem Its ultimate aim is to improve the Singapore population's health and health administrations by integrating...

  • Enterprise Sales Leader for Big Data and Cloud Solutions

    3 weeks ago

    Singapore Databricks Full time

    At Databricks, we are on a mission to accelerate innovation in big data, cloud, and AI. As an Enterprise Account Executive, you will play a critical role in driving revenue growth and expanding our business in the ASEAN region.The key responsibilities of this position include:Elevating Databricks' Unified Analytics Platform powered by Spark and launching the...

  • Databricks Engineer and Data Infrastructure Specialist

    2 days ago

    Singapore AVENSYS CONSULTING PTE. LTD. Full time

    Job OverviewAVENSYS CONSULTING PTE. LTD. is a leading global IT professional services company headquartered in Singapore. We offer a comprehensive range of services, including enterprise solution consulting, business intelligence, business process automation, and managed services.About the RoleWe are currently seeking a highly skilled Databricks Engineer to...

  • Field engineering, generative ai product specialist

    8 hours ago

    Singapore Databricks Inc. Full time

    As the Product Specialist for Generative AI, you will lead the enablement, expertise, and backline support for our customer-facing field team with Generative AI, including supporting RAG (retrieval augmented generation), open source models like Llama2, MPT, Mistral, 3rd party models including Open AI, Vector Indexing, and foundational model pre-training...

  • Databricks Engineer: Cloud Native Expert

    6 days ago

    Singapore ZENITH INFOTECH (S) PTE LTD. Full time

    We are seeking a highly skilled Databricks Engineer to join our team at Zenith Infotech (S) Pte Ltd. This is a 6-12 months contract role that offers a competitive salary of $7,000-$8,250.The ideal candidate will have a strong background in data engineering, with expertise in AWS or Azure services, Databricks, and/or Informatica IDMC. They will be responsible...