Data Engineer

7 days ago

Singapore EVOLUTION RECRUITMENT SOLUTIONS PTE. LTD. Full time
Roles & Responsibilities

Databricks Engineer (Databricks, Python or PySpark)

12 Months Renewable Contract


The Databricks Engineer is responsible for migrating from as-is on-perm data warehouse to Databricks and should have sound knowledge of Databricks. Requires good understanding of data architecture, cloud solutions, building and maintaining a robust, integrated, and governed data infrastructure. The role involves extracting valuable insights from data while ensuring data security, compliance, and high-quality data management.


Roles And Responsibilities:

  • Lead end to end data migration project from on-premises environments to Databricks with minimal downtime.
  • Work with architects and lead solution design to meet functional and non-functional requirements.
  • Hands on experience in Databricks to design and implement the solution on AWS.
  • Hands on experience in configuring Databricks clusters, writing Pyspark codes, build CI/CD pipelines for the deployments.
  • Highly experienced in optimization techniques (Zordering, Auto Compaction, vacuuming)
  • Process near real time data through Auto Loader, DLT pipelines
  • Must have strong background in Python and able to identify, communicate and mitigate risks and issues.
  • Identify and resolve data-related issues and provide support to ensure data availability and integrity.
  • Optimize AWS, Databricks resource usage to control costs while meeting performance and scalability requirements.
  • Stay up to date with AWS, Databricks services, and data engineering best practices to recommend and implement new technologies and techniques.
  • Proactively implement engineering methodologies, standards, and leading practices.

Requirements / Qualifications:

  • Bachelor’s or Master’s degree in computer science, data engineering, or a related field.
  • Minimum 5 years of experience in data engineering, with expertise in AWS or Azure services, Databricks, and/or Informatica IDMC.
  • Proficiency in programming languages such as Python, Java, or Scala for building data pipelines.
  • Evaluate potential technical solutions and make recommendations to resolve data issues especially on performance assessment for complex data transformations and long running data processes.
  • Strong knowledge of SQL and NoSQL databases.
  • Familiarity with data modelling and schema design.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.
  • Databricks certifications, and Informatica certifications are a plus.

Preferred Skills:

  • Experience with big data technologies like Apache Spark and Hadoop on Databricks.
  • Experience in AWS Services focusing on data and architecting.
  • Knowledge of containerization and orchestration tools like Docker and Kubernetes.
  • Familiarity with data visualization tools like Tableau or Power BI.
  • Understanding of DevOps principles for managing and deploying data pipelines.
  • Experience with version control systems (e.g., Git) and CI/CD pipelines.
  • Knowledge of data governance and data cataloguing tools, especially Informatica IDMC.



Tell employers what skills you have

Tableau
Git
Cloud solutions
PySpark
Apache Spark
Scala
Azure
AWS
Hadoop
Informatica
Databricks
NoSQL
SQL
Python
Data Architecture
data infrastructure
Power BI

  • Bioprocessing data engineer

    3 weeks ago

    Singapore Data Privacy Full time

    Job Description We're seeking a talentedBioprocess Data Engineerto join our team and play a crucial role in establishing the framework and systems for bioprocess data in a greenfield facility. As a Bioprocess Data Engineer, you'll bridge the gap between digital infrastructure and process engineers, creating a user-friendly interface for data...

  • Senior Data Center Support Engineer

    2 weeks ago

    Singapore NTT DATA Full time

    About the RoleWe are seeking a skilled Senior Data Center Support Engineer to join our team at NTT DATA. As a key member of our technical support group, you will be responsible for providing high-level technical support and field engineering services to clients.Job ResponsibilitiesProvide second-line remote and onsite technical support to clients, resolving...

  • Frontend Software Engineer

    3 weeks ago

    Singapore Twelve Data Full time

    At Twelve Data, we're on a mission to make the world's financial data universally accessible. We aggregate and store vast volumes of data, leveraging custom terabytes-scale distributed time series databases to deliver high-speed performance.We're seeking an experienced Frontend Software Engineer to join our core development team. As a key member, you'll...

  • Strategic Data Insights Lead

    3 weeks ago

    Singapore NTT DATA Full time

    Transform Business with NTT DATAAbout the Role:We are seeking a seasoned Strategic Data Insights Lead to join our team at NTT DATA. As a key member of our analytics capacity, you will be responsible for driving business value using data analytics, data visualization, and data modelling techniques and technologies.Key Responsibilities:Gather relevant...

  • Strategic Data Insights Specialist

    4 weeks ago

    Singapore NTT DATA Full time

    About the RoleWe are seeking a seasoned Strategic Data Insights Specialist to join our team at NTT DATA. This is an exciting opportunity for a highly skilled professional to work with data collection tools and software, producing actionable insights that drive business value.Key ResponsibilitiesData Analysis: Gather relevant information from multiple...

  • Data scientist

    3 weeks ago

    Singapore ECF Data, LLC Full time

    Description:Location: SingaporeBudget : 8000 -10000Responsibilities:1. Drive analytic projects for all pillars of the bank’s treasury and markets department, such as bond analytics, news generation FX trading signals, product recommendation and customer hyper-personalization.2. Conduct literature review and evaluate trade-offs between different...

  • Business Analyst

    4 weeks ago

    Singapore NTT DATA Full time

    Transform Business with Data InsightsAbout the RoleThe Business Analyst - Data Insights Specialist is a seasoned subject matter expert responsible for driving business value using data analytics, data visualization, and data modeling techniques and technologies. This role transforms data into critical information and knowledge to inform sound business...

  • Global Leader in Data Center Design

    3 weeks ago

    Singapore NTT DATA Full time

    About the RoleWe are seeking an experienced Senior Director, Design Management to lead our team of design professionals and oversee the development of high-quality data center designs for our clients in Japan and APAC regions.Job DescriptionThe ideal candidate will have a strong background in designing critical facilities and complex infrastructure, with at...

  • Red Hat OpenShift Engineer

    3 weeks ago

    Singapore NTT Data Singapore Full time

    We are seeking a highly skilled Red Hat OpenShift Engineer to join our team at NTT Data Singapore. As an OpenShift Engineer, you will play a critical role in ensuring the uptime, performance, and compliance of Red Hat OpenShift Container Platform (OCP) according to bank standards.You will work closely with architecture, development, operations, and security...

  • Data Center Energy Specialist

    3 weeks ago

    Singapore Vantage Data Centers Full time

    About Vantage Data CentersVantage Data Centers is a leading provider of data center services, committed to delivering innovative and sustainable solutions for hyperscalers, cloud providers, and large enterprises. With a presence in North America, EMEA, and Asia Pacific, we have evolved data center design to achieve dramatic gains in reliability, efficiency,...

  • Data Architect Specialist

    2 days ago

    Singapore TRIVI DATA CONSULTING PTE. LTD. Full time

    Unlock Business Potential with Data Engineering ExpertiseWe are seeking a seasoned Data Engineer to join our team at NITYO PROGRESSION CONSULTANCY PTE. LTD. The ideal candidate will have a proven ability to design scalable data pipelines, implement machine learning models, and deliver data-driven solutions that optimize decision-making processes.About the...

  • Senior Leader, Data Centre Engineering Specialist

    4 weeks ago

    Singapore KEPPEL DATA CENTRES HOLDING PTE. LTD. Full time

    About the RoleWe are seeking an exceptional Senior Manager, Technical Solutions to lead our Data Centre Engineering projects from inception to completion. This is a key position that requires strong technical expertise and leadership skills.Key Responsibilities:Develop and manage technical solutions for Data Centre Engineering projects, ensuring timely...

  • Big Data Support Specialist

    2 weeks ago

    Singapore NTT DATA Services Full time

    About NTT DATANTT DATA is a trusted global innovator of business and technology services.We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long term success. Job OverviewThe Software Development Senior Analyst role requires strong data engineering concepts and programming experience. This...

  • Performance Test Engineer

    2 weeks ago

    Singapore NTT DATA Full time

    About the RoleWe are seeking a skilled Performance Test Engineer to join our team at NTT DATA. As a key member of our technical team, you will play a critical role in ensuring the performance and reliability of our software applications.As a Performance Test Engineer, you will be responsible for designing, developing, and executing performance tests to...

  • Quality Assurance Engineer

    3 weeks ago

    Singapore NTT DATA Full time

    About NTT DATA NTT DATA is a global technology services leader, dedicated to helping clients innovate and transform for long-term success. With over $30 billion in revenue and more than 50 years of experience, we serve 75% of the Fortune Global 100. Your Role: We are seeking a skilled Quality Assurance Engineer to join our team. As a Quality Assurance...

  • Cloud and Data Centre Solutions Engineer

    4 weeks ago

    Singapore ST ENGINEERING CLOUD AND DATA CENTRE SOLUTIONS PTE. LTD. Full time

    Job Title: Cloud and Data Centre Solutions EngineerWe are seeking an experienced Cloud and Data Centre Solutions Engineer to join our team at ST Engineering Cloud and Data Centre Solutions Pte. Ltd.Estimated Salary: SGD 120,000 - 180,000 per annumAs a Cloud and Data Centre Solutions Engineer, you will play a key role in designing, implementing, and managing...

  • High-Tech Solutions Architect for Data Center Expansion

    3 weeks ago

    Singapore Vantage Data Centers Full time

    About Vantage Data CentersWe are a leading provider of data center infrastructure, committed to delivering reliable, efficient, and sustainable solutions to our customers.Job SummaryWe are seeking an experienced Customer Solutions Engineer to join our team. In this role, you will be responsible for supporting the development and execution of technical...

  • Field Support Engineer for Collaboration Technologies

    4 weeks ago

    Singapore NTT DATA Full time

    About the RoleNTT DATA is seeking an experienced Field Support Engineer to join our team. In this role, you will provide second-line remote and onsite technical support to clients, ensuring their collaboration technologies are configured, installed, and operational.Key ResponsibilitiesConfigure, install, and test client infrastructure to ensure it meets...

  • Global Collaboration Support Engineer

    3 weeks ago

    Singapore NTT DATA Full time

    About the RoleWe are seeking a talented Global Collaboration Support Engineer to join our team. As a key member of our support team, you will be responsible for providing exceptional technical support and engineering services to clients worldwide.Key Responsibilities:Configure, install, test, and operate client infrastructure at site locations.Install and...

  • Senior Python Software Engineer

    2 weeks ago

    Singapore NTT DATA Full time

    About the JobNTT DATA is seeking an experienced Senior Python Software Engineer to join our team. This is a fantastic opportunity to work with a global leader in business and technology services.Job SummaryWe are looking for a highly skilled professional with 5+ years of hands-on experience in developing serverless applications using AWS Lambda, DynamoDB,...