High-Performance Data Architect

2 days ago


Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full time
Job Summary

A High-Performance Data Architect at Elliott Moss Consulting PTE. LTD. is responsible for designing and implementing scalable data architectures on Databricks within AWS environments, ensuring seamless data integration and governance. The role involves leading end-to-end data migration projects, configuring Databricks clusters, writing PySpark code, and developing CI/CD pipelines for deployment.

As a key member of the team, you will apply optimization techniques like Z-ordering, auto-compaction, and vacuuming to enhance performance. You will also process near real-time data using Auto Loader and DLT pipelines, optimize AWS and Databricks resources for performance and cost efficiency, and stay updated with advancements in AWS, Databricks services, and data engineering practices.

We are seeking an experienced professional with a minimum of 5 years in data engineering, expertise in Databricks, AWS/Azure services, or Informatica IDMC. Strong knowledge of Python, PySpark, SQL, and NoSQL databases is required, along with hands-on experience with Apache Spark, Hadoop, and cloud-based data architectures.

Key Responsibilities
  • Design and implement scalable data architectures on Databricks within AWS environments.
  • Lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime.
  • Configure Databricks clusters, write PySpark code, and develop CI/CD pipelines for deployment.
  • Apply optimization techniques like Z-ordering, auto-compaction, and vacuuming for enhanced performance.
  • Process near real-time data using Auto Loader and DLT pipelines.
  • Optimize AWS and Databricks resources for performance and cost efficiency.
Requirements
  • Minimum 5 years of experience in data engineering, with expertise in Databricks, AWS/Azure services, or Informatica IDMC.
  • Strong knowledge of Python, PySpark, SQL, and NoSQL databases.
  • Hands-on experience with Apache Spark, Hadoop, and cloud-based data architectures.
  • Proficiency in building data pipelines using Python, Java, or Scala.
  • Expertise in data modelling, schema design, and data governance.
Preferred Skills
  • Experience with containerization and orchestration tools like Docker and Kubernetes.
  • Knowledge of data visualization tools like Tableau or Power BI.
  • Familiarity with DevOps principles, CI/CD pipelines, and version control systems like Git.
  • Understanding of data cataloguing tools and data governance, especially Informatica IDMC.
Compensation

The estimated salary for this role is SGD $120,000 - SGD $180,000 per annum, depending on experience and qualifications.



  • Singapore CAPGEMINI SINGAPORE PTE. LTD. Full time

    Overview of the RoleWe are seeking an experienced Data Architect to join our team in Singapore. The ideal candidate will have a strong background in designing and implementing high-performance data solutions.In this role, you will be responsible for architecting and implementing robust data pipelines and ETL processes to ensure data quality, accuracy, and...


  • Singapore NTT DATA Services Full time

    About this RoleWe are seeking a highly skilled High-Performance Network Architect to join our team in Singapore. The successful candidate will design, build, and maintain high-performance network architectures that support our business operations.Salary Estimate: SGD 120,000 - 180,000 per annumJob DescriptionThis role requires a deep understanding of...


  • Singapore PEOPLE PROFILERS (SERVICES) PTE. LTD. Full time

    Job OverviewWe are seeking an experienced High-Performance Software Architect to join our team at PEOPLE PROFILERS (SERVICES) PTE. LTD.About the RoleThis is a challenging and rewarding opportunity for a seasoned software architect to lead the design and development of high-performance, scalable software systems.Key ResponsibilitiesLead end-to-end...


  • Singapore NTT DATA SINGAPORE PTE. LTD. Full time

    Unlock Your Potential as a Performance TesterWe are seeking a highly skilled High-Performance Test Engineer to join our team at NTT DATA SINGAPORE PTE. LTD.About the RoleThe successful candidate will be responsible for delivering high-quality performance testing solutions that meet the demands of our dynamic business environment. This role requires a strong...


  • Singapore NTT DATA Full time

    Role OverviewNTT DATA is looking for a skilled Performance Tester to join our team. As a Performance Tester, you will play a critical role in ensuring the high-quality and performance of our applications.Your Key ResponsibilitiesDesign, execute, and analyze performance tests to identify and troubleshoot performance issues.Collaborate with development, QA,...


  • Singapore NTT DATA Full time

    Job DescriptionReq ID: 293528 NTT DATA is committed to attracting exceptional, innovative, and passionate individuals who can contribute to our growth. We strive to create a diverse and inclusive environment, fostering collaboration and innovation. We are currently seeking a Low Latency Network Architect to join our dynamic team in Singapore. The ideal...


  • Singapore TIKTOK PTE. LTD. Full time

    About TIKTOK PTE. LTD.TIKTOK PTE. LTD. is a leading global platform for short-form mobile video, committed to inspiring creativity and bringing joy to users worldwide.Job DescriptionWe are seeking an experienced High-Performance System Architect or Core Service Developer to join our Global Live Service Architecture team. This role offers a unique opportunity...


  • Singapore TIKTOK PTE. LTD. Full time

    About TikTok PTE. LTD.TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. With global offices across the globe, including Los Angeles, New York, London, Paris, Berlin, Dubai, Singapore, Jakarta, Seoul, and Tokyo, we strive to create a unique e-commerce experience for our users.Why Choose This...


  • Singapore TRIVI DATA CONSULTING PTE. LTD. Full time

    Unlock Business Potential with Data Engineering ExpertiseWe are seeking a seasoned Data Engineer to join our team at NITYO PROGRESSION CONSULTANCY PTE. LTD. The ideal candidate will have a proven ability to design scalable data pipelines, implement machine learning models, and deliver data-driven solutions that optimize decision-making processes.About the...


  • Singapore MARQUEE SEMICONDUCTOR SINGAPORE PTE. LTD. Full time

    At Marquee Semiconductor Singapore PTE. LTD., we are seeking a highly skilled Senior RTL Engineer to join our team.About the RoleThe estimated salary for this position is SGD 180,000 - SGD 250,000 per annum, commensurate with experience and qualifications.We are looking for an exceptional individual with expertise in architecting and micro-architecting...


  • Singapore KOK TONG CONSTRUCTION PTE LTD Full time

    Job SummaryWe are seeking an experienced Software Engineer to join our team at KOK TONG CONSTRUCTION PTE LTD. As a key member of our development team, you will play a crucial role in designing and architecting high-quality software solutions.About the RoleDesign and Development: You will work closely with our business users and system architects to gather...


  • Singapore NTT DATA Full time

    Job Overview">Industry-Leading Innovator Seeks Expertise in Banking Middleware SolutionsWe are a global leader in business and technology services, driving innovation and excellence for our clients. As a Senior Associate Software Engineer, you will play a crucial role in managing system development, implementation, and support of banking middleware...


  • Singapore MATRIX PROCESS AUTOMATION PTE. LTD. Full time

    Job Summary:"> We are seeking an experienced High-Performance Oracle Database Architect to join our team at Matrix Process Automation PTE. LTD. As a key member of our technical staff, you will be responsible for designing, implementing, and optimizing high-performance database solutions that meet the evolving needs of our clients."> In this role, you will...


  • Singapore LION CITY INNOVATIONS PTE. LTD. Full time

    Lion City Innovations PTE. LTD.Company OverviewLion City Innovations PTE. LTD. is a pioneering company that empowers innovation in the heart of Singapore, driving technological advancements and creating cutting-edge solutions.Salary Range: SGD 140,000 - SGD 200,000 per annumAbout the RoleWe are seeking an experienced Senior Software Engineer to join our team...


  • Singapore NTT DATA Services Full time

    Job Overview:We are seeking an experienced High-Performance Network Infrastructure Specialist to join our team in Singapore.About the Role:This is a challenging opportunity for a seasoned professional to design and implement high-performance network architectures, ensuring seamless data transmission and minimizing latency.Key Responsibilities:Design and...


  • Singapore Changi Airport Full time

    Job Description:We are seeking a highly skilled Cloud Architect to lead the development and maintenance of our high-performance middleware infrastructure at Changi Airport. This is an exciting opportunity to work with cutting-edge technology and contribute to the growth of our airport's digital capabilities.About the Role:This is a challenging role that...


  • Singapore NTT DATA Services Full time

    We are seeking a highly skilled High-Performance Network Infrastructure Engineer to join our team in Singapore. This role will be responsible for designing, implementing, and maintaining high-performance network infrastructure solutions.About the Role:Design and implement high-performance network infrastructure solutionsMaintain and optimize existing network...


  • Singapore PEOPLE PROFILERS (SERVICES) PTE. LTD. Full time

    Job OverviewAt People Profilers (Services) Pte. Ltd., we are seeking a seasoned Senior Software Engineer to spearhead our technical endeavors. With an estimated salary of $150,000-$200,000 per annum, this role is designed for individuals who possess exceptional expertise in software architecture, system design, and distributed systems.About the RoleWe are on...


  • Singapore NVIDIA Full time

    We are seeking a seasoned Senior Solution Architect to join our NVIDIA team in Singapore.This is an exceptional opportunity to expand your career in GPU Infrastructure and artificial intelligence, driving the latest breakthroughs in deep learning and AI while empowering the world's most successful companies.Key Responsibilities: Collaborate with our Asia...

  • Data architect

    3 weeks ago


    Singapore Infosys Full time

    We are looking for a highly experienced and skilled Data Architect to join our team. The ideal candidate will have 12-15 years of experience in architecting solutions of data engineering, focusing on ELT and Py Spark/Hadoop workloads. In addition to strong solution and delivery skills, the ideal candidate will also have a view on business growth and...