Databricks Engineer
5 days ago
Job Description:
- Responsible for migrating from as-is on-perm data warehouse to Databricks and sound knowledge of Databricks.
- Requires good understanding of data architecture, cloud solutions, building and maintaining a robust, integrated, and governed data infrastructure.
- The role involves extracting valuable insights from data while
ensuring data security, compliance, and high-quality data management.
Roles And Responsibilities:
• Lead end to end data migration project from on-premises environments to Databricks with minimal downtime.
• Work with architects and lead solution design to meet functional and non-functional requirements.
• Hands on experience in Databricks to design and implement the solution on AWS.
• Hands on experience in configuring Databricks clusters, writing Pyspark codes, build CI/CD pipelines for the
deployments.
• Highly experienced in optimization techniques (Zordering, Auto Compaction, vacuuming)
• Process near real time data through Auto Loader, DLT pipelines
• Must have strong background in python and able to identify, communicate and mitigate risks and issues.
• Identify and resolve data-related issues and provide support to ensure data availability and integrity.
• Optimize AWS, Databricks resource usage to control costs while meeting performance and scalability
requirements.
• Stay up to date with AWS, Databricks services, and data engineering best practices to recommend and
implement new technologies and techniques.
• Proactively implement engineering methodologies, standards, and leading practices.
Requirements:
• Bachelor’s or master’s degree in computer science, data engineering, or a related field.
• Minimum 5 years of experience in data engineering, with expertise in AWS or Azure services, Databricks, and/or
Informatica IDMC.
• Proficiency in programming languages such as Python, Java, or Scala for building data pipelines.
• Evaluate potential technical solutions and make recommendations to resolve data issues especially on
performance assessment for complex data transformations and long running data processes.
• Strong knowledge of SQL and NoSQL databases.
• Familiarity with data modelling and schema design.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• Databricks certifications, and Informatica certifications are a plus.
Preferred Skills:
• Experience with big data technologies like Apache Spark and Hadoop on Databricks.
• Experience in AWS Services focusing on data and architecting.
• Knowledge of containerization and orchestration tools like Docker and Kubernetes.
• Familiarity with data visualization tools like Tableau or Power BI.
• Understanding of DevOps principles for managing and deploying data pipelines.
• Experience with version control systems (e.g., Git) and CI/CD pipelines.
• Knowledge of data governance and data cataloguing tools, especially Informatica IDMC.
Tell employers what skills you have
Version Control
PySpark
Apache Spark
Scalability
Scala
Kubernetes
Azure
AWS
Informatica
Databricks
Data Governance
Data Engineering
Data Migration
Python
Containerization
Data Architecture
Java
Orchestration
Power BI
Data Visualization
-
Singapore Databricks Inc. Full timeAs the Product Specialist for Generative AI, you will lead the enablement, expertise, and backline support for our customer-facing field team with Generative AI, including supporting RAG (retrieval augmented generation), open source models like Llama2, MPT, Mistral, 3rd party models including OpenAI, Vector Indexing, and foundational model pre-training...
-
Singapore Databricks Inc. Full timeThe Databricks Customer Enablement team is on a mission to upskill Data and AI professionals in our customer organizations to use Databricks and accelerate customer success and outcomes. We are looking for a Customer Enablement Architect, reporting to our APJ Regional Enablement Lead, to be based in Singapore to support some of our regional customers. You...
-
Databricks Engineer
5 days ago
Singapore AVENSYS CONSULTING PTE. LTD. Full timeRoles & ResponsibilitiesJob DescriptionAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in...
-
Databricks Engineer
3 weeks ago
Singapore AVENSYS CONSULTING PTE. LTD. Full timeRoles & ResponsibilitiesJob DescriptionAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in...
-
Data/ Databricks Engineer
3 weeks ago
Singapore AVN DIGITAL PTE. LTD. Full timeRoles & ResponsibilitiesAVN-Digital is an expert in providing technology-enabled business solutions and services. Since inception, AVN-Digital has helped clients Cyber-security, Data Analytics and Robotic Process Automation to improve their operations and profitability, focus on core competencies and achieve business results such as increased agility,...
-
Databricks Engineer
3 weeks ago
Singapore KRISE SOLUTIONS PTE. LTD. Full timeRoles & ResponsibilitiesRequirements: Experience: minimum 4+ years in AWS cloud engineering(L2/L3) roles, specifically in support/operations. Must be experience with L2/L3 on GCC and GCP. Technical Skills: Strong expertise in AWS services setup/management (at a min. S3, IAM, EC2, SageMaker, AWS Networking), monitoring tools, and scripting...
-
Data/ Databricks Engineer
3 weeks ago
Singapore AVENSYS CONSULTING PTE. LTD. Full timeRoles & ResponsibilitiesAvensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service...
-
Databricks Engineer
3 weeks ago
Singapore ZENITH INFOTECH (S) PTE LTD. Full timeRoles & ResponsibilitiesZenith Infotech (S), a tech recruitment firm with 27 years of presence in Singapore, is assisting one of our reputable clients to lookout for Databricks Engineer with 5 years of experience. Job Scope:• Lead end to end data migration project from on-premises environments to Databricks with minimal downtime.• Work with architects...
-
Databricks Engineer Lead
2 weeks ago
Singapore AVENSYS CONSULTING PTE. LTD. Full timeAt Avensys Consulting Pte Ltd, we are seeking an experienced Databricks Engineer to join our team. This exciting opportunity will challenge you to lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime.About the RoleLead solution design and implementation of Databricks solutions on AWS.Design and implement...
-
Singapore Databricks Inc. Full timeAs the Product Specialist for Generative AI, you will lead the enablement, expertise, and backline support for our customer-facing field team with Generative AI, including supporting RAG (retrieval augmented generation), open source models like Llama2, MPT, Mistral, 3rd party models including Open AI, Vector Indexing, and foundational model pre-training...
-
Cloud Data Engineering Leader
1 month ago
Singapore Databricks Full timeEstimated Salary: $150,000 - $200,000 per yearDatabricks is seeking a highly skilled Cloud Data Engineering Leader to guide the technical evaluation phase in a hands-on environment throughout the sales process. As a key member of our pre-sales team, you will work closely with customers to help them achieve tangible data-driven outcomes through the use of our...
-
Databricks Engineer
3 weeks ago
Singapore ZENITH INFOTECH (S) PTE LTD. Full timeRoles & ResponsibilitiesA Databricks Engineer is responsible for migrating from as-is on-perm data warehouse to Databricks and sound knowledge of Databricks. Requires good understanding of data architecture, cloud solutions, building and maintainirobust, integrated, and governed data infrastructure. The role involves extracting valuable insights from...
-
Databricks Engineer: Cloud Native Expert
3 weeks ago
Singapore ZENITH INFOTECH (S) PTE LTD. Full timeWe are seeking a highly skilled Databricks Engineer to join our team at Zenith Infotech (S) Pte Ltd. This is a 6-12 months contract role that offers a competitive salary of $7,000-$8,250.The ideal candidate will have a strong background in data engineering, with expertise in AWS or Azure services, Databricks, and/or Informatica IDMC. They will be responsible...
-
Delivery Solutions Architect
1 month ago
Singapore Databricks Full timeAbout the Role:As a Delivery Solutions Architect at Databricks, you will play a pivotal role in empowering our customers to solve complex data problems. Your primary responsibility will be to collaborate with our sales and field engineering teams to accelerate the adoption and growth of the Databricks Data Intelligence Platform. You will work closely with...
-
Databricks Data Architect
2 weeks ago
Singapore ZENITH INFOTECH (S) PTE LTD. Full timeWe are seeking an experienced Databricks Data Architect to join our team at ZENITH INFOTECH (S) PTE LTD.Job Description:A Databricks Data Architect is responsible for designing and implementing data architecture on Databricks, ensuring high-quality data management, security, and compliance. The role involves migrating from as-is on-prem data warehouse to...
-
Databricks Data Engineering Professional
3 weeks ago
Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full timeWe are seeking a skilled Data Engineer with 3 to 5+ years of experience in Databricks and Python to join Elliott Moss Consulting PTE. LTD.Job OverviewThis role is ideal for individuals who have expertise in designing, developing, and optimizing data pipelines and analytics solutions on Databricks. Experience with AWS Cloud is highly preferred.The successful...
-
Databricks Solutions Architect
2 weeks ago
Singapore AVN DIGITAL PTE. LTD. Full timeUnlock Your Data PotentialAVN Digital Pte. Ltd., a technology-enabled business solutions and services expert, seeks a highly skilled Databricks Solutions Architect to design and implement data storage solutions using AWS services and Databricks' Delta Lake.This is an exciting opportunity for a seasoned professional to expand their skill set, achieve job...
-
Singapore ZENITH INFOTECH (S) PTE LTD. Full timeUnlock Your Potential as a Databricks EngineerAbout the Role:We are seeking an experienced Databricks Engineer to lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime. This is a fantastic opportunity for someone who is passionate about working with big data technologies and has expertise in AWS or Azure...
-
APJ Regional Customer Success Strategist
1 month ago
Singapore Databricks Full timeAbout This OpportunityWe are seeking a seasoned professional to join our team as a Customer Enablement Architect, based in Singapore. In this role, you will work closely with multiple teams including Sales, Field Engineering and Professional Services account teams to support customer onboarding and ongoing training needs.As a key member of the Databricks...
-
Solutions architect
1 month ago
Singapore Databricks Inc. Full timeAs a Pre-sales Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve...