Data Engineering Expert

2 weeks ago

Singapore 3 CUBED BUSINESS CONSULTING PTE. LTD. Full time

**1. Data Engineering & Platform Knowledge (Must)**
- Strong understanding of Hadoop ecosystem: HDFS, Hive, Impala, Oozie, Sqoop, Spark (on YARN).
- Experience in data migration strategies (lift & shift, incremental, re-engineering pipelines).
- Knowledge of Databricks architecture (Workspaces, Unity Catalog, Clusters, Delta Lake, Workflows).

**2. Testing & Validation (preferred)**
- Data reconciliation (source vs. target).
- Performance benchmarking.
- Automated test frameworks for ETL pipelines.

**3. Databricks-Specific Expertise (preferred)**
- Delta Lake: ACID transactions, time travel, schema evolution, Z-ordering.
- Unity Catalog: Catalog/schema/table design, access control, lineage, tags.
- Workflows/Jobs: Orchestration, job clusters vs. all-purpose clusters.
- SQL Endpoints / Databricks SQL: Designing downstream consumption models.
- Performance Tuning: Partitioning, caching, adaptive query execution (AQE), photon runtime.

**4. Migration & Data Movement (preferred)**
- Data migration from HDFS/Cloudera to cloud storage (ADLS/S3/GCS).
- Incremental ingestion techniques (Change Data Capture, Delta ingestion frameworks).
- Mapping Hive Metastore to Unity Catalog (metastore migration).
- Refactoring HiveQL/Impala SQL to Databricks SQL (syntax differences).

**5. Security & Governance (Nice to have)**
- Mapping Cloudera Ranger/SSO policies Unity Catalog RBAC.
- Azure AD / AWS IAM integration with Databricks.
- Data encryption, masking, anonymization strategies.
- Service Principal setup & governance.

**6. DevOps & Automation (Nice to have)**
- Infrastructure as Code (Terraform for Databricks, Cloud storage, Networking).
- CI/CD for Databricks (GitHub Actions, Azure DevOps, Databricks Asset Bundles).
- Cluster policies & job automation.
- Monitoring & logging (Databricks system tables, cloud-native monitoring).

**7. Cloud & Infra Skills (Nice to have)**
- Strong knowledge of the target cloud (AWS/Azure/GCP):

- Storage (S3/ADLS/GCS).
- Networking (VNETs, Private Links, Security Groups).
- IAM & Key Management. 9. Soft Skills
- Ability to work with business stakeholders for data domain remapping.
- Strong documentation and governance mindset.
- Cross-team collaboration (infra, security, data, business).

  • Data Center Project Engineer

    6 days ago

    Singapore NTT DATA Full time $80,000 - $120,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Data Centre...

  • Data Centre Delivery Services Engineer

    6 days ago

    Singapore NTT Data Singapore Full time $120,000 - $180,000 per year

    ResponsibilitiesRead, interpret, and validate data centre floor plans and technical drawingsDesign and plan DC hall layouts for newly allocated spaces, including rack positioning, structured cabling, power, and cooling designReview and assess vendor-submitted data centre plans, evaluate proposed technologies, and clarify design...

  • Data Center Project Engineer

    4 days ago

    Singapore NTT DATA, Inc. Full time

    Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive. Your day at NTT DATA The Data...

  • Site Admin

    2 weeks ago

    Singapore JOB EXPERT PTE. LTD. Full time

    Roles & Responsibilities Working time: 8am - 6pm, 6 days a week Responsibilities: - Handle filing, data entry, and general admin duties - Track daily attendance, manpower, and equipment usage - Prepare and submit reports, permits, and safety documents - Liaise with contractors, suppliers, and site staff - Keep records of deliveries, materials, and documents...

  • Data Security Expert

    1 week ago

    Singapore Crédit Agricole CIB Full time

    **Description du poste**: **Position** Data Security Expert is a technical expert role within the Data Security Services team and will be responsible for managing, administration and supporting the Data Security and Endpoint Security technologies for the bank. Person will be the technical point of contact for the Run team and the related activities to data...

  • Network Engineering Specialist

    2 weeks ago

    Singapore NTT DATA Full time

    **Req ID**: 133019 We are currently seeking a Network Engineering Specialist to join our team in Singapore, Central Singapore (SG-01), Singapore (SG). Job Title: Network Engineering Specialist Summary: The Networking Engineering Specialist will be responsible for production and non-production Device Design/Policy/functionality for service implementation....

  • Senior Data Scientist

    1 week ago

    Singapore NTT DATA Full time $120,000 - $180,000 per year

    Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it's a place where you can grow, belong and thrive.Your day at NTT DATAThe Senior Data...

  • Head Chef

    2 weeks ago

    Singapore JOB EXPERT PTE. LTD. Full time

    Roles & Responsibilities Job Description Plan and direct food preparation and culinary activities in the kitchen. Modify menus or create new ones that meet quality standards. Estimate food requirements, which include food or labour costs budgeting as set by Management. Recruit and supervise kitchen staff, and schedule them according to operational needs....

  • Assistant Engineer

    2 days ago

    Singapore job expert Full time $40,000 - $60,000 per year

    Job DescriptionTo perform product set up for both testers, handlers, probers, and other equipmentTo troubleshoot equipment issues (handler/prober/tester) and ensure low machinedown timeTo perform setup conversion, periodic maintenance/calibrationWork with respective department for equipment installation/deinstallation andbuyoffCreate and update equipment and...

  • Data Migration Engineer

    4 days ago

    Singapore Capgemini Engineering Full time $120,000 - $180,000 per year

    About the job you are consideringCapgemini Global Insights & Data business line is a market leader in the data, platform, and analytics across all regions and across many sectors (including financial services, consumer products, manufacturing & life sciences). Our offerings include end-2-end data integration to cloud platform, complete suite of AI...