Data Engineer Kafka

7 days ago

Singapore Thinkreh Full time

**Job Description for Data Enginer**

**Responsibilities**:
As Data Enginer, you will be responsible for building data pipelines from various systems to big data using big data technologies likes spark, python/scala, Kafka, SQL, Powe BI etc.
- Perform Structured/unstructured data ingestion into Hadoop and/or relational database
- Implement Batch/Streaming data ingestion pipelines using ETL and Big Data Technolgies
- Good hands on experience in **Python, PySpark/SPARK, kafka, HIVE/Impala and Beeline,SQL, Spark SQL, Power BI**
- ETL Experience using one of the ETL technologies (SSIS, informatica, Talend etc.) with strong SQL
- Experience in integrating data processes with architecture requirements used across company
- Understand Hadoop-ecosystem and Data Engineering activities as well as loading data from several disparate datasets and documentation
- Strong ability to troubleshoot and resolve data issues
- Analytical skill to perform data profiling and data visulization
- Experience in Agile and Waterfall frameworkWork
- Work closely with engineering and operations to document business processes
- Work independently and with team members to understand database structure and business processes

**Qualifications**:

- Graduate degree computer or related technical fields
- Minimum of 5 to 8 years working experience in technical data analysis, data warehousing using bis
- Experience in at least one or more languages: SPARK/PySpark, Python
- Experience with Hadoop ecosystem, preferably cloudera hadoop, Kafka Streaming
- ETL Experience using one of the ETL technologies (SSIS, informatica, Talend etc.) with strong SQL
- Hands On Experience on Power BI
- Good to have Hands-on experienceon the Hadoop, PYSPARK, Python, HIVE, Kafka ,Beelin, SparkSQL etc
- Excellent communication skills
- Passionate about data and analyzing business needs
- Previous experience on a data team in an agile environment preferred
- Hands-on experience on the Hadoop ecosystem, HDFS, Hadoop, Spark, Scala preferred
- Develop in-depth plans and major milestones that must be approved by top management during the planning and design phases of the project.

**Salary**: $3,773.50 - $11,809.84 per month

**Benefits**:

- Health insurance

Schedule:

- Monday to Friday

Supplemental pay types:

- Performance bonus

**Experience**:

- Big data: 5 years (required)
- HANDOOP: 4 years (required)
- PYTHON SPARK: 4 years (required)
- ETL: 4 years (required)
- Kafka: 4 years (required)

  • Kafka Engineer

    1 week ago

    Central Singapore PERSOLKELLY Full time

    Salary: SGD 6,800 Duration: 1 Year Contract (Renewable & Convertible) Working Hours: Weekdays, Office Hours **Kafka Engineer** **Salary: SGD 6,800** **Duration: 1 Year Contract (Renewable & Convertible)** **Working Hours: Weekdays, Office Hours** **Working Location: East / Central** - Handle the setup, configuration, and management of Kafka clusters,...

  • Kafka Developer

    3 days ago

    Singapore EVANTAGE HR PTE. LTD. Full time

    **Role Overview** **Key Responsibilities** - Design and implement Kafka Streams and KSQL-based transformations for real-time data processing. - Develop and maintain CDC pipelines using Kafka Connect, Debezium, or custom connectors. - Handle initial data loads from large relational datasets via manual exports and ensure seamless transition to incremental...

  • Kafka Engineer

    1 day ago

    Singapore CXC SINGAPORE PTE. LTD. Full time

    Our client is a global technology consulting and services company that enables enterprises across industries to drive superior competitive advantage, customer experiences and business outcomes by harnessing digital and cloud technologies **What you will do**: - Develop secure and stable Kafka platforms - Optimization and tuning for peak performance -...

  • Kafka Developer

    2 weeks ago

    Singapore NSEARCH GLOBAL PTE. LTD. Full time

    Our client, one of Asia-Pacific’s leading organizations is looking for: **Kafka Developer** **Responsibilities** - Design and implement Kafka Streams and KSQL-based transformations for real-time data processing. - Develop and maintain CDC pipelines using Kafka Connect, Debezium, or custom connectors. - Handle initial data loads from large relational...

  • Senior Apache Kafka Engineer

    3 days ago

    Singapore beBeeKafka Full time $150,000 - $200,000

    Job Opportunity:We are looking for an experienced Sr. Apache Kafka Engineer to lead the re-architecture and open-source adoption of our event streaming platform.This role demands deep technical expertise in Apache Kafka, strategic thinking for migration planning, and a passion for automation and team enablement.Key Responsibilities:Provide technical...

  • Kafka - DevOps

    1 week ago

    Central Singapore Emprego SG Full time

    **Location** Singapore, Central Singapore **Job Type** Contract **Salary** $6,500 - $8,500 Per Month **Date Posted** 7 hours ago Additional Details **Job ID** 3532 **Job Views** 1 Roles & Responsibilities **Role:Kafka - DevOps** **JD**: **Key Responsibilities** - Experience in architecting a highly resilient **Apache Kafka/Confluent Kafka...

  • Platform Engineer

    3 days ago

    Singapore Hunter Bond Full time

    **Job title**: Platform Engineer - Middleware / Consul / Kafka **Client**: Elite FinTech **Location**:Singapore **Skills**:Consul, Kafka, Ansible, Terraform, Git, Python The role: My client are looking for a talented Engineer with expertise in either Consul or Kafka. Essential Skills: - Consul OR - Kafka - Linux - Ansible - Terraform - Git - Python

  • Kafka Systems Architect

    2 days ago

    Singapore beBeeKafkaArchitect Full time $200,000 - $250,000

    Kafka Systems Architect RoleWe are seeking a skilled Kafka Systems Architect with extensive experience in designing and implementing large-scale Kafka systems. The ideal candidate will have a proven track record of leading successful Kafka migrations and deployments.Key ResponsibilitiesDevelop and implement scalable Kafka architectures to meet business...

  • Data Engineer

    5 days ago

    Singapore NodeFlair Full time

    **Job Summary**: **Salary** S$4,000 - S$7,000 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** ETL Analytics Hive Impala Spark kafka SQL Hadoop Python Requirement - 5+ years of relevant experience in data engineering/analytics space - Expertise in SQL and data analysis and strong hands-on expertise with...

  • Data Engineer

    7 days ago

    Singapore Hays Full time

    **Your new company** Your new company is a leader in the financial service space, focusing on digitalisation of the finance industry. As a Data engineering Lead, you will be apart of the growing team which will help shape the future of the company, working on a myriad of interesting challenges and latest technology. **Your new role** - Be able to drive,...