Data Engineer Spark

Overview We are seeking a skilled Data Engineer to join our team in Singapore. The ideal candidate will have strong expertise in SQL, Python, Spark, and Hive , with hands-on experience building scalable data pipelines, data models, and analytical solutions. You will work closely with data scientists, analysts, and business stakeholders to deliver...

**Responsibilities**: - Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data - Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch - Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and...

**Job Summary**: **Job Type** **Seniority** Mid **Years of Experience** Information not provided **Tech Stacks** ETL AWS Google Cloud pySpark BigQuery RedShift Apache Azure Spark SQL Python **Key Responsibilities**: - Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL - Collaborate with...

This is a 12 months contract assigned to our client Work Location: To be confirmed Salary Range : $6,000-$8,000 **Job Description**: 1. Primary Skill: Data Analytics 2. Data Engineering 3. Hands on programming skills using Python, Spark, ETL pipeline related such as Kafka, airflow. Skills Requirement (Intermidiate Level) 1. Apache Kafka 2. Apache...

**Responsibilities**: - Design, implement, and maintain data pipelines for the migration of data from on-premises systems to Hadoop-based platforms. - Develop and optimize data processing jobs using Spark, Hive, and Python. - Manage job orchestration and scheduling using Control-M, ensuring timely and accurate data delivery. - Collaborate with...

Job Description - About Us - Roles and Responsibility **Job scopes**: - The Data Engineer will lead the enterprise-wide data transformations projects. The person will involve in developing and automating data processing pipelines for financial and Investment data modelling, analysis, and reporting from various data sources systems; The primary...

We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams...

We are seeking a skilled and motivated Data Engineer with expertise in Hadoop, Spark, OpenShift Container Platform (OCP), and DevOps practices. As a Data Engineer, you will be responsible for designing, developing, and maintaining efficient data pipelines, processing large-scale datasets. Your expertise in Hadoop, Spark, OCP, and DevOps will be crucial in...

We are looking Data Engineer with below skills Core Java - Fundamental knowledge of distributed architectures, SOA, Micro-services, PaaS - Good knowledge of data manipulation using distributed data processing systems such as Spark SQL - Required to have good SPARKS skills - Hands-on experience on HDFS (Hadoop), spark, impala, hive, as well as database...

We are seeking a highly skilled Data Engineer to join our team and contribute to building scalable, reliable, and high-performance data platforms. The ideal candidate will have strong expertise in Python, SQL, Spark , and hands-on experience with cloud-based big data platforms such as Databricks . Key Responsibilities: Design, develop, and maintain...