Data Engineer(Spark Developer)

**JD for Spark Developer** **Summary**: This role will be responsible for designing, developing, and maintaining Applications using Apache Spark open-source framework. They work with different aspects of the Spark ecosystem, including Spark SQL, Data Frames, Datasets, and streaming. **Role**: - Help write analytics code, services and components in Java,...

**Key Responsibilities**: - Design, develop, and maintain robust, scalable data pipelines and ETL processes. - Optimize data storage, retrieval, and processing using SQL, Spark, and Hive. - Collaborate with cross-functional teams to gather requirements and translate them into technical solutions. - Implement data quality, validation, and monitoring...

**The Opportunity** Key Responsibilities - Working with the latest tools and techniques - Hands-on coding, usually in a pair programming environment - Working in highly collaborative teams and building quality code **Essential Skills & Prerequisites**: - Hand-on Development experience with Spark, ScalaSpark and Distributed computing. - 4 to 6...

Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data - Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch - Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into...

We are hiring Platform Engineer - Big Data (Hadoop / Spark / Hive) with below requirements; **Responsibilities** - Design, implement, and manage big data platforms built on Hadoop, Spark, and Hive. - Develop and optimize data ingestion, transformation, and storage frameworks for high-volume, high-velocity datasets. - Maintain and fine-tune Hadoop clusters...

A global consultancy firm in Singapore is seeking a Lead Data Engineer to design and implement scalable data architectures. The ideal candidate will have 8+ years of experience, with in-depth knowledge of Apache Spark and Databricks, driving data solutions for innovative projects. This role offers a collaborative environment, focusing on data-driven...

**Key Responsibilities**: - Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL. - Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver appropriate data solutions. - Develop and optimize ETL (Extract, Transform, Load) processes to ensure data...

Design, implement, and manage big data platforms built on Hadoop, Spark, and Hive. - Develop and optimize data ingestion, transformation, and storage frameworks for high-volume, high-velocity datasets. - Maintain and fine-tune Hadoop clusters (HDFS, YARN) including capacity planning, monitoring, and troubleshooting. - Build and support Hive-based data...

Data Engineer This is a 12 months contract assinged to our client Work Location: To be confirmed Salary Range : $6,000-$9,500 **Job Description**: 1. Data Engineer with hands on programming skills using Python, Spark, ETL pipeline related such as Kafka, airflow. 2. Experience on Data Warehouse Tools Skillsets Requirement 1. Data Engineering 2. Data...

A leading professional services firm in Singapore seeks a Senior Data Engineer to design and implement data pipelines and databases. You will collaborate with experienced teams to deliver data-driven solutions using modern technologies. The ideal candidate has over 6 years in data engineering, with specific experience in Apache Spark and Databricks. This...