Data Engineer
6 days ago
Experience: 1 to 9 Years
Role: Data Engineer
Key Responsibilities:
- Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
- Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch
- Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions
- Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch
- Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration
- Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components
- Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation.
Key Requirements:
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field
- Proficiency in Scala programming language and familiarity with functional programming concepts
- In-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
- Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization
- Experience with distributed computing, parallel processing, and working with large datasets
- Familiarity with big data technologies such as Hadoop, Hive, and HDFS
- Strong problem-solving and analytical skills with the ability to debug and resolve complex issues
- Familiarity with version control systems (e.g., Git) and collaborative development workflows
- Excellent communication and teamwork skills with the ability to work effectively in cross-functional teams
-
Data Engineer(Cloud Migration)
6 days ago
Singapore NTT DATA Full timeRequirements / Qualifications - Bachelor's or master's degree in computer science, data engineering, or a related field. - Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC. - Proficiency in programming languages such as Python, Java, or Scala for building data pipelines. - Evaluate...
-
Data Engineer
6 days ago
Singapore DOZER DATA PTE. LTD. Full time**About the job**: This position is needed to build a highly scalable, reliable, and efficient data platform that makes it easy for users to get deep insights from vast amounts of distributed data. This role will be responsible for driving key technical decisions to completion for the organization and also guiding other engineers. **Responsibilities**: In...
-
Azure Data engineer
2 weeks ago
Singapore NTT Data Singapore Full timeJob Responsibilities: Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...
-
Ms20220804 Data Engineer
2 weeks ago
Singapore NTT DATA Singapore PTE Ltd Full timeJob Description - About Us - Roles and Responsibility **Job scopes**: - The Data Engineer will lead the enterprise-wide data transformations projects. The person will involve in developing and automating data processing pipelines for financial and Investment data modelling, analysis, and reporting from various data sources systems; The primary...
-
Data Engineer
2 weeks ago
Singapore TRIVI DATA CONSULTING PTE. LTD. Full timeKey Responsibilities - Design, implement, and maintain ETL workflows for managing large-scale datasets. - Develop cloud-based data pipelines (AWS, Azure) for scalable and real-time analytics delivery. - Optimize data models to ensure high performance and reliable reporting. - Create advanced dashboards and visualizations using tools like Qlik Sense and...
-
Senior Data Scientist
2 weeks ago
Singapore Data Science Full timeGet to know the team You'll be part of the SNP Data Science Personalization team, an established group dedicated to enhancing personalization experiences for Grab Food/Mart/Transport. We aim to develop cutting-edge Deep Learning models to deliver best user experiences. Our team comprises passionate Data Scientists with various expertise. If personalization...
-
Ts Implementation Engineer
2 weeks ago
Singapore NTT DATA Full time**Make an impact with NTT DATA** Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it’s a place where you can grow, belong and thrive. **Your day at NTT DATA** The...
-
Director
11 hours ago
Singapore GALA ENGINEER PTE. LTD. Full timeThe Operation Director is responsible for leading, guiding and directing all operational areas to ensure successful achievement of operational objectives. They must create operational plans that fully align with the organizations’ overall strategic plan, while maintaining a focus on operational effectiveness, efficiency, and customer satisfaction. This...
-
Data Business Analyst
2 weeks ago
Singapore NTT Data Singapore Full timeResponsibilities: - Installation of SQL Server / Oracle - Creation of new database - Database Instance Hardening, Patching on identification of Vulnerabilities - Configuration and Testing of Backup-Restore scripts - Configuration of AOAG (SQL Server) / DataGuard (Oracle) - Testing of AOAGroup/Instance Switchover to D.R. and Switchback to PROD - Migration of...
-
Senior Manager, Mechanical Engineering, Apac
11 hours ago
Singapore Vantage Data Centers Full timeAbout Vantage Data Centers Vantage is committed to being a workplace of inclusion, equity, respect and acceptance. We celebrate diversity and intentionally seek out opportunities to learn from one another’s experience. Vantage Data Centers powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud providers and...