Data Engineer
2 weeks ago
We are seeking a highly skilled Data Engineer to join our team at Unison Consulting Pte Ltd. As a Data Engineer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.
Key Responsibilities:- Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
- Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
- Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
- Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
- Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch
- Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch
- Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards
- Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration
- Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components
- Ensure data quality and integrity throughout the data processing lifecycle
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
- Optimize data engineering workflows for containerized deployment and efficient resource utilization
- Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability
- Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
- Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
- Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
- Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
- More than 5 years of experience as a Data Engineer
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related discipline
- Possession of Quantexa certification as a Data Engineer or Data Architect, with proficiency in the tool
- Demonstrated experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments
- Expertise in the Scala programming language and familiarity with functional programming principles
- Prior experience with the Quantexa tool is highly desirable
- Comprehensive understanding of Apache Spark architecture, including RDDs, DataFrames, and Spark SQL
- Advanced proficiency in designing and developing data infrastructure utilizing Hadoop, Spark, and associated tools (HDFS, Hive, Pig, etc.)
- Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration via Kubernetes
- Proficiency in programming languages commonly employed in data engineering, including Spark, Python, Scala, or Java
- Knowledge of DevOps methodologies, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
- Experience with Graphana, Prometheus, and Splunk will be considered an added advantage
- Background in integrating and utilizing Elasticsearch for data indexing and search applications
- Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization techniques
- Experience with distributed computing, parallel processing, and handling large datasets
- Proficient in performance tuning and optimization methods for Spark applications and Elasticsearch queries
- Strong problem-solving and analytical capabilities with the capacity to debug and resolve intricate issues
- Familiarity with version control systems (e.g., Git) and collaborative development environments
Unison Consulting Pte Ltd is an equal opportunities employer and welcomes applications from all qualified candidates. We offer a competitive salary and benefits package, as well as opportunities for professional growth and development.
-
Data Engineer
3 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob SummaryWe are seeking a highly skilled and experienced Big Data Engineer to lead our data engineering efforts. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environments, such as Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/Senior Consultant preferably.Key...
-
Data Engineer with Big Data Expertise
2 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob SummaryWe are seeking a highly skilled and experienced Big Data Engineer to lead our data engineering team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environments, such as Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/Senior Consultant preferably. The...
-
Data Center Solutions Engineer
7 days ago
Singapur, Singapore Vantage Data Centers Full timeAbout Vantage Data CentersWe're committed to creating a workplace of inclusion, equity, respect, and acceptance. Our diverse team celebrates individuality and strives to learn from each other's experiences.Job SummaryThe Customer Solutions Engineer plays a critical role in supporting the successful development and execution of technical solutions programs....
-
Data Engineer
1 month ago
Singapur, Singapore NTT Data Singapore Full timeJob SummaryWe are seeking a highly skilled and experienced Data Engineer to join our team at NTT Data Singapore. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environments, such as Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/Senior Consultant preferably.Key...
-
Data Center Solutions Engineer
3 weeks ago
Singapur, Singapore Vantage Data Centers Full timeAbout Vantage Data CentersWe are a workplace of inclusion, equity, respect, and acceptance, celebrating diversity and seeking opportunities to learn from each other's experiences.Job SummaryThe Customer Solutions Engineer will support the successful development and execution of a technical solutions program. This key sales solutions & engineering team member...
-
Reporting and Analytics Developer/Data Engineer
2 months ago
Singapur, Singapore NTT Data Singapore Full timeJob Summary We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. Overall experience can be 4 to 10 years. Consultant/ Senior Consultant preferably. The successful candidate will...
-
Data Engineer
1 month ago
Singapur, Singapore NTT Data Singapore Full timeWork Location: SingaporeContract Duration - 12 months renewable Responsibilities: Work closely with data stewards, data analysts and business end-users to implement and support data solutions. Design and build robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources using Python. Familiar with SDLC...
-
Data Scientist
3 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob Title: Data EngineerLocation: SingaporeContract Duration: 12 months renewableAbout the Role:We are seeking a highly skilled Data Engineer to join our team in Singapore. As a Data Engineer, you will be responsible for designing and building robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources...
-
Data Scientist
3 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob Title: Data EngineerLocation: SingaporeContract Duration: 12 months renewableAbout the Role:We are seeking a highly skilled Data Engineer to join our team in Singapore. As a Data Engineer, you will be responsible for designing and building robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources...
-
Data Scientist
2 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob Title: Data EngineerLocation: SingaporeContract Duration: 12 months renewableAbout the Role:We are seeking a highly skilled Data Engineer to join our team in Singapore. As a Data Engineer, you will be responsible for designing and building robust and scalable data ingestion and data management solutions for batch loading and streaming from data sources...
-
IT Operations Manager
3 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob Title: IT Operations Manager (Data Centre)We are seeking an experienced IT Operations Manager to lead our Data Centre operations team. The successful candidate will be responsible for ensuring the smooth operation of our data centre infrastructure, including servers, storage, network equipment, and power systems.Key Responsibilities:Leadership and Team...
-
Data Centre Operations Manager
4 weeks ago
Singapur, Singapore NTT Data Singapore Full timeJob Title: Data Centre Operations ManagerWe are seeking a highly skilled Data Centre Operations Manager to join our team at NTT Data Singapore. As a key member of our IT operations team, you will be responsible for ensuring the smooth operation of our data centre facilities.Key Responsibilities:Leadership and Team Management: Lead a team of support...
-
Data Center Solutions Expert, APAC
1 week ago
Singapur, Singapore Vantage Data Centers Full timeAbout Vantage Data CentersVantage Data Centers is a leading provider of data center services, committed to delivering high-quality infrastructure solutions to hyperscalers, cloud providers, and large enterprises. Our mission is to power, cool, protect, and connect the technology of the world's most innovative companies.Job SummaryWe are seeking a highly...
-
IT Operations Manager(Data Centre)
1 month ago
Singapur, Singapore NTT Data Singapore Full timeSpearheaded daily IT Infra and DC operations ensuring system functionality, availability, data security, and compliance with industry standard.Lead a team of support engineers, assign R&R, performance review and roster arrangement to ensure 24x7 operations coverage.Conduct routine site inspection to assess the condition of the facilities, equipment, and...
-
Software Development Engineer
3 weeks ago
Singapur, Singapore NTT DATA Full timeAbout the RoleNTT DATA is seeking a talented Associate Software Application Development Engineer to join our team. As a key member of our development team, you will work on complex projects, collaborating with senior engineers to deliver high-quality software solutions.Key ResponsibilitiesDevelop and maintain software applications using.NET C#, Java, or...
-
Software Development Engineer
3 days ago
Singapur, Singapore NTT DATA Full timeAbout the RoleNTT DATA is seeking a talented Associate Software Application Development Engineer to join our team. As a key member of our development team, you will work on complex projects, collaborating with senior engineers to deliver high-quality software solutions.Key ResponsibilitiesDevelop and maintain software applications using.NET C#, Java, or...
-
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Senior Principal Enterprise Architect
1 month ago
Singapur, Singapore NTT DATA Full timeAbout the RoleWe are seeking a highly skilled Senior Principal Enterprise Architect to join our team at NTT DATA. As a key member of our organization, you will play a critical role in driving business growth and innovation through the development and implementation of strategic enterprise architecture solutions.Key ResponsibilitiesSupport the client manager...
-
Network Engineer
3 weeks ago
Singapur, Singapore NTT DATA Full timeAbout the RoleNTT DATA is seeking a highly skilled Network Service Delivery Field Support Engineer to join our team. As a key member of our technical support team, you will be responsible for providing expert-level technical support to our clients, ensuring the smooth operation of their network infrastructure.Key ResponsibilitiesProvide third-line technical...
-
Software Development Engineer
3 days ago
Singapur, Singapore NTT DATA Full timeAbout the RoleNTT DATA is seeking a talented Software Application Development Engineer to join our team. As a key member of our development team, you will be responsible for working on complex projects, writing and debugging code, and collaborating with senior members to ensure the successful delivery of applications.Key ResponsibilitiesDevelop and maintain...
