Senior Big Data Engineer
2 months ago
We are seeking a highly skilled Spark Scala Developer to collaborate with our dynamic team in delivering cutting-edge data processing solutions. The ideal candidate will possess a deep understanding of big data technologies, particularly Apache Spark, and expertise in Scala programming.
Key Responsibilities- Design and Development: Implement high-performance data processing pipelines using Apache Spark and Scala, ensuring data integrity, performance, and scalability.
- Optimization: Enhance the performance of existing data processing jobs by tuning Spark jobs, improving data storage, and optimizing the overall data flow.
- Data Integration: Work with various data sources, including structured, semi-structured, and unstructured data, to integrate them into the data processing framework.
- Collaboration: Collaborate with data engineers, data scientists, and other stakeholders to define requirements, refine processes, and ensure successful project delivery.
- Testing: Develop unit tests and integration tests to ensure the reliability and accuracy of the data processing pipelines.
- 5+ Years of experience as a Spark Scala Developer.
- Minimum 5 Years of experience in Scala Programming.
- Proficient in Scala, with expertise in functional programming paradigms.
- Strong experience in Apache Spark and Scala programming.
- Familiarity with Hadoop, HDFS, Hive, or similar big data technologies.
- Experience with ETL processes, data integration, and handling large-scale datasets.
- Understanding of data warehousing concepts and principles.
- Proficiency in SQL and data query optimization.
- Experience with real-time data processing using Kafka or similar technologies is a plus.
- Experience with version control systems like Git.
- Familiarity with CI/CD pipelines and tools like Jenkins or similar.
- Mandatory Skills: Minimum 5+ Years in Scala.
-
Big Data Engineer
6 months ago
Singapur, Singapore Virtusa Full timeBig Data Engineer - 123101 Description Responsibilities Good understanding and completion of projects using waterfall/Agile methodology. • Good understanding of analytics and data warehouse implementations. • Ability to troubleshoot complex issues ranging from system resource to application stack traces. • Strong SQL, data modelling and data...
-
Big Data Development Engineer
2 months ago
Singapur, Singapore TikTok Full timeAbout the TeamTikTok's Tea platform is a cutting-edge analytics platform used by over 10,000 employees globally. As a key member of our global engineering team, you will be responsible for deploying, configuring, and maintaining big data platforms, as well as implementing and optimizing infrastructure for big data storage, processing, and computation.**Key...
-
Senior Big Data Test Development Engineer
1 month ago
Singapur, Singapore PatSnap Full timeAbout PatSnap Patsnap empowers IP and R&D teams by providing better answers, so they can makefaster decisions with more confidence. Founded in 2007, Patsnap is the global leaderin AI-powered IP and R&D intelligence. Our domain-specific LLM, trained on ourextensive proprietary innovation data, coupled with Hiro, our AI assistant, deliversactionable insights...
-
Software Engineer
4 months ago
Singapur, Singapore Epam Full timeDescription We are looking for a Big Data Engineer to join our team! The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environments e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing jobs as...
-
Big Data Development Engineer
6 months ago
Singapur, Singapore TikTok Full timeAbout the team :Tea(Toutiao Event Analyze) is an in-house analytics platform that is heavily used by more than 10,000 employees globally. This is a data middle platform product built on user behavior data, providing data analysis services for various domestic and international businesses within ByteDance/TikTok, including Douyin, Toutiao, Xigua, TikTok,...
-
Senior Data Engineer
4 weeks ago
Singapur, Singapore Rapsys Technologies Pte Ltd. Full timeSenior Data EngineerExperience Required: 5-10 YearsPosition Summary:We are seeking an experienced Senior Data Engineer with 5 to 10 years of hands-on experience in building and managing data pipelines and big data infrastructure. The ideal candidate will have a strong technical background in data engineering tools, big data ecosystems, and programming...
-
Singapur, Singapore Careers@Gov Full time[What the role is] Senior/Lecturer (Big Data and Analytics) /School of Informatics & IT [What you will be working on] Conduct lessons and carry out all relevant responsibilities related to the teaching role in full-time and part-time course offerings in areas related to Data Analytics, Machine Learning, Data Engineering and Data Management, and other...
-
Senior/Lecturer/ School of Informatics
4 months ago
Singapur, Singapore Careers@Gov Full time[What the role is] Senior/Lecturer/ School of Informatics & IT (Big Data and Analytics) [What you will be working on] Conduct lessons and carry out all relevant responsibilities related to the teaching role in full-time and part-time course offerings in areas related to Data Analytics, Machine Learning, Data Engineering and Data Management, and other...
-
Singapur, Singapore Careers@Gov Full time[What the role is] Senior/Lecturer (Big Data and Analytics) /School of Informatics & IT [What you will be working on] Conduct lessons and carry out all relevant responsibilities related to the teaching role in full-time and part-time course offerings in areas related to Data Analytics, Machine Learning, Data Engineering and Data Management, and other...
-
Senior Data Engineer
2 weeks ago
Singapur, Singapore Centre for Strategic Infocomm Technologies Full timeRoleAs a senior data engineer in a team that is responsible for mission-critical data analytics systems, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. You will be involved in complex technical projects to collect, parse and process large datasets with the aim to turn them...
-
Big Data Engineer
4 months ago
Singapur, Singapore TikTok Full timeTeam Introduction We are an incredible team with passion. We enjoy learning new things and taking on challenges. Team culture here is open and inclusive. Everyone can make things happen, good ideas will always win.Today, we are continuously increasing investment in AI technology to make content creation much easier for everyone, and we always take the...
-
Senior Data Engineer
6 months ago
Singapur, Singapore HKT Full timeThis is a Full-time position with a Hybrid working arrangement (1-2 days in office). Role Overview We are passionate about harnessing the power of data to drive innovation and empower businesses. We are seeking a highly technical and experienced Senior Data Engineer to join our dynamic team. The successful candidate will be a key player in...
-
Java Developer with
5 months ago
Singapur, Singapore Unison Consulting Pte Ltd Full timeWe are seeking an experienced Java Developer with expertise in Google Cloud Platform (GCP) and Big Data technologies. The ideal candidate will have a strong background in Java development, cloud computing, and data engineering to help us build and maintain scalable, high-performance systems.Key Responsibilities: Design, develop, and maintain Java-based...
-
Senior Data Engineer
2 months ago
Singapur, Singapore SAP Full timeData Engineering Role at SAPSAP is a market leader in end-to-end business application software and related services. We are looking for a Senior Data Engineer to join our team and help us design, build, and operate our data pipelines and data storage aspects of our solutions.Key Responsibilities:Design and implement data pipelines to extract, transform, and...
-
Senior Associate Software Engineer
1 month ago
Singapur, Singapore NTT DATA Full timeJob Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...
-
Data Engineer
2 months ago
Singapur, Singapore Epam Full timeJob SummaryWe are seeking a highly skilled Data Engineer to design, develop, and maintain data ingestion and processing jobs in our big data environment. The ideal candidate will have a strong understanding of analytics, data warehousing, and data modeling, as well as experience with big data technologies such as Python, PySpark, and ETL tools like...
-
Data Engineer
4 months ago
Singapur, Singapore Centre for Strategic Infocomm Technologies Full timeAbout this roleWe are seeking a Data Engineer to join our Data & Systems Engineering team. In this role, you will be responsible for enhancing and optimizing the data flow between our systems and refining our data pipeline architecture. You will collaborate closely with cross-functional teams to gather requirements, design scalable solutions, and implement...
-
Data Engineer
5 months ago
Singapur, Singapore Unison Consulting Pte Ltd Full timeWe are seeking a skilled and motivated Data Engineer with expertise in Google Cloud Platform tools such as BigQuery, Composer, Dataproc, and ETL processes. The ideal candidate will have a strong background in designing, implementing, and optimizing data pipelines and workflows. You will play a key role in ensuring our data infrastructure is scalable,...
-
Singapur, Singapore United Overseas Bank Full timeVP, Data Scientist, Big Data Analytics Centre, Data Management Office Posting Date: 18-May-2023 Location: Singapore (City Area), Singapore, 048624 Company: United Overseas Bank Ltd About UOB United Overseas Bank Limited (UOB) is a leading bank in Asia with a global network of more than 500 branches and offices in 19 countries and...
-
Big Data Platform Architect
2 months ago
Singapur, Singapore U3 Full timeJob SummaryThe Enterprise Data Solutions Architect is responsible for designing and implementing scalable data processing architectures in hybrid cloud environments. This includes defining architecture and strategy for batch workloads, enabling multitenancy, and capacity planning at enterprise scale.Key ResponsibilitiesDesign and develop batch data...