Big Data Developer
3 weeks ago
Our client is looking for an experienced and highly skilled Senior Big Data Developer to join the dynamic team. The ideal candidate will have over 10 years of extensive working knowledge and hands-on expertise in big data technologies, especially in Spark on Scala or PySpark, and Hive. This role will involve the design, development, and performance tuning of Spark applications while working with cutting-edge big data tools and distributed systems.
As a key player, you will work closely with our data engineering team to provide robust, scalable solutions for data processing and analytics.
Responsibilities:
- Spark Expertise: Design, develop, and optimise Spark applications using Scala, PySpark, and Hive.
- Big Data Engineering: Work with big data tools and techniques, leveraging the Hadoop ecosystem for processing large datasets.
- Programming: Utilize strong programming skills in Java, Scala, or Python to build high-performance applications.
- Distributed Systems: Develop a deep understanding of distributed systems and ensure system efficiency.
- Database Management: Work with RDMS, Data Warehouses, and Unix Shell scripting for end-to-end data management and optimization.
- Automation: Implement workflows using Airflow and Ctrl-M for scheduling and monitoring.
- Collaboration: Work with cross-functional teams to ensure seamless data pipeline integrations.
Qualifications:
- 10+ years of experience working with Spark on Scala or PySpark and Hive.
- Strong expertise in Java, Scala, or Python programming.
- Proficient in the Hadoop ecosystem (HDFS, YARN, MapReduce, HBase, etc.).
- Solid understanding of distributed systems and their performance considerations.
- Familiarity with RDMS, Data Warehouses, and Unix Shell scripting.
- Experience with workflow management tools like Airflow and Ctrl-M.
- Strong analytical and problem-solving skills.
- Experience in performance tuning for Spark applications.
- Familiarity with cloud-based data platforms.
- Experience in a fast-paced, agile development environment.
EA REG:R1108137
EA LIC:95C5411
Tell employers what skills you have
PySpark
Airflow
Scala
Big Data
Hadoop
Workflow Management
Data Engineering
Spark
Distributed Systems
Python
Performance Tuning
Unix Shell Scripting
Java
Python Programming
-
Senior Big Data Engineer
4 weeks ago
Singapore NTT Data Singapore Full timeRequirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Proficiency in distributed computing principles and strong experience with Hadoop,...
-
Senior Big Data Engineer
3 weeks ago
Singapore NTT Data Singapore Full timeJob Requirements We are seeking a highly skilled Data Engineer to join our team at NTT Data Singapore. The ideal candidate will have a Bachelor's degree in Computer Science or a related field and at least 7 years of IT experience, with a minimum of 5 years in a similar role. Key Responsibilities • Proficient understanding of distributed computing...
-
Senior Big Data Developer
2 weeks ago
Singapore Helius Full timeTalend Big Data Developer RoleAt Helius, we are seeking a skilled Big Data Developer to join our team. The ideal candidate will have experience with Talend Big Data version 7+ and Hadoop, with a strong understanding of data models and data flows.Key Responsibilities:Design and develop production-grade Talend Big Data jobs with consideration for IT...
-
Big Data Development Engineer
2 days ago
Singapore TikTok Full timeAbout the teamByteDance/TikTok's Tea analytics platform is a leading in-house data platform used by over 10,000 employees globally. As a key member of our global engineering team, you will be responsible for building and maintaining the platform that supports exabyte-level data, trillion-level events, and millisecond-level response...
-
Big Data Solutions Developer
2 days ago
Singapore Nityo Infotech Full timeAs a key member of our data engineering team at Nityo Infotech, you will play a crucial role in designing, building, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.The ideal candidate will have a solid understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL, as well as experience integrating and working...
-
Big Data Engineer
7 days ago
Singapore SOFTENGER (SINGAPORE) PTE. LTD. Full timeRoles & ResponsibilitiesRole: Contract (1 year, Renewable)Position: Big Data Support EngineerMandatory Skills: Big Data, Data pipeline, Apache and HadoopExperience: 6 - 8 yrsPOSITION GENERAL DUTIES AND TASKS:- Strong in data engineering concepts and programming- Experience in building and optimizing ‘big data’ data pipelines, architectures and data...
-
Scala Developer with Big Data Expertise
6 days ago
Singapore KRIS INFOTECH PTE. LTD. Full timeAt KRIS INFOTECH PTE. LTD., we are seeking an experienced Scala developer to join our team.Job SummaryWe are looking for a highly skilled Scala developer with expertise in big data processing using Spark and Hadoop.About the RoleThis is a challenging opportunity to work on large-scale data processing projects using Scala, PySpark, and Hive.We require someone...
-
Big Data Solutions Lead
1 day ago
Singapore Citi Full timeCiti is seeking a highly experienced Senior Big Data Solutions Lead to join our Applications Development team. As a key member of our technology organization, you will play a critical role in driving the development and implementation of innovative big data solutions.About the RoleThis position is responsible for leading the design, development, and...
-
Senior Big Data Engineer
1 day ago
Singapore ITCAN Full timeAbout ITCANWe are a leading technology company that delivers innovative solutions to our clients.Job Title: Senior Big Data EngineerSalary Range:$170,000 - $220,000 per yearAbout the Role:We are seeking an experienced Senior Big Data Engineer to join our team. The ideal candidate will have extensive experience in designing and implementing large-scale big...
-
Big Data Framework Engineer
3 weeks ago
Singapore TANGSPAC CONSULTING PTE LTD Full timeRoles & ResponsibilitiesAs a Data Framework Engineer at TANGSPAC CONSULTING PTE LTD, you will be responsible for creating frameworks and technical features that enable faster operationalisation of data models, analytical models, and user-generated contents. This includes partnering with data scientists to enable faster adoption of AL/ML model-based systems,...
-
Big Data Engineering Professional
1 day ago
Singapore Virtusa Full timeWe are seeking an experienced Big Data Engineer to join our team at Virtusa in Moulmein-Kallang, Central Singapore. The successful candidate will have a strong background in big data engineering and experience working with Python, PySpark, Linux, and ETL tools like Informatica.ResponsibilitiesDesign and implement scalable big data solutions using Hive,...
-
Senior Big Data Architect
7 hours ago
Singapore Unison Consulting Pte Ltd Full timeAbout UsAt Unison Consulting Pte Ltd, we are a forward-thinking consulting firm that provides expert solutions to businesses in the digital age.Job Title: Senior Big Data ArchitectJob DescriptionWe are seeking an experienced Senior Big Data Architect to join our team. The ideal candidate will have a strong background in big data engineering and architecture,...
-
Big Data Integration Specialist
1 day ago
Singapore Helius Full timeJob DescriptionAbout the RoleWe are seeking a highly skilled Big Data Integration Specialist to join our team at Helius. As a key member of our data integration team, you will be responsible for designing and implementing production-grade Talend Big Data jobs that meet our IT organization's architecture standards.Key ResponsibilitiesDevelop and maintain...
-
Senior ETL Developer with Big Data Expertise
3 weeks ago
Singapore ELLIOTT MOSS CONSULTING PTE. LTD. Full timeSenior ETL Developer PositionWe are seeking a highly skilled Senior ETL Developer to join our team at ELLIOTT MOSS CONSULTING PTE. LTD. The ideal candidate will possess strong proficiency in ETL solutions such as Informatica BDM, experience with big data technologies like Cloudera, and advanced knowledge of programming languages such as Python, Scala, and...
-
Big Data Engineering Specialist
7 hours ago
Singapore Unison Consulting Pte Ltd Full timeAbout UsUnison Consulting Pte Ltd is a leading consulting firm that specializes in data engineering solutions.We are seeking a highly skilled Big Data Engineering Specialist to join our team.Job DescriptionJob Title: Big Data Engineering SpecialistEstimated Salary: $120,000 - $180,000 per annumAs a Big Data Engineering Specialist at Unison Consulting Pte...
-
Senior Big Data Solution Architect
4 weeks ago
Singapore Tencent Singapore Full timeResponsibilities: Cloud & Smart Industries Group (CSIG) is responsible for promoting the company's cloud and industry Internet strategy. CSIG explores the interactions between users and industries to create innovative solutions for smart industries via technological advancements such as cloud, AI, and network security. While driving the digitalization of...
-
Senior Big Data Architect
6 days ago
Singapore WEB SYNERGIES (S) PTE LTD Full timeAbout WEB SYNERGIES (S) PTE LTDWe are a dynamic company seeking an experienced Big Data Architect to join our team.Job Summary:As a seasoned professional, you will be responsible for designing and implementing large-scale data architectures using Big Data technologies such as Hadoop and Spark. You will work closely with cross-functional teams to gather,...
-
Big Data Engineering Expert
1 day ago
Singapore Unison Consulting Pte Ltd Full timeUnison Consulting Pte Ltd is seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have 5+ years of experience in designing, developing, and implementing Spark Scala applications and data processing pipelines.About the RoleThis is a key role that requires expertise in integrating Elasticsearch with Spark to enable efficient...
-
Senior Big Data Solution Architect
4 weeks ago
Singapore Tencent Singapore Full timeJob Responsibilities:As a Senior Big Data Solution Architect at Tencent Singapore, you will be responsible for designing and implementing Big Data solutions tailored to customer business scenarios and technical requirements. You will need to deeply understand customers' needs and pain points regarding Big Data products and solutions.Recommend the core...
-
Data Engineer for Big Data Migration
1 day ago
Singapore U3 Full timeAbout the RoleWe are seeking an experienced Data Engineer to join our team at U3 Infotech. As a key member of our data engineering team, you will be responsible for designing and implementing scalable data pipelines and data services modules with robustness in mind to support the growing demands from business users.Key ResponsibilitiesDesign and implement...