Big Data Engineering Lead
1 day ago
We're looking for a seasoned Big Data Engineering Lead with expertise in Scala, Python, and PySpark to lead our client data engineering team. You'll be responsible for designing and implementing scalable, efficient, and fault-tolerant data pipelines, as well as mentoring team members and driving technical innovation.
**Key Responsibilities**:
- Design and develop large-scale data pipelines using Scala, Python, and PySpark
- Lead and mentor a team of data engineers to build and maintain data architectures
- Collaborate with cross-functional teams to identify data requirements and implement data-driven solutions
- Ensure data quality, integrity, and security across all data pipelines
- Develop and maintain technical documentation for data pipelines and architectures
- Stay up-to-date with industry trends and emerging technologies in big data, cloud computing, and data engineering
- Drive technical innovation and recommend new tools and technologies to improve data engineering capabilities
**Requirements**:
- 5+ years of experience in big data engineering, with expertise in Scala, Python, and PySpark
- Strong experience with big data technologies such as Apache Spark, Hadoop, and Kafka
- Experience with cloud-based data platforms such as AWS, GCP, or Azure
- Strong understanding of data architecture, data governance, and data security
- Excellent leadership and mentoring skills, with experience leading high-performing teams
- Strong communication and collaboration skills, with ability to work with cross-functional teams
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
-
Lead Big Data Engineer
4 days ago
Singapore NodeFlair Full time**Job Summary**: **Salary** S$8,000 - S$11,000 / Monthly **Job Type** **Seniority** Lead **Years of Experience** At least 8 years **Tech Stacks** ETL MariaDB Docker Data Extraction play Shell Script Analytics Sqoop Dataiku Cloudera HDFS Shell Java Hive Airflow Spark Bitbucket SQL Scala Hadoop Python **Job Purpose** At Great Eastern, we are building a...
-
Application Development Lead
2 weeks ago
Singapore Tangspac Full timeA large financial institution with large big data team is looking for Application Development Manager / Lead with the following responsibilities: - Leading an applicaiton development team - Utilizing big data tools to perform data ingestion, interface and data analytics - Building interfaces to big data (Hadoop) data store - Integration of various tools as...
-
Big Data Engineer
3 days ago
Singapore THE EDGE PARTNERSHIP HOLDINGS PTE. LTD. Full time**Big Data Engineer - 1-year contract **Key Responsibilities** - **Programming**:Utilize Java, Scala, or Python for efficient coding and problem-solving. - **Big Data Tools**:Work with various big data processing tools and techniques, including Airflow and Ctrl-M. - **Database Knowledge**:Leverage experience with RDBMS, data warehouses, and Unix shell...
-
Big Data Engineer
1 week ago
Singapore ALMR CONSULTING PTE. LTD. Full time $100,000 - $150,000 per yearJob Summary:We are looking for an experienced Big Data Engineer with at least 5 years of experience in managing data pipelines and processing within Big Data environments (e.g. Cloudera Data Platform). The role involves designing, developing, and maintaining data ingestion and transformation jobs to support analytics and reporting needs.Key...
-
Big Data Engineer
1 week ago
Singapore ALMR Consulting Pte Ltd Full time $150,000 - $200,000 per yearRoles & ResponsibilitiesJob Summary:We are looking for an experienced Big Data Engineer with at least 5 years of experience in managing data pipelines and processing within Big Data environments (e.g. Cloudera Data Platform). The role involves designing, developing, and maintaining data ingestion and transformation jobs to support analytics and reporting...
-
Big Data Engineer
4 days ago
Singapore SAKSOFT PTE LIMITED Full timeExperience: 9+ Years Role: Big Data Engineer Skills from Primary Skill Cluster Spark, Big Data, Hive, Java, 5+yrs with hands-on on Java is Must Database knowledge and should be able to write the queries is Must. Handson on Unix Shell scripting is Must. Knowledge in Spark SQL (Good to have). Understand on Regulatory reporting(Good to have). Design...
-
Big Data Platform Engineer
6 days ago
Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time**SINGAPORE, SINGAPORE /** **SOFTWARE ENGINEERING - DATA PLATFORMS /** **FULL-TIME** - Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of...
-
Principal Big Data Engineer
4 days ago
Singapore QUESSCORP HOLDINGS PTE. LTD. Full time**Job Description**: - Design and implement scalable data pipelines to collect, process, and store large volumes of structured and unstructured data. - Ensure data quality, integrity, and consistency across various data sources. - Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that...
-
Senior Engineer
3 days ago
Singapore Experis Full time**Senior Engineer (Big Data)**: - Location- Singapore- Job reference- BBBH130672_1691631585- Salary- Negotiable- Consultant name - Canete Christopher Miguel Consultant contact no. - EA License No. - 02C3423 - Consultant Registration No. - R23112144 **Job Description**: Experience in designing, creating, and managing large-scale data processing...
-
Big Data Engineer
1 week ago
Singapore Baidu, Inc. Full timeBuild the company's big data warehouse system, including batch and stream data flow construction. In-depth understanding of business systems, understanding of project customer needs, design and implement big data systems that meet user needs, and ensure smooth project acceptance. Responsible for data integration and ETL architecture design and development....