Lead Data Engineer

2 days ago

Singapore ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. Full time

**Your Role**:
As a **Lead Data Engineer**, you will spearhead the development and optimization of our data infrastructure to enable advanced analytics capabilities. Collaborating with cross-functional teams, you will ensure seamless integration and delivery of robust, scalable, and efficient data solutions. In this leadership role, you will mentor and guide a team of engineers while contributing to the strategic direction of the organization’s data landscape.

**Key Responsibilities**:

- **Data Engineering & Development**:Design and implement large-scale data processing pipelines using **Spark Scala**, handling structured and unstructured data.
Integrate **Elasticsearch**with Spark for efficient indexing, querying, and data retrieval.
- **Containerization & Orchestration**:Deploy data solutions on **OpenShift Container Platform (OCP)**using containerization and orchestration techniques.
Optimize workflows for containerized environments and ensure efficient resource utilization.
- **DevOps Integration**:Collaborate with DevOps teams to streamline deployment using **CI/CD pipelines**and tools such as **Jenkins**, **Bitbucket**, and **Ansible**.
Implement robust monitoring and logging frameworks using tools like **Grafana**, **Prometheus**, and **Splunk**to maintain platform stability.
- **Collaboration & Stakeholder Management**:Work with data scientists, analysts, and infrastructure engineers to understand requirements and translate them into technical solutions.
Communicate technical designs and processes effectively to both technical and non-technical stakeholders.
- **Leadership & Mentorship**:Lead and mentor a team of data engineers, fostering a culture of continuous learning and innovation.
Advocate for best practices in data engineering, DevOps, and cloud technologies.
- **Documentation & Research**:Document workflows, pipelines, and configurations to ensure team knowledge sharing.
Stay abreast of emerging technologies and best practices in data engineering and cloud infrastructure.

**Requirements**:

- **Education**:Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
- **Experience**:Minimum of 10 years in data engineering roles with expertise in **Hadoop**, **Spark**, and related tools (e.g., **HDFS**, **Hive**, **Ranger**).
Hands-on experience with **OpenShift Container Platform (OCP)**and container orchestration tools like **Kubernetes**.
Proven track record of building and scaling data infrastructure in large, complex environments.
- **Technical Skills**:Proficiency in programming languages such as **Scala**, **Python**, or **Java**.
Deep understanding of **DevOps**practices, including CI/CD pipelines and automation tools like **Docker**, **Jenkins**, and **Ansible**.
Familiarity with monitoring and logging tools such as **Grafana**, **Prometheus**, and **Splunk**.
Exposure to **cloud platforms**(AWS, Azure, GCP) and their data services is a plus.
- **Soft Skills**: Exceptional problem-solving abilities and a proactive approach to resolving technical challenges.
Strong communication and collaboration skills to work effectively across diverse teams.
Ability to balance multiple priorities and deliver high-quality results in a dynamic environment.

**Preferred Qualifications**:

- Experience in cloud-native data services (e.g., AWS Redshift, Azure Data Factory, GCP BigQuery).
- Knowledge of advanced analytics and machine learning pipelines.

  • Lead Mechanical Engineer

    4 days ago

    Singapore beBeeMECHANICAL ENGINEER Full time $150,000 - $200,000

    Lead Mechanical Engineer PositionAbout the Role:We are seeking an experienced and skilled Lead Mechanical Engineer to join our team. The ideal candidate will have a strong background in mechanical engineering, excellent project management skills, and the ability to lead a team of engineers.Key Responsibilities:Review contract documents and make necessary...

  • Lead Data Scientist

    5 days ago

    Singapore Data Analytics Full time

    Get to Know the Team At Grab, the Market Insights team drives impactful data-driven decisions by uncovering insights about the region's users, merchants, and driver partners. By analysing vast datasets across transactional and behavioural activities, we bridge the gap between raw data science capabilities and actionable business strategies. Our work is...

  • Data Engineer

    5 hours ago

    Singapore NTT DATA, Inc. Full time

    Join to apply for the Data Engineer role at NTT DATA, Inc. . Overview As part of our data-driven transformation initiatives, we are looking for a motivated Associate Data Engineer to join our team. This role is ideal for early-career professionals who are passionate about building scalable data pipelines, integrating diverse data sources, and enabling...

  • Data Engineer(Cloud Migration)

    1 week ago

    Singapore NTT DATA Full time

    Requirements / Qualifications - Bachelor's or master's degree in computer science, data engineering, or a related field. - Minimum 7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC. - Proficiency in programming languages such as Python, Java, or Scala for building data pipelines. - Evaluate...

  • Data Engineer

    1 week ago

    Singapore DOZER DATA PTE. LTD. Full time

    **About the job**: This position is needed to build a highly scalable, reliable, and efficient data platform that makes it easy for users to get deep insights from vast amounts of distributed data. This role will be responsible for driving key technical decisions to completion for the organization and also guiding other engineers. **Responsibilities**: In...

  • Lead Data Engineer

    1 week ago

    Singapore UARROW PTE. LTD. Full time

    Key Responsibilities: - Lead the design and implementation of large-scale data processing systems using Spark and Scala. - Architect, develop, and optimize data pipelines for efficient data transformation, aggregation, and processing of large datasets. - Work with cross-functional teams to define data engineering solutions that meet business and operational...

  • Azure Data engineer

    7 days ago

    Singapore NTT Data Singapore Full time $90,000 - $120,000 per year

    Job Responsibilities:Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Azure Data engineer

    3 days ago

    Singapore NTT Data Singapore Full time

    Job Responsibilities: Data Analysis and Interpretation: Perform in-depth analysis of structured and unstructured data sets to identify trends, patterns, and valuable insights that drive business decisions and strategies. Data Visualization and Reporting: Develop intuitive and visually appealing dashboards, reports, and presentations to communicate complex...

  • Data Engineering Lead

    3 days ago

    Singapore beBeeDataEngineering Full time $80,000 - $120,000

    Job Title: Data Engineering LeadWe are seeking a seasoned data engineering professional to join our team as a Data Engineering Lead. In this role, you will be responsible for designing, developing, and maintaining large-scale data systems that support various business applications.Key Responsibilities:Collaborate with cross-functional teams to identify data...

  • Data Engineering Lead

    1 week ago

    Singapore R Systems (Singapore) Pte Ltd Full time

    At least 3 full end to end project delivery experience in last 5+ years as Senior Data engineer as mentioned in the Qualifications and Job descriptions section above. - Have lead, at least 2 full projects as senior or lead data engineer - Familiarity with agile and DevOps principles, test-driven development, continuous integration, and other Software...