Data Engineering Lead

18 hours ago

Novena, Singapore Accolite Digital Full time

**Your Profile**

We are seeking a skilled and motivated Lead Data Engineer with expertise in Hadoop, Spark, OpenShift Container Platform (OCP), and DevOps practices. As a Data Engineer, you will be responsible for designing, developing, and maintaining efficient data pipelines, processing large-scale datasets. Your expertise in Hadoop, Spark, OCP, and DevOps will be crucial in ensuring the availability, scalability, and reliability of our ML Solutions.

**Responsibilities**:

- Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives
- Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions
- Ensure data quality and integrity throughout the data processing lifecycle
- Design and deploy data engineering solutions on OpenShift Container Platform (OCP) using containerization and orchestration techniques
- Optimize data engineering workflows for containerized deployment and efficient resource utilization
- Collaborate with DevOps teams to streamline deployment processes, implement CI/CD pipelines, and ensure platform stability
- Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
- Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
- Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
- Document data engineering processes, workflows, and infrastructure configurations for knowledge sharing and reference
- Stay updated with emerging technologies, industry trends, and best practices in data engineering and DevOps
- Provide technical leadership, mentorship, and guidance to junior team members to foster a culture of continuous learning and innovation to the continuous improvement of the analytics capabilities within the bank

**Requirements**:

- Bachelor's degree in Computer Science, Information Technology, or a related field
- Proven experience as a Data Engineer, working with Hadoop, Spark, and data processing technologies in large-scale environments
- Strong expertise in designing and developing data infrastructure using Hadoop, Spark, and related tools (HDFS, Hive, Pig, etc)
- Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration using Kubernetes
- Proficiency in programming languages commonly used in data engineering, such as Spark, Python, Scala, or Java
- Knowledge of DevOps practices, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
- Experience with Graphana, Prometheus, Splunk will be an added benefit
- Strong problem-solving and troubleshooting skills with a proactive approach to resolving technical challenges
- Excellent collaboration and communication skills to work effectively with cross-functional teams
- Ability to manage multiple priorities, meet deadlines, and deliver high-quality results in a fast-paced environment
- Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services is a plus

**Skills**
*

Pay: $8,000.00 - $18,000.00 per month

**Benefits**:

- Dental insurance
- Health insurance

Schedule:

- Monday to Friday

**Experience**:

- Scala: 1 year (required)
- java: 1 year (required)
- Spark: 1 year (required)

Ability to Commute:

- Douglas, MI 49406 (required)

Work Location: In person

  • Data Engineer

    3 days ago

    Novena, Singapore Incube8 Pte Ltd Full time

    Incube8 is a rapidly growing technology company with offices in Singapore (Software Engineers, QAs, DevOps) and United States (Marketing, Product, Data & Analytics, IT, Customer Support). We are seeking an action-oriented, goal-oriented, go-getter Data Engineer as part of the Data Science team for our data driven product development team in Singapore. As a...

  • Technical Lead

    2 weeks ago

    Novena, Singapore Blue Ocean Systems Infotech Pvt. Ltd. Full time

    Hi Urgent opening Role: Technical Lead Key Responsibilities: Lead and oversee data engineering initiatives to ensure high-quality data solutions.Work independently with mínimal supervision while being proactive and responsive .Collaborate with stakeholders in the Greater China region to gather requirements and facilitate design playback .Implement best...

  • Lead Machine Learning Engineer

    2 weeks ago

    Novena, Singapore Accolite Digital Full time

    Who we are Enterprise Analytics (EA) stands at the forefront of UOB's vision, aiming to establish the bank as a leader in deploying cutting-edge Machine Learning models. Comprising a team of experts, we focus on enhancing the bank's capabilities in supporting key areas such as Risk, Wealth, Retail, etc., with a strong emphasis on coordination in data...

  • Project Officer

    4 days ago

    Novena, Singapore Nanyang Technological University Full time

    **Key Responsibilities: **Job Requirements**: - Degree (MS or BS) in Computer Science, Computer Engineering, Electrical/Electronic Engineering, Mechanical Engineering, engineering, or equivalent industry experience. - Python programming - Experiences in machine learning for image processing with PyTorch - Experiences in articulated human modelling or...

  • Database Administrator

    2 weeks ago

    Novena, Singapore ST Engineering Full time $60,000 - $100,000 per year

    Job ID: Location: ST Engineering Jurong East Bui, SG- Description: - **Responsibilities** - Managing, monitoring and maintaining databases - Maintaining database backup and recovery - Provide solution related to database design and SQL writing to Application Developers - Implement change requests related to development in database such as table design,...

  • Research Fellow

    5 days ago

    Novena, Singapore Nanyang Technological University Full time

    Research project: Connecting within-host viral dynamics with the epidemiology of COVID-19: a multiscale computational infrastructure In this project, we aim to develop a multiscale modeling framework and computational infrastructure to connect within-host viral dynamics and between-host transmission dynamics. The multiscale model will help address key...

  • Vp Engineering

    1 week ago

    Novena, Singapore SWAT MOBILITY PTE. LTD. Full time

    8 years exp **Roles & Responsibilities** Join the ride at SWAT Mobility - If you're up for a huge challenge with one of the most exciting startups in the field of smart mobility! Our mission is to organize the movement of cities. We believe that technology will change how we move, and that our Core Mobility Engine has the power to fuel the transformation...

  • Temporary Research Assistant, Health Data Science

    4 days ago

    Novena, Singapore Nanyang Technological University Full time

    Research Assistant is required for concurrent research projects in the laboratory of Assistant Professor Lim Jue Tao. Roles & Responsibilities - undertaking literature reviews, - leading on analysing data using statistical and data science techniques, - leading on writing reports, presentations, and publication of results and findings in peer-reviewed...

  • Research Assistant/ Associate

    2 weeks ago

    Novena, Singapore Nanyang Technological University Full time

    Research Assistant/Research Associate is required for concurrent research projects in the laboratory of Assistant Professor Lim Jue Tao. Job responsibilities: - Undertaking literature reviews,- Leading on analysing data using statistical and data science techniques,- Leading on writing reports, presentations, and publication of results and findings in...

  • Research Assistant/associate, Health Data Science

    18 hours ago

    Novena, Singapore Nanyang Technological University Full time

    Research Assistant/Research Associate is required for concurrent research projects in the laboratory of Assistant Professor Lim Jue Tao. Job responsibilities: - Undertaking literature reviews,- Leading on analysing data using statistical and data science techniques,- Leading on writing reports, presentations, and publication of results and findings in...