AI Data Engineer

Role: Data Engineer
Salary Range: S$6000 – S$8000
Expected Years of Experience: 3 to 8 years
Duration: 1 Year Contract
Key Responsibilities:
Data Solution Design & Implementation: Design, develop, and implement scalable data solutions tailored for large-scale AI/ML projects and complex data engineering initiatives.
AI/ML Data Pipelines: Build, maintain, and optimize robust data pipelines and ETL frameworks using technologies like Spark (Java/Scala/PySpark). These pipelines will be crucial for operationalizing data that integrates with machine learning models and data lakes/data marts.
Performance and Troubleshooting: Proactively identify and resolve system performance issues and data processing bottlenecks in distributed data environments, ensuring data is available for AI/ML model consumption.
Engineering Best Practices: Champion best practices for code quality, data reliability, and workflow orchestration. This includes implementing automation strategies to improve efficiency and maintain the integrity of data throughout the AI/ML lifecycle.
Technical Leadership & Collaboration: Offer technical guidance and mentorship to team members. Actively participate in code reviews and design discussions to ensure solutions meet the specific needs of data scientists and machine learning engineers.
Key Requirements:
AI/ML Data Pipelines: Experience in designing, building, and maintaining end-to-end feature engineering pipelines that operationalize data for AI/ML model training and real-time inference.
Scalable Data Ingestion: Proven experience in building scalable data pipelines and ingestion frameworks using languages like PySpark, Scala, and Java to handle large-scale datasets for ML applications.
Cloud-Native & Distributed Systems: Hands-on expertise with cloud-native data ecosystems on platforms like AWS, Azure, or GCP. This includes experience with modern data lake and data warehousing solutions, as well as distributed storage like HDFS.
Programming & Automation: Strong programming skills in Python, Java, and Scala with deep expertise in Spark for processing and transforming structured and semi-structured data for AI/ML.
Data Architecture: Deep understanding of modern data lake, data warehouse, and feature store architectures. Ability to design performant data models and serving layers optimized for AI/ML model consumption.
Performance & MLOps: Experience with Spark performance tuning and job optimization for large-scale distributed data processing. Familiarity with MLOps practices for pipeline orchestration, monitoring, and deployment.
Real-time & Batch Processing: Expertise in both batch and real-time data processing, leveraging tools like Kafka and workflow orchestrators (e.g., Airflow, Kubeflow) for reliable data ingestion.
Added Advantage (Expanded):
Proficiency in scripting languages such as Bash, Python, and Perl for automation and monitoring of data jobs.
Strong knowledge of Linux/Unix systems administration, including services management, cron scheduling, memory/process debugging, and system performance tuning.
Experience with data orchestration tools like Apache Airflow, Apache Nifi, Control-M, or Oozie for workflow management.
Familiarity with CI/CD pipelines, Git, Jenkins, and automated deployment strategies for big data and Spark jobs.
Exposure to containerization and orchestration platforms like Docker and Kubernetes for deploying data services.
Working knowledge of monitoring and logging tools such as Clouder Manager, Grafana, Prometheus, ELK Stack, or Splunk for proactive issue detection.
Understanding of data security best practices, including Kerberos, LDAP, TLS encryption, and role-based access control (RBAC).
Familiarity with DevOps culture and agile methodologies to improve delivery efficiency and team collaboration.
#J-18808-Ljbffr