Data Pipeline Architect

2 weeks ago

Singapore beBeeDataEngineer Full time $120,000 - $180,000
Job Description:

We are seeking a Data Engineering professional to design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration. The ideal candidate will have expertise in collaborating with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.

The successful candidate will be responsible for implementing data integration and transformation processes to ensure optimal performance and reliability of data pipelines. They will also optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.

The role involves troubleshooting and resolving issues related to data pipelines, ensuring smooth operation and minimal downtime. The Data Engineer will work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.

Additionally, the successful candidate will develop and maintain documentation for data pipelines, processes, and system architecture. They will stay updated with the latest industry trends and best practices related to data engineering and AWS services.

Requirements:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Proficiency in Python, PySpark, and SQL for data processing and manipulation.
  • Minimum 5 years of experience in data engineering, specifically working with Apache Airflow and AWS technologies.
Preferred Qualifications:
  • AWS certification(s) related to data engineering or big data.
  • Experience working with big data technologies like Snowflake, Spark, Hadoop, or related frameworks.
  • Familiarity with other data orchestration tools in addition to Apache Airflow.
  • Knowledge of version control systems like Bitbucket, Git.

  • Data Pipeline Architect

    5 days ago

    Singapore beBeeDataPipeline Full time $90,000 - $120,000

    About the Role:We are seeking a highly skilled Data Pipeline Architect to join our team. As a key member of our data engineering group, you will be responsible for designing and implementing efficient data pipelines that meet the needs of our organization.Key Responsibilities:Design and develop scalable data pipelines using Python and AWS services (S3,...

  • Data Pipeline Architect

    2 weeks ago

    Singapore beBeeSoftwareEngineering Full time

    Key Role Responsibilities We seek a Software Engineer to work on multiple fronts, building and optimizing data pipelines for AI-driven initiatives. Design scalable and efficient workflows in support of analytics projects. Utilize Copilot Studio for AI-driven application development and automation. Implement AI Foundry solutions on Microsoft Azure,...

  • Senior Data Pipeline Architect

    2 weeks ago

    Singapore beBeeDataEngineering Full time $180,000 - $200,000

    Data Engineering ExpertWe are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining complex data pipelines using Python. Responsibilities Design and Develop Complex Data Pipelines: Collaborate with cross-functional teams to understand data...

  • Senior Data Pipeline Architect

    1 week ago

    Singapore beBeeDataEngineering Full time

    Data Engineering Expert We are seeking a highly skilled Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining complex data pipelines using Python. Responsibilities Design and Develop Complex Data Pipelines: Collaborate with cross-functional teams to...

  • **Data Pipeline Architect

    7 days ago

    Singapore beBeeDataEngineer Full time $80,000 - $120,000

    **Senior Data Engineer Position**Our organization is seeking a skilled Senior Data Engineer to develop and maintain our data pipelines. The ideal candidate will have expertise in designing, building, and optimizing data solutions.Key Responsibilities:Enhance the existing data pipeline using industry-standard tools and technologies.Redesign and implement data...

  • Data Pipeline Architect

    4 days ago

    Singapore beBeeDataEngineer Full time $100,000 - $130,000

    Job Title: Data EngineerWe are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will design, develop, and optimize data pipelines using AWS services.Responsibilities:Design and implement ETL/ELT pipelines using AWS Glue, Athena, S3, Redshift, and Iceberg.Implement monitoring and alerting mechanisms for data workflows to ensure...

  • Data Pipeline Architect

    7 days ago

    Singapore beBeeData Full time $90,000 - $120,000

    Job Title: Data Engineering Specialist We are seeking a skilled Data Engineer to create and implement data pipelines using AWS services, Databricks, and Informatica IDMC. Key Responsibilities: Develop ETL processes to cleanse, transform, and enrich data using Databricks' Spark capabilities and Informatica IDMC. Monitor and optimize data processing and...

  • Data Pipeline Architect

    2 weeks ago

    Singapore beBeeDataEngineer Full time

    Job Title We are looking for a skilled Data Engineering professional to lead the development and maintenance of robust and scalable data pipelines. Develop and implement ETL pipelines in Python to power industry models and analytics products. Collaborate with lead analysts to ensure data accuracy, completeness and utility value across multiple sources...

  • Data Pipeline Architect

    1 week ago

    Singapore beBeeData Full time $150,000 - $200,000

    Job DescriptionWe are seeking a talented Senior Data Engineer to join our team. In this role, you will be responsible for designing, developing, and maintaining data pipelines and ETL/ELT processes.Develop maintainable data pipelines using unit and integration testing.Collaborate with data teams to automate deployment and monitoring.Optimize data storage and...

  • Leading Data Pipeline Architect

    4 days ago

    Singapore beBeeDataEngineer Full time $90,000 - $120,000

    Job Title: Senior Data EngineerWe are looking for a skilled Senior Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing data pipelines using industry-standard tools and mechanisms.Responsibilities:Enhance the existing data pipeline using industry-standard tools and mechanisms.Redesign...