Data Engineer(Python

4 weeks ago


Singapur, Singapore Unison Consulting Pte Ltd Full time

Data Architecture and Design:

  • Collaborate with cross-functional teams to understand data requirements and design efficient and scalable data architectures.
  • Develop and maintain data models, schema designs, and data flow diagrams.

ETL Development:

  • Design, develop, and optimize Extract, Transform, Load (ETL) processes using Python and PySpark.
  • Implement robust data pipelines for efficient data extraction, transformation, and loading from various sources to data warehouses.

Data Processing and Transformation:

  • Leverage PySpark for large-scale data processing, ensuring high performance and reliability.
  • Implement data transformations, aggregations, and cleansing procedures to maintain data quality.

Data Integration:

  • Integrate data from various sources, including structured and unstructured data, ensuring consistency and accuracy.
  • Work closely with data scientists and analysts to understand their data needs and provide support for data integration into analytical models.

Performance Optimization:

  • Monitor and optimize data processing and ETL jobs for performance, scalability, and efficiency.
  • Troubleshoot and resolve issues related to data pipeline performance.

Data Quality and Governance:

  • Implement data quality checks and validation processes to ensure the accuracy and reliability of the data.
  • Enforce data governance policies and best practices.

Collaboration and Documentation:

  • Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand and address their data requirements.
  • Document data engineering processes, ETL workflows, and data architectures.

Technology Research and Adoption:

  • Stay abreast of the latest trends and advancements in data engineering and recommend the adoption of new technologies and tools to enhance efficiency.

Requirements

  • Minimum of 7 years of hands-on experience in data engineering, with a focus on Python and PySpark.
  • Proven experience in designing and implementing scalable and efficient ETL processes.
  • Strong knowledge of data modeling, data warehousing concepts, and database systems.
  • Experience with big data technologies and distributed computing, with a focus on PySpark.
  • Proficiency in working with cloud platforms such as AWS, Azure, or Google Cloud.
  • Strong problem-solving and troubleshooting skills.


  • Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using SAS, Spark, Python, and PySpark. Ensure the smooth flow of data from various sources to destination systems. Utilize SAS, Spark, Python, and PySpark for efficient data processing. Design and implement data models to support business requirements. Work closely with data architects to...

  • Data Engineer

    2 months ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using Spark, Python, and PySpark.Ensure the smooth flow of data from various sources to destination systems.Utilize SAS, Spark, Python, and PySpark for efficient data processing.Design and implement data models to support business requirements.Work closely with data architects to ensure...


  • Singapur, Singapore Unison Consulting Pte Ltd Full time

    We are in search of a proficient Data Engineer with expertise in Python, PySpark, Spark, and SQL. The perfect candidate will excel in constructing and enhancing data pipelines and structures. We need a team player who can collaborate effectively to design and deploy robust data solutions.Key Responsibilities:Design, construct, and manage efficient, scalable,...


  • Singapur, Singapore Unison Consulting Pte Ltd Full time

    We are looking for a skilled Data Engineer with a strong background in Python, PySpark, Spark, and SQL. The ideal candidate will have experience in building and optimizing data pipelines and architectures. This role requires a collaborative professional who can work across teams to design and implement robust data solutions.Key Responsibilities: Design,...

  • Python Data Engineer

    2 weeks ago


    Singapur, Singapore Crypto Full time

    We are a team to design, develop, maintain, and improve software for various ventures projects, , projects that are adjacent to our core businesses and are bootstrapped fast with a lean team. You will be actively involved in the design of various components behind scalable applications, from frontend UI to backend infrastructure.We build and maintain a data...


  • Singapur, Singapore Amaris Consulting Full time

    Job description Mission Description Develop resilient, high-performant, secure and cost-optimised cloud application with Python. Take care of the software life cycle, internal support N1/N2, develop ETL and data integration. Coordinate actions between the same team in Geneva and coordinate actions with internal teams...


  • Singapur, Singapore Amaris Consulting Full time

    Job description Mission Description Develop resilient, high-performant, secure and cost-optimised cloud application with Python. Take care of the software life cycle, internal support N1/N2, develop ETL and data integration. Coordinate actions between the same team in Geneva and coordinate actions with internal teams in Singapore. Assisting...

  • Software Engineer

    4 weeks ago


    Singapur, Singapore Flintex Consulting Pte Ltd Full time

    Job DescriptionJob Responsibilities: • Gather software requirements and design solutions for modules and small software systems• Develop and support software including applications, database integration, interfaces and new functionality enhancements.• Write both internal and external technical documentation• Deploy, install, and configure hardware...


  • Singapur, Singapore Airwallex Full time

    Airwallex is the leading financial technology platform for modern businesses growing beyond borders. With one of the worlds most powerful payments and banking infrastructure, our technology empowers businesses of all sizes to accept payments, move money globally, and simplify their financial operations, all in one single platform. Established in 2015,...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore Infosys Singapore & Australia Full time

    Infosys Consulting is a global management consulting firm helping some of the world’s most recognizable brands transform and innovate. Our consultants are industry experts that lead complex change agendas driven by disruptive technology. With offices in 20 countries and backed by the power of the global Infosys brand, our teams help the C-suite navigate...


  • Singapur, Singapore ST Engineering Full time

    Title: Engineer, data acquisition/engineering Job ID: Location: ST Engineering Hub, SG Description: Job Description The key responsibilities include data acquisition, data engineering, and building business intelligence solutions. This position works extensively with the Decision Science & Solutions (DSS) team in our commercial aerospace...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore U3 Full time

    Data Engineer  Job Description / Requirements / Skills: Strong hands-on experience of SQL skills in any RDBMS (experience with Snowflake is a plus). Strong understanding and experience of DBA knowledge and Excel. Strong hands-on experience in using Python to perform ETL/backend and stay up to date with latest libraries. Experience in automation...

  • Data Engineer

    2 months ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Build and optimizing ‘big data’ data pipelines, architectures and data sets. Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. To support and develop Data modeling, Implementation activities. Strong analytic skills related to working with...

  • Data Engineer

    2 hours ago


    Singapur, Singapore Amazon Asia-Pacific Holdings Private Limited - D43 Full time

    Do you want to contribute to a team working on cutting edge technology, solve new problems that didn’t exist before, and have the ability to see the impact of your successes?Amazon is shaping the future of digital video entertainment. We seek Data Engineers who can help us store, process and analyze vast amounts of data as we build products in the video...

  • Python Data Engineer

    4 weeks ago


    Singapur, Singapore Crypto.com Full time

    We are a team to design, develop, maintain, and improve software for various ventures projects, , projects that are adjacent to our core businesses and are bootstrapped fast with a lean team. You will be actively involved in the design of various components behind scalable applications, from frontend UI to backend infrastructure.We build and maintain a data...

  • Intern-Data Engineer

    2 weeks ago


    Singapur, Singapore Coca Cola Full time

    Description Summary: Position Overview:As a manufacturing company, Coca-Cola Company is constantly seeking ways to enhance accuracy in forecast models. This opportunity involves evolving and developing new models to improve predictability. We are looking for individuals with a passion for data modeling, strong analytical skills, and excellent business...

  • Software Engineer

    4 weeks ago


    Singapur, Singapore Flintex Consulting Pte Ltd Full time

    Job Responsibilities: • Gather software requirements and design solutions for modules and small software systems• Develop and support software including applications, database integration, interfaces and new functionality enhancements.• Write both internal and external technical documentation• Deploy, install, and configure hardware and software...

  • Data Engineer

    2 weeks ago


    Singapur, Singapore CommScope Full time

    In our 'always on' world, we believe it's essential to have a genuine connection with the work you do. RUCKUS Networks is hiring a Data Engineer who will work closely with our team in Singapore How You'll Help Us Connect the World You will join a dynamic and focused team to develop innovative big data applications in Analytics and Artificial...


  • Singapur, Singapore NTT DATA Full time

    Job Description Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer - 123101 Description Responsibilities Good understanding and completion of projects using waterfall/Agile methodology. • Good understanding of analytics and data warehouse implementations. • Ability to troubleshoot complex issues ranging from system resource to application stack traces. • Strong SQL, data modelling and data...