Data Engineer(Python

3 weeks ago


Singapur, Singapore Unison Consulting Pte Ltd Full time

Data Architecture and Design:

  • Collaborate with cross-functional teams to understand data requirements and design efficient and scalable data architectures.
  • Develop and maintain data models, schema designs, and data flow diagrams.

ETL Development:

  • Design, develop, and optimize Extract, Transform, Load (ETL) processes using Python and PySpark.
  • Implement robust data pipelines for efficient data extraction, transformation, and loading from various sources to data warehouses.

Data Processing and Transformation:

  • Leverage PySpark for large-scale data processing, ensuring high performance and reliability.
  • Implement data transformations, aggregations, and cleansing procedures to maintain data quality.

Data Integration:

  • Integrate data from various sources, including structured and unstructured data, ensuring consistency and accuracy.
  • Work closely with data scientists and analysts to understand their data needs and provide support for data integration into analytical models.

Performance Optimization:

  • Monitor and optimize data processing and ETL jobs for performance, scalability, and efficiency.
  • Troubleshoot and resolve issues related to data pipeline performance.

Data Quality and Governance:

  • Implement data quality checks and validation processes to ensure the accuracy and reliability of the data.
  • Enforce data governance policies and best practices.

Collaboration and Documentation:

  • Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand and address their data requirements.
  • Document data engineering processes, ETL workflows, and data architectures.

Technology Research and Adoption:

  • Stay abreast of the latest trends and advancements in data engineering and recommend the adoption of new technologies and tools to enhance efficiency.

Requirements

  • Minimum of 7 years of hands-on experience in data engineering, with a focus on Python and PySpark.
  • Proven experience in designing and implementing scalable and efficient ETL processes.
  • Strong knowledge of data modeling, data warehousing concepts, and database systems.
  • Experience with big data technologies and distributed computing, with a focus on PySpark.
  • Proficiency in working with cloud platforms such as AWS, Azure, or Google Cloud.
  • Strong problem-solving and troubleshooting skills.


  • Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using SAS, Spark, Python, and PySpark. Ensure the smooth flow of data from various sources to destination systems. Utilize SAS, Spark, Python, and PySpark for efficient data processing. Design and implement data models to support business requirements. Work closely with data architects to...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Design, implement, and maintain scalable and efficient data pipelines using Spark, Python, and PySpark.Ensure the smooth flow of data from various sources to destination systems.Utilize SAS, Spark, Python, and PySpark for efficient data processing.Design and implement data models to support business requirements.Work closely with data architects to ensure...

  • Software Engineer

    3 weeks ago


    Singapur, Singapore Flintex Consulting Pte Ltd Full time

    Job DescriptionJob Responsibilities: • Gather software requirements and design solutions for modules and small software systems• Develop and support software including applications, database integration, interfaces and new functionality enhancements.• Write both internal and external technical documentation• Deploy, install, and configure hardware...

  • Software Developer

    4 weeks ago


    Singapur, Singapore Squarepoint Capital Full time

    Squarepoint is a global investment management firm that utilizes a diversified portfolio of systematic and quantitative strategies across financial markets that seeks to achieve high quality, uncorrelated returns for our clients. We have deep expertise in trading, technology and operations and attribute our success to rigorous scientific research. As a...

  • Data Engineer Intern

    4 weeks ago


    Singapur, Singapore Razer Full time

    Job Responsibilities :Intern will be playing a key role in the revamp of existing data pipelines and building new pipelines that are essential for data analyst/data scientist to perform analytics.By studying and understanding the data and pipelines, the intern will perform data profiling, data processing, automation and deployment to ensure that data is...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore Unison Consulting Pte Ltd Full time

    Build and optimizing ‘big data’ data pipelines, architectures and data sets. Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. To support and develop Data modeling, Implementation activities. Strong analytic skills related to working with...

  • Python Data Engineer

    4 weeks ago


    Singapur, Singapore Crypto.com Full time

    We are a team to design, develop, maintain, and improve software for various ventures projects, , projects that are adjacent to our core businesses and are bootstrapped fast with a lean team. You will be actively involved in the design of various components behind scalable applications, from frontend UI to backend infrastructure.We build and maintain a data...

  • Software Engineer

    3 weeks ago


    Singapur, Singapore Flintex Consulting Pte Ltd Full time

    Job Responsibilities: • Gather software requirements and design solutions for modules and small software systems• Develop and support software including applications, database integration, interfaces and new functionality enhancements.• Write both internal and external technical documentation• Deploy, install, and configure hardware and software...

  • Data Engineer(AI/ML)

    3 weeks ago


    Singapur, Singapore MATRIX PROCESS AUTOMATION PTE. LTD. Full time

    Roles & ResponsibilitiesSupport MLOps projects. The person will be working on Machine Learning Servers and Data Robot along with the projects that require AI model deploymentRequirements:Software development experience in making Artificial Intelligence(AI) /Machine Learning(ML) modelsGood understanding of programming language such as Python, R and...

  • Big Data Engineer

    1 week ago


    Singapur, Singapore TikTok Full time

    About the team :Libra is a large-scale online one-stop A/B testing platform developed by TikTok Data Platform .Provide experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference,...

  • Data Engineer

    3 weeks ago


    Singapur, Singapore PSA Singapore Full time

    Requirements Gathering (both functional and non-functional), Modular & Re- usable Design. • Proficient in ETL using programming language /tools such as Python and/or SSIS and/or Informatica Power Centre • Able to develop data applications including integration with ICT systems, build APIs and web applications via .Java and/or Python •...

  • Senior Data Engineer

    3 weeks ago


    Singapur, Singapore Helius Full time

    Responsibilities Design and develop data pipelines and data infrastructure to support efficient and scalable data processing and analytics. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and translate them into technical solutions. Implement and maintain data integration and ETL (Extract, Transform,...


  • Singapur, Singapore U3 Full time

    Python/SQL Backend Developer Python/SQL Backend Developer will support Data Ecosystem projects. Job Description/ Requirements / Skills: Strong hands-on Python software development skills with experience in multiple enterprise projects. Strong hands-on SQL skills in any RDBMS. Some experience in automation testing and CICD pipelines. ...

  • Python Developer

    3 weeks ago


    Singapur, Singapore Elitez Recruitment Services Private Limited Full time

    Job Title: Python DeveloperLocation: SingaporeJob type: ContractExperience: 5+ yearsRequirement MUST have relevant Software Development experience in Python and relevant software development certifications will be an added advantage. Certified Software Testing Professional and/or Certified Software Quality Analyst or an equivalent certification will be an...

  • Senior Python

    4 weeks ago


    Singapur, Singapore Helius Full time

    Company Introduction Our Client provides loans to businesses in Asia for growth based activities using revenue and alternative data to underwrite. Small businesses in Asia are highly “bankable” yet over 50% are unable to obtain a loan. The simple reason boils down to economics. They solve this problem by providing an automated digital platform to...


  • Singapur, Singapore MISC Berhad Full time

    Description KEY ACCOUNTABILITIES Manage, plan and maintain the facilitation of support for data analysis across Eaglestar’s growing global fleet . Project-manage and deliver on data-related implementations ensuring that deliverables are met within agreed scope and timelines Manage, develop, and provide Eaglestar’s internal/external stakeholders...


  • Singapur, Singapore Unison Consulting Pte Ltd Full time

    We are seeking a talented and experienced Data Engineer cum SAS Consultant to join our team. The ideal candidate will possess a strong background in data engineering, SAS programming, and proficiency in Python and Spark or PySpark. The role involves designing, developing, and implementing data solutions while leveraging SAS tools to provide insightful...

  • Big Data Engineer

    3 weeks ago


    Singapur, Singapore Virtusa Full time

    Big Data Engineer - 111170 Description Job Summary We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate must have a minimum of 4 years of experience managing data engineering jobs in big data environment , Cloudera Data Platform. Overall experience can be 5 to 10 years.. The successful candidate will...

  • Data Engineer

    4 weeks ago


    Singapur, Singapore NodeFlair Full time

    1. Key Outputs:  Collaborate with both non-technical and technical team members from cross-functional business and IT teams to gather, elicit and document data requirements and translate them into: Data Models and Mapping Specifications Develop and govern data modeling standards, best practices, and guidelines to ensure consistency and maintainability...


  • Singapur, Singapore OnLoop Full time

    We are seeking a motivated and talented Data Engineering Intern with a special focus on prompt engineering to join our growing team. This internship offers an exciting opportunity to gain hands-on experience in the field of data engineering while contributing to the development of cutting-edge natural language processing (NLP) models. The ideal candidate is...