Python Data Engineer

3 weeks ago


Singapore HTC SOFTWARE SINGAPORE PTE LTD Full time
Roles & Responsibilities

We are seeking a talented Data Engineer experience in Quantexa with expertise in Hadoop, Scala, Spark, Elastic, Open Shift Container Platform (OCP) and DevOps practices.

As a Data Engineer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.

You will collaborate with cross-functional teams to build scalable and efficient data processing pipelines and search applications.

Knowledge and experience in the Compliance / AML domain will be a plus.

Working experience with Quantexa tool is a must.

Requirements

  • Bachelor's or master's degree in computer science, Software Engineering, or a related discipline
  • Possession of Quantexa certification as a Data Engineer or Data Architect, with proficiency in the tool
  • Proficiency in programming languages commonly employed in data engineering, including Spark, Python, Scala
  • Demonstrated experience as a Data Engineer, utilizing Hadoop, Spark, and data processing technologies in large-scale environments
  • Expertise in the Scala programming language and familiarity with functional programming principles and Anomaly Detection.
  • Prior experience with the Quantexa tool is highly desirable
  • Comprehensive understanding of Apache Spark architecture, including RDDs, DataFrames, and Spark SQL
  • Advanced proficiency in designing and developing data infrastructure utilizing Hadoop, Spark, and associated tools (HDFS, Hive, Pig, etc.)
  • Experience with containerization platforms such as OpenShift Container Platform (OCP) and container orchestration via Kubernetes
  • Knowledge of DevOps methodologies, CI/CD pipelines, and infrastructure automation tools (e.g., Docker, Jenkins, Ansible, BitBucket)
  • Experience with Graphana, Prometheus, and Splunk will be considered an added advantage
  • Background in integrating and utilizing Elasticsearch for data indexing and search applications
  • Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization techniques
  • Experience with distributed computing, parallel processing, and handling large datasets
  • Proficient in performance tuning and optimization methods for Spark applications and Elasticsearch queries
  • Strong problem-solving and analytical capabilities with the capacity to debug and resolve intricate issues
  • Familiarity with version control systems (e.g., Git) and collaborative development environments
Tell employers what skills you have

Apache Spark
Scala
Kubernetes
Big Data
Data Modeling
Pipelines
Architect
Hadoop
Software Engineering
Data Engineering
SQL
Python
Performance Tuning
Docker
Orchestration

  • Singapore DIGIWORLD TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesBachelor's degree in Computer Science or related field plus 5+ years of relevant work experience in Python, PySpark and Oracle Responsible for technical design, writing, support and operations of efficient, scalable, and extensible data integrations and ingestions & participate in architecture and design efforts. Conduct PoCs for...

  • Python Data Engineer

    3 weeks ago


    Singapore HTC SOFTWARE SINGAPORE PTE LTD Full time

    Roles & ResponsibilitiesWe are seeking a talented Data Engineer experience in Quantexa with expertise in Hadoop, Scala, Spark, Elastic, Open Shift Container Platform (OCP) and DevOps practices.As a Data Engineer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch.You will...

  • Python Developer

    4 weeks ago


    Singapore NTT DATA SINGAPORE PTE. LTD. Full time

    Roles & Responsibilities EMAIL ID : Interested candidates may also send their resume via email to mike.ramos@nttdata.comOnly shortlisted candidates would be contacted for interview.Role: Python Developer - 12 months Renewable contractExperience: Minimum of 6 yearsLocation : CentralResponsibilities:Corporate Tooling team in GMET Shared Platforms is...


  • Singapore DIGIWORLD TECHNOLOGIES PTE. LTD. Full time

    Roles & Responsibilities Bachelor's degree in Computer Science or related field plus 5+ years of relevant work experience in Python, PySpark and Oracle Responsible for technical design, writing, support and operations of efficient, scalable, and extensible data integrations and ingestions & participate in architecture and design efforts. Conduct PoCs for...

  • Data Engineer

    3 weeks ago


    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesWe are looking for a proactive and experienced Data Engineer who possesses strong expertise in SQL and Python, with the ability to design, implement, and maintain robust data solutions. Familiarity with .NET is highly desirable. In this role, you will work closely with cross-functional teams to ensure smooth and effective data-driven...

  • Data Engineer

    3 weeks ago


    Singapore ASTEK SINGAPORE INNOVATION TECHNOLOGY PTE. LTD. Full time

    Roles & ResponsibilitiesSummary:We are looking for a skilled Data Engineers to support and enhance an AI-powered Search solution with Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) capabilities. This solution will be implemented across Corporate Services and various departments, providing efficient and accurate responses by extracting...

  • Data Engineer

    3 days ago


    Singapore RANDSTAD PTE. LIMITED Full time

    Roles & ResponsibilitiesTo apply, It will be great if you could share your CV to hoonteck.tan@randstad.com.sg. Alternatively, you can apply at https://lnkd.in/gC9neEkPBe part of a large scale data project Exposure to latest technologies due to high investment in data within the companyAbout the companyOur client is an established company who is a market...

  • Data engineer

    4 weeks ago


    Singapore TOTAL EBIZ SOLUTIONS PTE. LTD. Full time

    Roles & ResponsibilitiesJD for Software Engineer (Data Engineering) for Tech Stack Products -Design, develop, and maintain scalable data pipelines.-Collaborate with cross-functional teams to integrate data systems.-Ensure data quality and integrity across platforms.-Optimize workflows for efficiency and performance.-Perform other miscellaneous duties as...

  • Python Engineer

    3 weeks ago


    Singapore CFB BOTS PTE. LTD. Full time

    Roles & ResponsibilitiesCFB Bots Pte Ltd is a technology service provider specializing in Intelligent Automation. We partner with large enterprises in their Digital Transformation journey and help them and their employees thrive in the Future of Work. Our mission is to co-create the Digital Workforce of the Future, and our vision is to make work enjoyable....


  • Singapore INFOCEPTS PTE. LTD. Full time

    Roles & ResponsibilitiesInfoCepts is a global leader of end-to-end data & analytics solutions with nearly 20 years of experience, also named as Gartner's 2020, 2021 and 2022 customers' choice for Data & Analytics providers. We continue to grow rapidly year over year, now employing more than 1,200 people in offices across the globe. As we have grown, we have...

  • Data engineer

    2 weeks ago


    Singapore FLINTEX CONSULTING PTE. LTD. Full time

    Roles & ResponsibilitiesData engineer (Azure) – Synapse and Pyspark, Python, Datawarehouse and Power BI , Azure DevopsSkills & Experience•Bachelor's Degree in Computer Science or Engineering with 2 years of experience in Azure Data engineering, Python, Pyspark or Big Data development•Sound Knowledge of Azure Synapse analytics for pipelines,...

  • Data Engineer

    3 weeks ago


    Singapore INTELLECT MINDS PTE. LTD. Full time

    Roles & ResponsibilitiesCAT-1 cleared/eligible candidate is required for this role. Degree in Computer Science or equivalent. At least 5 years' experience of working as a platform support engineer or a software engineer or a data engineer. Solid working knowledge of implementing the optimal data structures and algorithms to create efficient and scalable...

  • Data Engineer

    4 weeks ago


    Singapore INTELLECT MINDS PTE. LTD. Full time

    Roles & ResponsibilitiesCAT-1 cleared/eligible candidate is required for this role. Degree in Computer Science or equivalent. At least 5 years' experience of working as a platform support engineer or a software engineer or a data engineer. Solid working knowledge of implementing the optimal data structures and algorithms to create efficient and scalable...

  • Data Engineer

    3 weeks ago


    Singapore HTC SOFTWARE SINGAPORE PTE LTD Full time

    Roles & ResponsibilitiesBachelor's or Master's degree in Computer Science, Engineering, or a related field.Proven experience as a Data Engineer and Blockchain Developer.Proficiency in programming languages such as Python, Java, or C++.Experience with big data technologies (e.g., Hadoop, Spark).Strong understanding of blockchain technologies (e.g., Ethereum,...

  • Data Engineer

    3 weeks ago


    Singapore HTC SOFTWARE SINGAPORE PTE LTD Full time

    Roles & ResponsibilitiesBachelor's or Master's degree in Computer Science, Engineering, or a related field.Proven experience as a Data Engineer and Blockchain Developer.Proficiency in programming languages such as Python, Java, or C++.Experience with big data technologies (e.g., Hadoop, Spark).Strong understanding of blockchain technologies (e.g., Ethereum,...

  • Data Engineer

    3 weeks ago


    Singapore APAR TECHNOLOGIES PTE. LTD. Full time

    Roles & ResponsibilitiesWe are looking for a proactive and experienced Data Engineer who possesses strong expertise in SQL and Python, with the ability to design, implement, and maintain robust data solutions. Familiarity with .NET is highly desirable. In this role, you will work closely with cross-functional teams to ensure smooth and effective data-driven...

  • Senior Data Engineer

    3 weeks ago


    Singapore HAYS SPECIALIST RECRUITMENT PTE. LTD. Full time

    Roles & ResponsibilitiesWe are looking for Senior Data Engineer to join Data Engineer team squad. You will be responsible for platform administration, operational support, small change and release management. Larger, more complex changes are delivered by an L3 squad and project teams.Key Responsibilities:Release Management and Support activity (~60%):·...

  • Data Engineer

    3 weeks ago


    Singapore IOTALENTS PTE. LTD. Full time

    Roles & ResponsibilitiesThe successful candidate will provide operational support for Data Platforms.Responsibilities:Work closely with data product team and business end-users to implement and support data platforms using best-of-breed technology and methodology. Work closely with product vendors to design, deploy and integrate different data platforms...

  • Data Engineer

    3 weeks ago


    Singapore YOUZU (SINGAPORE) PTE. LTD. Full time

    Roles & ResponsibilitiesWe are looking for a skilled and motivated Data Engineer to join our team. You will play a key role in designing, maintaining, and expanding our data pipelines while actively shaping the future of our BI system. If you are passionate about data engineering, ETL processes, and driving business insights, we'd love to hear from...

  • Data Engineer

    3 weeks ago


    Singapore RANDSTAD PTE. LIMITED Full time

    Roles & Responsibilitiesabout the companyA Singapore-based game publisher dedicated to bringing high-quality casual games to a global audience. Seeking a Senior Data Engineer to design and manage data pipelines and infrastructure for our advanced user growth platform. This role will ensure smooth data integration and enable real-time analytics...