Data Engineer(Python, Pyspark, Spark

Key Responsibilities: Analyze and understand existing SAS programs and datasets Design and develop PySpark code that replicates the functionality of existing SAS programs Optimize PySpark code for performance and scalability Test and validate PySpark code to ensure it produces the same results as the original SAS programs Identify and resolve issues and...

Key Responsibilities:Create Spark Scala/PySpark jobs for data transformation and aggregation.Produce unit tests for Spark transformations and helper methods.Use Spark and Spark-SQL to read parquet data and create tables in Hive using the Scala API.Work closely with Business Analysts to review test results and obtain sign-off.Prepare necessary design and...

Roles & Responsibilities Create Spark Scala/Pyspark jobs for data transformation and aggregation. Produce unit tests for Spark transformations and helper methods. Use Spark and Spark-SQL to read the parquet data and create the tables in hive using the Scala API. Work closely with Business Analysts team to review the test results and obtain sign off Prepare...

Data Architecture and Design: Collaborate with crossfunctional teams to understand data requirements and design efficient and scalable data architectures. Develop and maintain data models, schema designs, and data flow diagrams.ETL Development: Design, develop, and optimize Extract, Transform, Load (ETL) processes using Python and PySpark. Implement robust...

Job Summary:SalaryS$7,500 - S$9,500 / MonthlyJob TypeSeniorityMidYears of ExperienceAt least 5 yearsTech StacksETL Informatica Analytics SAP pySpark Cloudera Hive Impala Linux SQL PythonJob Summary:- a minimum of 4 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. Overall experience can be 5 to 10 years....

JD: Minimum 5 years of total IT experience; at least 4 years of experience in development with PySpark & Python; Good knowledge and experience on Big Data platform, Big Data Eco system tools, SPARK, Python, Etc. Experience with Banking domain is preferable. Strong understanding of SDLC processes including all the project documentations. Ability to work under...

Roles & ResponsibilitiesOur company, a mobile gaming market leader, seeks a talented individual for regional expansion.TasksParticipate in the design of DWH, DataLake & DeltaLake Build a culture of data governance Participate in business analysis and design data based on that analysis Develop and maintain a continuous data delivery pipeline Develop a data...

Excellent Python Technical aptitude, including understanding of data science concepts and tools (e.g. Hadoop, Spark, Python) Knowledge on Spark, Python modelling packages, JIRA, Bitbucket Experience of User Acceptance Testing and requirements gathering Experience working with large and complex datasets Effective communicator Strong team player

Responsibility: Develop and maintain data pipelines implementing ETL processes in Bigdata environment Take responsibility for Hadoop development and implementation Experience in Python, Spark, PySpark, and Hive Understanding of data warehousing and data modeling techniques Knowledge on Data lake architectureRequirements: Bachelor's degree in Computer Science...

Responsibilities: Responsible for using Data wrangling techniques to convert one 'raw' form into another including data visualization, data aggregation, training a statistical model etc Work with various relational and nonrelational data sources with the target being Azurebased SQL Data Warehouse & Cosmos DB repositories Clean, unify and organize messy and...

Responsibilities: Develop enterprise grade Data Products using Spark and other Big Data frameworks Benchmark existing solutions, identify gaps, and propose solutions to eliminate them Be able to work within a team of data engineers, data scientists, Devops engineers, businessanalysts, project managers around the proposed solution and contribute to the team...

Our company, a mobile gaming market leader, seeks a talented individual for regional expansion.Tasks Participate in the design of DWH, DataLake & DeltaLake Build a culture of data governance Participate in business analysis and design data based on that analysis Develop and maintain a continuous data delivery pipeline Develop a data platform for product and...

Roles & ResponsibilitiesKerry Consulting is working together with a MNC to hire for a Data Engineer as part of their data migration team.ResponsibilitiesDevelop and optimize SQL, Hive SQL, and Spark scripts to ensure efficient data processing. Collaborate with cross-functional teams to understand data requirements and ensure data integrity throughout the...

Roles & Responsibilities EMAIL ID : Interested candidates, please email your resume to Only shortlisted candidates would be contacted for interview.Role: BAU Support - 12 months Renewable contractExperience: Minimum of 8 yearsLocation : East CoastBenefits:Renewable contract (Performance-based, with possibility of conversion to perm) Medical Insurance...

I am working with a client in the financial industry, with a well-recognised name brand and a stable work environment. Looking to hire a data scientist for their growing analytics team.Job responsibilities: Develop and evaluate machine learning models and advanced analytics to address business problems at a enterprise level Implement and deploy models into...

Job Title: Data Engineer (Python, Java or Scala)Welcome to our company! We are currently looking for a skilled Data Engineer to join our team. In this role, you will play a key part in designing, constructing, and managing data pipelines to support our data analytics and business intelligence projects. Collaboration with various teams is essential to create...

Data EngineerThis is a 12 months contract assinged to our clientWork Location: To be confirmedSalary Range : $6,000-$9,500Job Description:Data Engineer with hands on programming skills using Python, Spark, ETL pipeline related such asKafka, airflow.Experience on Data Warehouse ToolsSkillsets RequirementData EngineeringData Warehouse ToolsPython (Programming...

Job Summary:SalaryS$5,500 - S$9,167 / MonthlyJob TypeSeniorityMidYears of ExperienceAt least 3 yearsTech StacksSnowflake Analytics pySpark Presto Hive Airflow Spark SQL PythonPosition OverviewAutodesk is seeking an experienced data analyst reporting to the Senior Manager of Engineering to join the Licensing and Compliance team that administers Autodesk's...

Responsibilities: Proficient in Operating System such as AWS Azure, Linux, Red Hat Enterprise Linux and Microsoft Windows Integrate Terraform Cloud with CI/CD pipelines for automated infrastructure deployment Monitor and optimize Terraform Cloud performance, scalability, and reliability Proactive handling of project and supporting project implementation...

We are seeking a talented and experienced Spark Scala Developer with expertise in Elasticsearch to join our team. As a Spark Scala Developer, you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch. You will collaborate with cross-functional teams to build scalable and efficient...