
Big Data Engineer
4 days ago
**Roles and Responsibilities:
- **Develop Spark Scala and PySpark jobs for data transformation and aggregation.**
- **Create unit tests for Spark transformations and helper methods.**
- **Utilize Spark and Spark SQL to read parquet data and create tables in Hive using the Scala API.**
- **Collaborate closely with the Business Analysts team to review test results and obtain sign-off.**
- **Prepare design and operations documentation for future reference.**
- **Conduct peer code quality reviews and act as a gatekeeper for quality checks.**
- **Engage in hands-on coding, often in a pair programming environment.**
- **Work within highly collaborative teams to build high-quality code.
**Qualifications and Requirements:
- **4-10 years of experience as a Hadoop data engineer.**
- **Strong expertise in Hadoop, Spark, Scala, PySpark, Python, Hive, and Impala.**
- **Familiarity with Oracle, Spark streaming, Kafka, and machine learning (ML).**
- **Proficiency in Agile methodologies, CI/CD, Git, Jenkins, and Cloudera Distribution.**
- **Experience in Core Banking or Finance domains is desirable.**
- **Cloud experience with AWS is a plus.
-
Big Data Engineer
7 hours ago
Singapore THE EDGE PARTNERSHIP HOLDINGS PTE. LTD. Full time**Big Data Engineer - 1-year contract **Key Responsibilities** - **Programming**:Utilize Java, Scala, or Python for efficient coding and problem-solving. - **Big Data Tools**:Work with various big data processing tools and techniques, including Airflow and Ctrl-M. - **Database Knowledge**:Leverage experience with RDBMS, data warehouses, and Unix shell...
-
Data Engineer
1 day ago
Singapore NodeFlair Full time**Job Summary**: **Salary** S$5,000 - S$9,000 / Monthly **Job Type** **Seniority** Mid **Years of Experience** At least 5 years **Tech Stacks** Snowflake yarn Oracle Shell Script CI Shell UNIX Presto Java MSSQL Airflow Spark Linux Python Astek is proposing an opportunity for **Data Engineer** (Big Data) to support our project based in**...
-
Data Engineer
1 day ago
Singapore TALENTSIS PTE. LTD. Full timeWe are looking for a skilled **Data Engineer**with a focus on **Big Data**to design, develop, and manage large-scale data processing systems. If you're passionate about working with big data technologies and driving impactful solutions, we want to hear from you. **Key Responsibilities**: - Build and maintain big data pipelines using technologies like...
-
Big Data Engineer
2 days ago
Singapore SAKSOFT PTE LIMITED Full timeExperience: 9+ Years Role: Big Data Engineer Skills from Primary Skill Cluster Spark, Big Data, Hive, Java, 5+yrs with hands-on on Java is Must Database knowledge and should be able to write the queries is Must. Handson on Unix Shell scripting is Must. Knowledge in Spark SQL (Good to have). Understand on Regulatory reporting(Good to have). Design...
-
Data Engineer, Big Data
7 days ago
Singapore Grab Full time**Job Description**: **Life at Grab**: At Grab, every Grabber is guided by The Grab Way, which spells out our mission, how we believe we can achieve it, and our operating principles - the 4Hs: Heart, Hunger, Honour and Humility. These principles guide and help us make decisions as we work to create economic empowerment for the people of Southeast...
-
Big Data Engineer
3 days ago
Singapore Unison Consulting Pte Ltd Full time $90,000 - $120,000 per yearWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 5 years of experience managing data engineering jobs in big data environment e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, developing, and maintaining the data ingestion and processing...
-
Big Data Engineer
3 days ago
Singapore beBeeData Full timeBig Data Engineer PositionWe are seeking a skilled and experienced Big Data Engineer to join our team. The ideal candidate will have hands-on experience in building and maintaining data pipelines, working with cloud platforms (Alicloud, AWS, Microsoft Azure, or Google Cloud Platform), and ensuring efficient data integration, storage, and accessibility.Job...
-
Big Data Engineer
2 weeks ago
Singapore NOBILITY SERVICES SINGAPORE PTE. LTD. Full timeRoles & Responsibilities Job Requirements:· Degree in Information Technology or equivalent.· Must have 6- 8 years of experience in Data Warehousing· Extensive experience in ERD Design/ETL/Querying· Must have at least 5 years of Python Development in Data Engineering· Minimum 6 years of SQL experience· Must have experience in Big Data Technologies-...
-
Big Data Platform Engineer
4 days ago
Singapore Centre for Strategic Infocomm Technologies (CSIT) Full time**SINGAPORE, SINGAPORE /** **SOFTWARE ENGINEERING - DATA PLATFORMS /** **FULL-TIME** - Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team, you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of...
-
Big Data Engineer
4 weeks ago
Singapore NOBILITY PLACEMENT SERVICES PTE. LTD. Full timeRoles & ResponsibilitiesJob Requirements:· Degree in Information Technology or equivalent.· Must have 6- 8 years of experience in Data Warehousing.· Extensive experience in ERD Design/ETL/Querying.· Must have at least 5 years of Python Development in Data Engineering.· Minimum 6 years of SQL experience.· Experience in RestAPI is mandatory.· Experience...