Hadoop Admin

3 weeks ago

Singapur, Singapore Unison Consulting Pte Ltd Full time

We are seeking a skilled and experienced Hadoop Administrator to join our team. As a Hadoop Administrator, you will be responsible for the design, implementation, and maintenance of our Hadoop infrastructure. You will work closely with data engineers, data scientists, and other stakeholders to ensure the smooth functioning of our big data ecosystem.

Responsibilities:

  1. Install, configure, and maintain Hadoop clusters to ensure high availability and optimal performance.
  2. Monitor Hadoop cluster performance and troubleshoot any issues that arise.
  3. Manage and maintain data security and access controls within the Hadoop environment.
  4. Implement and manage data retention policies to ensure efficient use of storage resources.
  5. Collaborate with data engineers and data scientists to optimize Hadoop infrastructure for data processing and analysis.
  6. Develop and maintain documentation related to Hadoop configuration, processes, and procedures.
  7. Work with vendors to evaluate and implement new Hadoop-related technologies and tools.
  8. Provide technical support and guidance to users of the Hadoop platform.
  9. Participate in on-call rotation for after-hours support as needed.

Requirements

  1. Bachelor’s degree in Computer Science, Information Technology, or a related field.
  2. Proven experience as a Hadoop Administrator or similar role in a production environment.
  3. Strong understanding of Hadoop architecture and ecosystem components (HDFS, YARN, MapReduce, Hive, Pig, etc.).
  4. Experience with Hadoop cluster setup, configuration, and tuning for performance and scalability.
  5. Proficiency in Linux system administration and shell scripting.
  6. Experience with Hadoop security frameworks such as Kerberos and LDAP.
  7. Excellent problem-solving and troubleshooting skills.