Data Platform Engineer


CSIT develops products to advance the national security interests of Singapore. Our products are used in a wide range of operations, including but not limited to, Counter-terrorism and Computer Network Defence. We are looking for talented data platforms engineers to be part of the team that builds large-scale data platforms to power real-time analytics that identify, analyse and disrupt security threats.

Role

As a Data Platform Engineer, you will play a key role in designing, implementing and maintaining big data platforms used to drive timely and mission-critical decisions. You will have the opportunity to explore new data platform technologies, design scalable platform solutions and solve challenging problems with high volume, velocity and variety of data. You will have the freedom to determine the best way to deliver value, be it developing from scratch or building upon the best solutions from the community.

Responsibilities

  • Design, implement and maintain large-scale data platforms with monitoring, authentication, alert management and log management
  • Conduct performance and health monitoring to ensure optimal performance and availability
  • Conduct resource planning and scaling to support increasing workload
  • Work closely with infrastructure, application and data science teams to implement solutions and resolve technical challenges together

Requirements

Minimum requirements

  • A degree in Computer Science, Computer Engineering or a related discipline
  • Knowledge of OS, networking and distributed computing
  • Experience with one or more of the following technologies: Spark, Kafka, Nifi, HBase, Hive, Hadoop, Kudu, Solr/ElasticSearch

Nice-to-haves

  • Certification in the above technologies
  • Experience in supporting data platform with multi-terabyte or billions of records
  • Experience in trouble-shooting and resolving issues (hardware, networking, OS and software) with self-hosted Hadoop cluster

 

As we are an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.