Data Engineer

Duration: 2-4 Months, > 4 Months

Internship Type: Undergraduate, Pre-University


Description

Raw data flows at high speed and high volume across enterprise systems. The multiplicity of file format and erroneous file content often make integration and maintenance of systems a challenging task. Data Engineers need to find trends in this complex environment and develop algorithms to make raw streaming data useful to the organisation. As a Data Engineer intern, you will:

  • Gain valuable experience in big data manipulation and acquire deep knowledge in database design and programming. 
  • Learn to manage, analyse and extract database-centric distinctive information and enrich them in real time to provide value to the end user.
  • Be able to hone your technical skills using multiple programming languages and scripting tools.


Broad Deliverables

  • Conduct technology scan and proof-of-concept.
  • Data acquisition, query and manipulation to address business issues.
  • Design, develop, test and deliver the system.
  • Present the system to management and users.


Possible Projects*

  • DPI accelerator (using Snort/Regex)
  • Pcap search engine
  • KSQL real-time stream processing


Prerequisites/Skills Required

  • Prior knowledge in programming


*List is non-exhaustive as projects are subject to availability at point of application