Share Job Posting:

About the Job

Overview

One of our top e-commerce clients is currently looking out for a Data Engineer to provide clean, usable data to drive the success of the Company.


Responsibilities
- To architect the design of the data platform, lead strategic planning, and execution for all the data platform changes.
- Build stream processing systems to capture request/event-level data from source systems, do the pre-processing, compute additional metrics for analysis and make the data available for downstream systems in real-time.
- All Metrics to be available to end-users in real-time either through dashboards or for ad-hoc Querying, Metric computation logic to be completely transparent to end-users.
- Data platform design to focus on scalability, low latency, high throughput, and less maintenance effort. Automatic test cases to validate the accuracy of the metrics.
- Data Platform to have backfill mechanism to handle computation logic
changes.
 
Skill Sets
- Bachelors/master’s in computer science form top tier Engineering School
- 4+ years prior engineering experience in building scalable systems
- Prior experience in building stream processing systems (Kafka/Kinesis) - Mandatory
- Proven ability to work in a fast-paced environment.
- History and Familiarity of server-side development of APIs, databases, DevOps and systems.
- Prior experience on Spark SQL, Scala, Hadoop, Hive – preferable.
- Fanatic about building scalable, reliable data products