Planetary scan
- Class
- Data engineering
- Discovered by
- Varun Kothari
- Role
- MERN + Data Engineering
- Period
- PES University
- Status
- Archived
A scalable MERN web application for academic publications and conference showcases, backed by real data engineering: ETL pipelines that move data through Apache Spark, Kafka, and Hadoop for real-time analytics and batch ingestion, exposed to the app through REST APIs.
Surface features
Feature 01ETL pipelines across Spark, Kafka & Hadoop
Feature 02Real-time analytics and batch data ingestion
Feature 03REST APIs for live data access
Hazard detected · the hard part
Integrating a big-data pipeline with a user-facing web app — and root-causing production data issues across both halves.
Composition · stack
- MongoDB
- Express.js
- React
- Node.js
- Apache Spark
- Kafka
- Hadoop