Automatic Text Summarizer using NLP

This project is a text-summarization tool in Python that condenses long articles or documents into a short summary using two complementary approaches: an extractive method (TextRank, which ranks and selects the most representative existing sentences) and an abstractive method using a fine-tuned transformer model (BART/T5) that generates new summary text.

Users can paste in text or upload a PDF/DOCX document, choose the desired summary length, and select between the extractive (faster, more literal) and abstractive (more fluent, occasionally paraphrased) modes, comparing outputs side by side.

The project is wrapped in a Streamlit web app with a simple word-count reduction indicator, and evaluates summary quality using ROUGE scores against reference summaries from a benchmark dataset, giving a practical introduction to both classical and transformer-based NLP summarization techniques.

Detailed Project Description

Automatic Text Summarizer using NLP applies machine learning to a real dataset: it prepares the data, trains and evaluates a model, and exposes the trained model through a simple interface so predictions can be made on new inputs.

How It Works
1. The dataset is collected/loaded, cleaned and explored (missing values, encoding, scaling).
2. Features are engineered and the data split into training and test sets.
3. One or more models are trained and compared using accuracy, precision/recall, F1 or RMSE.
4. The best model is saved and served through a web/GUI form that returns predictions.

Technology Roles
- Python
- NLP
- TextRank
- Hugging Face Transformers
- Streamlit

Implementation Steps
1. Collect the dataset (public repository or own data) and understand each attribute.
2. Set up the environment (Python, Hugging Face Transformers, NLTK/Gensim) and load the data into a notebook.
3. Clean the data: handle missing values, outliers, encoding and scaling; visualise distributions.
4. Split into train/validation/test sets and engineer relevant features.
5. Train several candidate models, tune hyper-parameters and compare metrics.
6. Save the best model and wrap it in a simple web/GUI form for live predictions.
7. Document results with plots, confusion matrix / error charts and conclusions.

Key Features
- Data pre-processing pipeline
- Model comparison
- Evaluation metrics and plots
- Prediction interface

Expected Output
On completion the student will have evaluation results (metrics, confusion matrix or error plots) and a working prediction form.

Deliverables
- Notebook / source code with comments
- Cleaned dataset and trained model file
- Evaluation report with metrics and plots
- Prediction interface