🔮 Crypto Sentiment Predictor & Explainable AI

A production-ready data engineering and machine learning pipeline that predicts cryptocurrency price movements based on news sentiment, price history, and explainable AI (XAI).

✨ What It Can Do

• Multi-Coin Support: Tracks 5 major cryptocurrencies (Bitcoin, Ethereum, Binance Coin, Solana, Cardano).
• Automated Data Pipeline: Scrapes recent crypto news articles, and 90-day historical prices from CoinGecko.
• Deep Learning Sentiment Analysis: Uses HuggingFace's FinBERT to classify news sentiment (Bullish/Bearish/Neutral) specific to each coin.
• Ensemble Forecasting: Combines a time-series LSTM neural network with Facebook's Prophet model to predict whether a coin will go UP or DOWN, along with a dynamic confidence score.
• AI Explainability (SHAP): Doesn't just give you a prediction — it uses SHAP (SHapley Additive exPlanations) to tell you exactly which words in the news triggered the Bullish or Bearish sentiment (e.g., "declining", "accumulation").
• Premium Glassmorphism Dashboard: A beautiful, real-time Streamlit dashboard built with custom CSS, Plotly charts, and responsive UI.

🚫 What It Does NOT Do

• It is NOT a trading bot: This project provides analysis and forecasting, it does not execute trades, manage a portfolio, or connect to an exchange API for buying/selling.
• It is NOT financial advice: The predictions are based purely on NLP sentiment and historical trends for educational/portfolio purposes.
• It does NOT require Paid API Keys: The entire pipeline uses free, public APIs (CoinGecko public endpoints, standard RSS/News feeds) and local HuggingFace models.

---

🛠️ Architecture & Tech Stack

crypto-sentiment/
├── scrapers/              # Data ingestion
│   ├── news_scraper.py    # Fetches crypto news via RSS/feedparser
│   ├── price_scraper.py   # CoinGecko API (90-day history)
│   └── reddit_scraper.py  # Public Reddit JSON (No Auth needed)
├── preprocessing/         # Data cleaning & NLP features
│   ├── cleaner.py         # Text sanitization
│   └── feature_engineer.py# Merges text sentiment with price history
├── models/                # ML models
│   ├── sentiment_model.py # FinBERT (Local HuggingFace model)
│   └── forecasting_model.py # LSTM + Prophet ensemble
├── explainability/        # XAI
│   └── shap_explainer.py  # Computes word-level impact on predictions
├── dashboard/             # Streamlit UI
│   └── app.py             # Glassmorphism frontend
├── prefect_pipeline.py    # Prefect orchestration
└── config.py              # Central configuration

Tech Stack:

• Data/ML Pipeline: Python, Pandas, PyTorch (LSTM), Prophet, Transformers (FinBERT)
• Explainable AI: SHAP
• Orchestration: Prefect
• Frontend: Streamlit, Plotly, Custom Vanilla CSS

---

🚀 Quick Start

1. Setup Environment

# Clone the repository and setup virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

2. Configure Settings

cp .env.example .env

Note: The pipeline uses public endpoints. You do not need to configure Reddit or CoinGecko API keys unless you want to bypass standard public rate limits.

3. Run the ML Pipeline

This will fetch the latest news, download price history, run FinBERT, train the LSTM/Prophet models, and generate SHAP explanations for all 5 coins.

python prefect_pipeline.py

(First run will download the FinBERT weights from HuggingFace).

4. Launch the Dashboard

streamlit run dashboard/app.py

Open http://localhost:8501 in your browser to view the live dashboard!

---

🧠 How the ML Pipeline Works (Step-by-Step)

1. Scraping: scrape_prices_task pulls the last 90 days of prices from CoinGecko. scrape_news_task pulls recent crypto news articles.
2. Preprocessing: The pipeline filters news articles by coin-specific keywords (e.g., articles mentioning "vitalik" go to Ethereum).
3. Sentiment: Every relevant article is passed through FinBERT to get a bullish/bearish/neutral score.
4. Forecasting:
   • The LSTM model is dynamically trained on the 90-day history + daily sentiment scores to predict the next price movement.
   • Prophet is trained on the raw time-series data to provide a baseline forecast.
   • The Ensemble combines both. If Prophet and LSTM agree on the direction (UP/DOWN), the confidence score is boosted.
5. Explainability: SHAP breaks down the FinBERT model to figure out exactly which words in the text pushed the model towards its conclusion.
6. Delivery: Results are saved atomically to a JSON file which the Streamlit dashboard polls in real-time.