Phishing Detection with Explainable Machine Learning

Security Operations Centers (SOCs) are drowning in “black-box” vendor alerts. When an automated email security appliance flags a message as malicious without explaining why, it shifts the investigative burden directly back onto the analyst. If the AI cannot explain its reasoning, it is not a solution; it is just another alert generator.

This Phishing Detection with Explainable Machine Learning solution is rooted in my master’s thesis research. It rejects the hype of autonomous AI replacing analysts and instead focuses on building an interpretable, Random Forest-based detection engine. By isolating specific behavioral and linguistic features, this model flags malicious emails and explicitly tells the analyst which exact cues—such as a mismatched reply-to header combined with artificial urgency—triggered the alert.

The Research Premise: Behavior over Signatures

Traditional signature-based detection fails against zero-day phishing campaigns and polymorphic infrastructure. Modern adversaries rotate domains and IPs continuously. To counter this, our detection model focuses on the invariant elements of a phishing attack: the psychological manipulation of the target and the structural anomalies of the delivery mechanism.

We leverage a Random Forest algorithm because it provides an optimal balance between high predictive accuracy and interpretability. Unlike deep neural networks, a Random Forest allows us to extract feature importance scores, turning a mathematical prediction into a human-readable narrative.

Feature Engineering: The Core of the Model

The model evaluates incoming messages against distinct families of engineered features:

• Header Anomalies: Reply-To mismatch, DKIM/SPF soft-fails, spoofed sender domains, and anomalous routing paths.
• Sender Behavior: Historical communication baselines, time-of-day anomalies, and first-time sender metrics.
• Linguistic & Grammar Signals: Natural Language Processing (NLP) to detect urgency markers (“immediate action required”), financial imperatives (“invoice attached”), and semantic deviations typical of non-native corporate communication.
• Lexical URL Patterns: Entropy of embedded URLs, presence of typosquatting, use of IP-based links, and the ratio of dots/hyphens in the domain string.
• Brand Impersonation Cues: Discrepancies between the display name (e.g., “IT Helpdesk”) and the actual envelope sender address.
• Attachment Metadata: File extension mismatches (e.g., .pdf.exe), anomalous document macros, and encrypted archive detections.

The Explainability Imperative

When the model flags an email, it outputs a confidence score alongside the top contributing features.

Instead of a generic Alert: Malicious Email, the integration with a SOAR or case management platform (like TheHive) presents the analyst with:

Confidence: 89% | Primary Drivers: Urgency_Language_Score (High), Reply_To_Mismatch (True), Domain_Age (< 48 hours).

This explainability fosters analyst trust, drastically reduces Mean Time to Triage (MTTT), and provides immediate context for incident response containment.

Architecture and Analyst Workflow

1. Dataset Preparation: Aggregating historical, labeled corporate email data alongside open-source phishing corpora, while strictly enforcing data privacy and PII masking.
2. Feature Extraction: Python pipelines utilize basic NLP and regex to parse EML files and extract the designated behavioral and lexical features.
3. Model Training & Validation: Training the Scikit-learn Random Forest model, tuning hyperparameters to optimize for a low False Positive Rate (FPR) to prevent disrupting business communications.
4. SOC Integration: Deploying the model as an API endpoint. When the email gateway detects a borderline message, it queries the model.
5. Human-in-the-Loop Review: The model enriches the SIEM/SOAR alert. The analyst makes the final determination based on the provided explainable context.
6. Feedback Loop: Analyst classifications are fed back into the dataset to retrain the model, addressing concept drift as adversary tactics evolve.

Tool Stack

• Data Science: Python, Scikit-learn, Pandas, NumPy, Jupyter Notebooks.
• Feature Engineering: NLTK/Spacy (for basic intent/urgency parsing), custom Python parsing scripts.
• Security Operations: Seamless conceptual integration with SIEMs (Splunk/ELK) and SOAR/Case Management (TheHive, Shuffle).

Evaluation Metrics

We measure the model’s efficacy not by vendor claims, but by rigorous statistical evaluation:

Metric	Definition	SOC Impact
Precision	Of all emails flagged as phishing, how many actually were?	High precision minimizes analyst fatigue from false positives.
Recall	Of all actual phishing emails, how many did the model catch?	High recall ensures sophisticated attacks don’t slip through to the inbox.
F1 Score	The harmonic mean of Precision and Recall.	The primary metric for balancing the operational trade-off.
False Positive Rate (FPR)	How often legitimate email is blocked.	Must be aggressively minimized to prevent business disruption.

90-Day Research-to-Production Roadmap

• Days 1-30 (Data & Baseline): Curate and sanitize a historical dataset of 10,000+ labeled corporate emails. Build the Python feature extraction pipeline. Establish the baseline metrics using a simple logistic regression model.
• Days 31-60 (Training & Explainability): Train the Random Forest model. Extract feature importance matrices (SHAP values) to map the mathematical weights to human-readable traits. Tune the model to achieve a >95% F1 score in the lab environment.
• Days 61-90 (Shadow Deployment & SOC Integration): Deploy the model in “shadow mode” against a live mail stream (logging predictions without blocking). Pipe the explainable outputs into TheHive for analyst review. Refine thresholds based on real-world false positives before transitioning to active alerting.

Machine learning in cybersecurity is not magic; it is applied mathematics. By prioritizing explainable, feature-driven models over black-box AI, we empower security teams to detect advanced social engineering while maintaining complete visibility into the why behind every alert.