Mike Sosa
Back to portfolio

Breaking the Phishing Detector: Adversarial ML Research

  • AI Security
  • Adversarial ML
  • Application Security
  • Python

Application Security Researcher, BYU Marriott School of Business, 2020 to 2022

The problem

Machine learning is good at catching phishing, but attackers adapt. A model that looks strong in testing can be fooled by small, deliberate changes to a URL or its metadata. If security teams are going to trust ML-based detection, someone has to find out where it breaks before an attacker does.

The approach

I worked both sides: building the detector, then attacking it.

  • Building the detector: I engineered end-to-end phishing-detection pipelines in Python and scikit-learn, covering data ingestion, cleaning, normalization, labeling, and feature engineering. I extracted URL lexical features and WHOIS/DNS metadata, then trained supervised classifiers, including Random Forest, Support Vector Machine, and logistic regression models.
  • Attacking it: I ran evasion and adversarial-example testing, manipulating URL lexical, domain, and metadata features to find the conditions under which malicious samples could slip past the model.
  • Hardening it: I evaluated each classifier with precision, recall, F1-score, confusion matrices, and false-positive and false-negative analysis. Then I used feature-level error analysis to find the dependencies an attacker could exploit and to make the models more resilient.
  • Securing the software around it: I designed a platform-agnostic application security validation framework based on secure development requirements and the OWASP Top 10. It automated repeatable security checks and explored adding SAST, DAST, and CI/CD security validation to catch vulnerabilities earlier.

I also looked past evasion at the broader adversarial ML threats: data poisoning and model extraction, and how manipulating training data, inputs, or observable model behavior can undermine an ML-enabled security system.

The result

The research showed where phishing classifiers leaned on features an attacker could manipulate, and that error analysis fed directly into hardening the models. It also gave me hands-on experience in what's now called AI security, years before most teams were thinking about it.

What it shows about how I work: I build the defense, then attack it myself to find where it fails.

I've since started a new open-source project that builds on this research: phishing-detector-adversarial-robustness. It's fresh code and data, not the original university work.