OrthoEraser: Coupled-neuron orthogonal projection for concept erasure
Author Identifier (ORCID)
Abstract
Text-to-image (T2I) models face significant safety risks from adversarial induction, yet current concept erasure methods often cause collateral damage to benign attributes when suppressing selected neurons entirely. This occurs because sensitive and benign semantics exhibit non-orthogonal superposition, sharing activation subspaces where their respective vectors are inherently entangled. To address this issue, we propose OrthoEraser, which leverages sparse autoencoders (SAE) to achieve high-resolution feature disentanglement and subsequently redefines erasure as an analytical orthogonalization projection that preserves the benign manifold’s invariance. OrthoEraser first employs SAE to decompose dense activations and segregate sensitive neurons. It then uses coupled neuron detection to identify non-sensitive features vulnerable to intervention. The key novelty lies in an analytical gradient orthogonalization strategy that projects erasure vectors onto the null space of the coupled neurons. This orthogonally decouples the sensitive concepts from the identified critical benign subspace, effectively preserving non-sensitive semantics. Experimental results on safety demonstrate that OrthoEraser achieves high erasure precision, effectively removing harmful content while preserving the integrity of the generative manifold, and significantly outperforming SOTA baselines. Warning text in red states that the paper contains results from unsafe models, highlighting a caution about the reliability or safety of the findings presented.
Keywords
feature disentanglement, generative model safety, manifold preservation, orthogonal projection
Document Type
Conference Proceeding
Date of Publication
1-1-2026
E-ISSN
16113349
ISSN
03029743
Volume
17014 LNCS
Publication Title
Computer Vision – ECCV 2026
Publisher
Springer
School
School of Science
ISBN
[9783032372314]
Copyright
subscription content
First Page
546
Last Page
563
Recommended Citation
Shi, C., Wu, W., Shen, F., Zhu, X., Hu, K., & Wang, Z. (2026). OrthoEraser: Coupled-neuron orthogonal projection for concept erasure. In A. Leonardis, K. Schindler, & T. Tuytelaars (Eds.), Computer vision – ECCV 2026 (pp. 546–563). Springer. https://doi.org/10.1007/978-3-032-37232-1_30