Security Challenge in Open Models
Embedding robust safeguards into large language models is essential for helpful and harmless responses.
- Proprietary systems demonstrate robust safety control.
- Their methods and trade-offs remain largely undisclosed.
- Achieving comparable safety in open models remains a persistent challenge.
- Post-trained variants often suffer from excessive refusals and a drop in overall quality.
Suan is a new preference optimization algorithm. It was created to overcome these shortcomings. Practical criterion: if an open model refuses too often and loses quality, a method is needed that corrects safety alignment without sacrificing helpfulness.
Direct Gradient Optimization
Suan formulates the optimization objective directly at the gradient level. This approach bypasses standard variational inference. As a result, training becomes more interpretable and stable.
| Aspect | What Suan changes relative to the standard approach |
|---|---|
| Objective formulation | Direct gradient level instead of standard variational inference |
| Training dynamics | More interpretable and stable |
The selection criterion is to look at where the objective is formulated. The direct gradient level yields more transparent training dynamics.
Evaluation and Results
Suan underwent extensive evaluations. They cover a diverse set of competitive baselines and benchmarks.
- The algorithm achieves excellent safety alignment.
- The algorithm fully preserves response helpfulness.
Practical criterion: compare safety and helpfulness together. Evaluating only refusals does not show overall quality. Evaluating only quality does not show protection.
Output Data
The paper is titled "Suan: Rectifying Direct Preference Safety Alignment in Large Language Models."
- Authors — Oleksandr Cherednichenko, Roman Klypa.
- arXiv:2609.08634 (cs).
- Submission date — September 8, 2026.
- Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI).
- DOI: https://doi.org/10.48550/arXiv.2609.08634 (pending registration).
- Submission history: from Oleksandr Cherednichenko; [v1] Tue, 8 Sep 2026 12:04:38 UTC (426 KB).
- Cite as arXiv:2609.08634 [cs.LG] (or arXiv:2609.08634v1 [cs.LG] for this version).
Practical criterion: use the arXiv ID and DOI for citation. The DOI is currently pending registration.
