ASI09 - Human-Agent Trust Exploitation
Exploiting the persuasive nature of agents to manipulate users into unsafe actions.
Intent & Description
'
🎯 Intent
Prevent agents from being weaponized to manipulate human users through social engineering or deceptive interactions.
📋 Context
Agents can be highly persuasive and build trust with users over time. Compromised agents can exploit this trust to manipulate users into revealing sensitive information or taking harmful actions.
💡 Solution
Implement transparency in agent capabilities and limitations. Require independent verification for high-stakes decisions. Add friction for irreversible actions. Monitor for manipulation patterns. Educate users about AI limitations.'
Real-world Use Case
Source
📌 TL;DR
Prevent trust exploitation. Be transparent about AI limits, verify high-stakes decisions, add friction for irreversible actions.
Advantages
- Protects users from manipulation
- Builds appropriate trust calibration
- Supports ethical AI deployment
- Reduces social engineering risk
Disadvantages
- Transparency may reduce engagement
- Friction may impact user experience
- Trust calibration is subjective