I think the difference in this instance is that ailerons deflect
airflow upward wheras spoilers actually interrupt lift