Adapting to Multiple Affective States in Spoken Dialogue