The two-volume set LNCS 6974 and LNCS 6975 constitutes the refereed proceedings of the Fourth International Conference on Affective Computing and Intelligent Interaction, ACII 2011, held in Memphis,TN, USA, in October 2011. The 135 papers in this two volume set presented together with 3 invited talks were carefully reviewed and selected from 196 submissions. The papers are organized in topical sections on recognition and synthesis of human affect, affect-sensitive applications, methodological issues in affective computing, affective and social robotics, affective and behavioral interfaces, relevant insights from psychology, affective databases, Evaluation and annotation tools.