Multimodal Interaction Mechanism of Speech Videos Based on ELAN

GUAN Yue

Oct. 2021

Article Contents

Article Navigation > Journal of University of Science and Technology Beijing ( Social Sciences Edition) > 2021 > 37(5): 499-510

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

Citation:

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

Citation:

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

PDF( 1159 KB)

Multimodal Interaction Mechanism of Speech Videos Based on ELAN-A Case Study of TED Educational Speech Video

GUAN Yue

College English Education Center, Nanfang College. Guangzhou, Guangzhou 510970, China

Received Date: 2020-09-01
Available Online: 2021-09-13
Publish Date: 2021-10-25

Abstract

Abstract

In recent years, multimodal discourse analysis has become a hot topic in the study of linguistics. Speech video is a typical dynamic multimodal discourse. This paper adopts modal density foreground-background continuum as methodological framework and ELAN multimodal analysis software as the modal annotation tool. According to multimodal interaction analysis theory proposed by Norris, this paper analyzes the role of each communicative mode in the construction of this speech information and the interactive mechanism between these modes in a TED educational speech video. It is found that modal intensity and modal density of each communicative mode in the problem introduction part, problem posing part, problem analysis part and problem solving part of this speech video are different. Spoken language and other communicative modes show the relationship of reinforcement, complementarity and juxtaposition, thereby jointly constructing the meaning of multimodal interaction.
- TED educational speech video,
- multimodal interaction,
- communicative modes,
- ELAN

FullText(HTML)

References(28)

References

[1]	KRESS G & VAN LEEUWEN T. Reading Images: The Grammar of Visual Design [M]. London: Routledge, 1996.
[2]	NORRIS S. Analyzing Multimodal Interaction: A Methodological Framework [M]. New York: Routledge, 2004.
[3]	张佐成, 刑建玉, 王彦, 等. 多模态即席话语研究 [M]. 广州: 世界图书出版公司, 2014.
[4]	KRESS G & VAN LEEUWEN T. Reading Images: The Grammar of Visual Design (2nd ed. )[M]. London: Routledge, 2006.
[5]	GU Y. Multimodal text analysis: a corpus linguistic approach to situated discourse[J]. Text & Talk, 2006, 26(2):127-167.
[6]	O’HALLORAN K L. Multimodal Discourse Analysis: Systemic-Functional Perspectives [M]. London: Continuum, 2004.
[7]	BALDRY A & THIBAULT P. Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Course Book [M]. London: Equinox, 2006.
[8]	李战子. 多模式话语的社会符号学分析[J]. 外语研究,2003(5):1-8. doi: 10.3969/j.issn.1005-7242.2003.05.001
[9]	李战子,陆丹云. 多模态符号学: 理论基础, 研究途径与发展前景[J]. 外语研究,2012(2):1-8. doi: 10.3969/j.issn.1005-7242.2012.02.001
[10]	胡壮麟. 社会符号学研究中的多模态化[J]. 语言教学与研究,2007(1):1-10.
[11]	张德禄. 多模态话语分析综合理论框架探索[J]. 中国外语,2009,6(1):24-30. doi: 10.3969/j.issn.1672-9382.2009.01.006
[12]	张德禄,郭恩华. 多模态话语分析的双重视角−社会符号观与概念隐喻观的连接与互补[J]. 外国语,2013,36(3):20-28.
[13]	张德禄. 系统功能理论视阈下的多模态话语分析综合框架[J]. 现代外语,2018,41(6):731-743.
[14]	梁兵,蒋平. 旅游语篇多模态话语分析与中国文化对外传播[J]. 外语学刊,2015(2):155-158.
[15]	国防. 多模态话语分析研究热点及趋势分析−基于文献计量学方法[J]. 外语与外语教学,2016(3):58-66,146.
[16]	管乐. 动态演讲视频的多模态话语分析[J]. 浙江理工大学学报(社会科学版),2017,38(1):25-32.
[17]	RENDLE-SHORT J. Managing the transition between talk and silence in the academic monologue[J]. Research on Language and Social Interaction, 2005(38):179-218.
[18]	MORTON J. Genre and disciplinary competence: a case study of contextualization in an academic speech genre[J]. English for Specific Purposes, 2009(28):217-229.
[19]	TARDY C M. Building Genre Knowledge [M]. West Lafayette, Indiana: Parlor Press, 2009.
[20]	赵燕飞. 运用TED演讲辅助英语视听说教学[J]. 教学与管理,2016(21):103-106.
[21]	李颖. 虚拟现实技术在英语演讲中的实训原理分析[J]. 中国外语,2018,15(6):73-78.
[22]	刘森,吴鸿皓. “同一”理论对英语语篇教学的启示−基于美国总统就职演讲的修辞分析[J]. 外语电化教学,2019(4):67-72, 80.
[23]	林岩. 英语演讲教学中的思辨能力培养−“说服策略”单元课例研究[J]. 中国外语,2020,17(3):63-69.
[24]	NORRIS S. Modal density and modal configurations: multimodal actions [C] // Jewitt C. The Routledge Handbook of Multimodal Analysis. London: Routledge, 2009: 78-90.
[25]	NORRIS S. Identity In (Inter)action——Introducing Multimodal (Inter)action Analysis [M]. Berlin: De Gruyter Mouton, 2011.
[26]	NORRIS S. The micropolitics of personal national and ethnicity identity[J]. Discourse and Society, 2007, 18(5):653-674. doi: 10.1177/0957926507079633
[27]	王立非,文艳. 应用语言学研究的多模态分析方法[J]. 外语电化教学,2008(3):8-12. doi: 10.3969/j.issn.1001-5795.2008.03.002
[28]	管乐. 多模态话语分析的新发展−《多模态互动分析》述评[J]. 语言教育,2017,5(1):85-90.