Multimodal Interaction Mechanism of Speech Videos Based on ELAN

GUAN Yue

Volume 37 Issue 5

Oct. 2021

Turn off MathJax

Article Contents

Abstract

References

Article Navigation > Journal of University of Science and Technology Beijing ( Social Sciences Edition) > 2021 > 37(5): 499-510

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

Citation:

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

Citation:

GUAN Yue. Multimodal Interaction Mechanism of Speech Videos Based on ELAN[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 499-510.

PDF( 1159 KB)

Multimodal Interaction Mechanism of Speech Videos Based on ELAN-A Case Study of TED Educational Speech Video

GUAN Yue

College English Education Center, Nanfang College. Guangzhou, Guangzhou 510970, China

Received Date: 2020-09-01
Available Online: 2021-09-13
Publish Date: 2021-10-25

Abstract

Abstract

In recent years, multimodal discourse analysis has become a hot topic in the study of linguistics. Speech video is a typical dynamic multimodal discourse. This paper adopts modal density foreground-background continuum as methodological framework and ELAN multimodal analysis software as the modal annotation tool. According to multimodal interaction analysis theory proposed by Norris, this paper analyzes the role of each communicative mode in the construction of this speech information and the interactive mechanism between these modes in a TED educational speech video. It is found that modal intensity and modal density of each communicative mode in the problem introduction part, problem posing part, problem analysis part and problem solving part of this speech video are different. Spoken language and other communicative modes show the relationship of reinforcement, complementarity and juxtaposition, thereby jointly constructing the meaning of multimodal interaction.
- TED educational speech video,
- multimodal interaction,
- communicative modes,
- ELAN

FullText(HTML)

References(28)

References

[1]	KRESS G & VAN LEEUWEN T. Reading Images: The Grammar of Visual Design [M]. London: Routledge, 1996.
[2]	NORRIS S. Analyzing Multimodal Interaction: A Methodological Framework [M]. New York: Routledge, 2004.
[3]	张佐成, 刑建玉, 王彦, 等. 多模态即席话语研究 [M]. 广州: 世界图书出版公司, 2014.
[4]	KRESS G & VAN LEEUWEN T. Reading Images: The Grammar of Visual Design (2nd ed. )[M]. London: Routledge, 2006.
[5]	GU Y. Multimodal text analysis: a corpus linguistic approach to situated discourse[J]. Text & Talk, 2006, 26(2):127-167.
[6]	O’HALLORAN K L. Multimodal Discourse Analysis: Systemic-Functional Perspectives [M]. London: Continuum, 2004.
[7]	BALDRY A & THIBAULT P. Multimodal Transcription and Text Analysis: A Multimedia Toolkit and Course Book [M]. London: Equinox, 2006.
[8]	李战子. 多模式话语的社会符号学分析[J]. 外语研究,2003(5):1-8. doi: 10.3969/j.issn.1005-7242.2003.05.001
[9]	李战子,陆丹云. 多模态符号学: 理论基础, 研究途径与发展前景[J]. 外语研究,2012(2):1-8. doi: 10.3969/j.issn.1005-7242.2012.02.001
[10]	胡壮麟. 社会符号学研究中的多模态化[J]. 语言教学与研究,2007(1):1-10.
[11]	张德禄. 多模态话语分析综合理论框架探索[J]. 中国外语,2009,6(1):24-30. doi: 10.3969/j.issn.1672-9382.2009.01.006
[12]	张德禄,郭恩华. 多模态话语分析的双重视角−社会符号观与概念隐喻观的连接与互补[J]. 外国语,2013,36(3):20-28.
[13]	张德禄. 系统功能理论视阈下的多模态话语分析综合框架[J]. 现代外语,2018,41(6):731-743.
[14]	梁兵,蒋平. 旅游语篇多模态话语分析与中国文化对外传播[J]. 外语学刊,2015(2):155-158.
[15]	国防. 多模态话语分析研究热点及趋势分析−基于文献计量学方法[J]. 外语与外语教学,2016(3):58-66,146.
[16]	管乐. 动态演讲视频的多模态话语分析[J]. 浙江理工大学学报(社会科学版),2017,38(1):25-32.
[17]	RENDLE-SHORT J. Managing the transition between talk and silence in the academic monologue[J]. Research on Language and Social Interaction, 2005(38):179-218.
[18]	MORTON J. Genre and disciplinary competence: a case study of contextualization in an academic speech genre[J]. English for Specific Purposes, 2009(28):217-229.
[19]	TARDY C M. Building Genre Knowledge [M]. West Lafayette, Indiana: Parlor Press, 2009.
[20]	赵燕飞. 运用TED演讲辅助英语视听说教学[J]. 教学与管理,2016(21):103-106.
[21]	李颖. 虚拟现实技术在英语演讲中的实训原理分析[J]. 中国外语,2018,15(6):73-78.
[22]	刘森,吴鸿皓. “同一”理论对英语语篇教学的启示−基于美国总统就职演讲的修辞分析[J]. 外语电化教学,2019(4):67-72, 80.
[23]	林岩. 英语演讲教学中的思辨能力培养−“说服策略”单元课例研究[J]. 中国外语,2020,17(3):63-69.
[24]	NORRIS S. Modal density and modal configurations: multimodal actions [C] // Jewitt C. The Routledge Handbook of Multimodal Analysis. London: Routledge, 2009: 78-90.
[25]	NORRIS S. Identity In (Inter)action——Introducing Multimodal (Inter)action Analysis [M]. Berlin: De Gruyter Mouton, 2011.
[26]	NORRIS S. The micropolitics of personal national and ethnicity identity[J]. Discourse and Society, 2007, 18(5):653-674. doi: 10.1177/0957926507079633
[27]	王立非,文艳. 应用语言学研究的多模态分析方法[J]. 外语电化教学,2008(3):8-12. doi: 10.3969/j.issn.1001-5795.2008.03.002
[28]	管乐. 多模态话语分析的新发展−《多模态互动分析》述评[J]. 语言教育,2017,5(1):85-90.

Relative Articles

[1]	ZENG Lei, ZHU Xinyu. The Study of Resemiotisation System of Projection: Taking Bilingual Picture Book of Children’s Literature as an Example[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2023, 39(5): 554-562. doi: 10.19979/j.cnki.issn10082689.2022100006
[2]	FAN Li. Multimodal Low-carbon Discourse：A Case Study of Promotional Materials for National Low Carbon Day (2020−2022)[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2023, 39(6): 739-753. doi: 10.19979/j.cnki.issn10082689.2023020107
[3]	LIU Qinyu, WU Yaxin. A Multi-modal Conversation Analytic Study on Child-initiated Directives in Family Iteractions[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2023, 39(3): 289-298. doi: 10.19979/j.cnki.issn10082689.2022110091
[4]	ZHOU Jin-you. A Contrastive Study of Disciplinary Knowledge-building from the Perspective of Multimodality[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2022, 38(6): 685-694. doi: 10.19979/j.cnki.issn10082689.2022100079
[5]	WANG Qin. Exploring the Multimodal Semiotic Design of EFL Digital Textbook:A Case Study on “English for the Humanities”[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(2): 154-159.
[6]	LI Xiao-hua, TANG Qing-ye. A Visualized Overview of Domestic Research on Multimodal Translation: Status Quo, Problems and Suggestions[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 534-542.
[7]	LI Zhan-zi, FAN Bing-bing. Construction of Multimodal Corpus and Its Pedagogic Application: A Case Analysis of Text-image Typology in “China-US Focus” Website[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 511-517.
[8]	ZENG Lei, ZHU Xin-yu. The Realization of Ideational Meaning of Multimodal Projection in Sinology Comics[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2020, 36(6): 1-9.
[9]	XIAO Hao-zhang, YANG Ying-ying. An Eco-analysis of the Multimodal Effects in China's EFL Education[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2018, 34(6): 1-9.
[10]	SUN Yi, ZHOU Jing. Interpretation of National Flag Multimodal Metaphor across the World[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2016, 32(5): 1-7.
[11]	ZHANG Chun-yan, CHEN Si-yi-ye. The Construe of Interactive Meaning in Multimodal Electronic Dictionaries—Taking LDOCEP for IOS as an Example[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2016, 32(3): 15-20.
[12]	CENG Lei. The Study of the Multimodal Discourse in College English Teaching[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2014, 30(5): 9-15.
[13]	WANG Jia. The Interaction of Multi-metaphor and Multi-metonymy in Political Cartoons[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2014, 30(6): 20-26.
[14]	CHEN Ming-yao, XIE Li. On Cognitive and Emotional Persuasion of Multimodal Metaphor[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 73-77.
[15]	TANG Qing-ye. The Multimodal Representation of Poverty[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 60-66.
[16]	LI Li. Paraphrasing Humor in Multi-modality[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 52-59.
[17]	PENG Yi, YANG Hai-yan. The Construal of the Integrative Meaning of Multimodal Commercial Advertising Discourse[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 1-6.
[18]	FENG De-zheng. Digital Technology and Multimodal Discourse Analysis[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 12-17.
[19]	ZHAO Xiu-feng, SU Hui-yan. Cognitive Meaning Construction of Metaphorical Multimodal Discourse——Blending on the Interplay of Multimetonymy and Multimetaphor[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 18-24,30.
[20]	CHEN Yu-min. Multimodal Construal of Appraisal: A Case Study of Olympics Television Public Service Advertising[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2008, 24(3): 108-114.

Supplements(0)

Cited By

Cited by

Periodical cited type(5)

1.	张亮，张伟英，周福战，于涛，李盛之，金博，魏小鹏. 主持传播多模态自洽呈现的计算范式研究. 新闻春秋. 2025(01): 50-61 .
2.	冯丹娃，李丹宇. 新文科背景下学生课堂参与度评价研究——基于讨论式教学视频的分析. 教育学术月刊. 2024(08): 67-76 .
3.	马中红，井常灿. 编织童话：基于多模态互动分析的数字游民在线叙事与身份认同. 新闻与写作. 2024(12): 26-37 .
4.	郑群，厉沁怡. 三分钟学术英语演讲中获奖组与非获奖组多模态介入及模态协同性对比研究. 中国ESP研究. 2023(03): 23-32+114-115 .
5.	魏小云. 英语TED教育类演讲的概念功能分析. 英语广场. 2022(23): 43-45 .