Corpus-based Study on the Levels of Multimodal Affordance Transformations: From the Perspective of Context Transfer

HAN Yanfang

doi:10.19979/j.cnki.issn10082689.2023100109

Volume 40 Issue 6

Dec. 2024

Turn off MathJax

Article Contents

Abstract

References

Article Navigation > Journal of University of Science and Technology Beijing ( Social Sciences Edition) > 2024 > 40(6): 60-70

HAN Yanfang. Corpus-based Study on the Levels of Multimodal Affordance Transformations: From the Perspective of Context Transfer[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2024, 40(6): 60-70. doi: 10.19979/j.cnki.issn10082689.2023100109

Citation:

HAN Yanfang. Corpus-based Study on the Levels of Multimodal Affordance Transformations: From the Perspective of Context Transfer[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2024, 40(6): 60-70. doi: 10.19979/j.cnki.issn10082689.2023100109

Citation:

PDF( 1568 KB)

Corpus-based Study on the Levels of Multimodal Affordance Transformations: From the Perspective of Context Transfer

doi: 10.19979/j.cnki.issn10082689.2023100109

HAN Yanfang

School of International Studies, Zhengzhou University, Zhengzhou 450001, China

Received Date: 2023-10-31
Publish Date: 2024-12-25

Abstract

Abstract

Multimodal semiotic construction, a prominent area of academic research, has undergone continuous development in both theoretical understanding and practical applications. However, there is a notable gap in our understanding of multimodal affordance transformation and its hierarchical aspects, which necessitates further exploration. This study, taking a contextual shift perspective and using the Olympic emblem symbol corpus as a foundation, investigates the levels and functions of multimodal affordance meaning transformation, focusing on genre elements, layout, and symbol selection. The research findings suggest that, under the context transfer (1) transformations in genre elements, layout, and semiotic choices display consistent patterns; (2) the use of multimodal affordance meaning transformation across different time periods is interconnected, and the symbolic features accumulated in the context are migrated to the new emblem, with semiotic hallmarks and optimal choices made through dimensional shifts, impacting knowledge shaping, role definition, and coherence; (3) intensifying of the degree of multimodal symbol meaning transformation, diminishing of the transparency of contextual shifts, and increasing of the abstraction level of affordance transformation, indicate that affordance meaning transformation is not just a contextualization process for transferring symbol resources but also a mechanism for symbol meaning innovation. Additionally, this study introduces a framework for understanding affordance transformation in social practices, providing valuable insights for analyzing the phenomena resulting from affordance transformation.
- multimodal affordance,
- context transfer,
- transformation,
- multimodal corpus

FullText(HTML)

References(34)

References

[1]	BATEMAN J A,WILDFEUER J & HIIPPALA T. Multimodality:Foundations,Research and Analysis:A Problem Oriented Introduction [M]. Berlin:De Gruyter Mouton. 2017.
[2]	张德禄,胡瑞云. 多模态话语建构中的系统、选择与供用特征[J]. 当代修辞学,2019(5):68-79.
[3]	FORCEVILLE C. Visual and Multimodal Communication:Applying Relevance Principle [M]. New York and London:Oxford University Press,2020.
[4]	张德禄. 多模态话语中符号意义构成研究[J]. 当代修辞学,2023(3):24-36.
[5]	GIBSON J J. The Ecological Approach to Visual Perception (Classic Edition) [M]. Psychology Press:New York and London,2014.
[6]	NORMAN D A. The Design of Everyday Things [M]. Doubleday:New York,1988.
[7]	NORMAN D A. Affordances,conventions and design [J]. Interactions,1999,6(3):38-43. doi: 10.1145/301153.301168
[8]	KRESS G. Against arbitrariness:The social production of the sign as a foundational issue in critical discourse analysis [J]. Discourse & Society,1993,4(2):169-191.
[9]	HALLIDAY M A K & MATTHIESSEN C M I M. Construing Experience Through Meaning:A Language-based Approach to Cognition [M]. London:Cotinuum,1999.
[10]	韩艳方. 多模态话语建构中模态协同的多维分析:系统功能视角[J]. 外语学刊,2022(1):35-40.
[11]	NORMAN D A. The Design of Everyday Things(Revised and Expanded Edition) [M]. New York:Basic Books,2013.
[12]	ELLESTROM L. Media Transformation:The Transfer of Media Characteristics Among Media [M]. Basingstoke:Palgrave Macmillan,2014.
[13]	ELLESTROM L. Transmedial Narration Narratives and Stories in Different Media [M]. Basingstoke:Palgrave Macmillan,2019.
[14]	KRESS G &VAN LEEUWEN T. Reading Images:the Grammar of Visual Design [M]. London:Routledge,1996.
[15]	张德禄,吴连春. 再语境化在英语阅读课培养学生核心素养中的作用:社会教育学视角[J]. 西安外国语大学学报,2021,29(1):48-53.
[16]	苗兴伟,李珂. 再语境化视角下企业生态身份的话语建构[J]. 外语教学,2021,42(2):1-6.
[17]	曾蕾,朱薪羽. 多模态视角下投射意义的再符号化研究——以儿童文学双语绘本为例[J]. 北京科技大学学报(社会科学版),2023,39(5):554-562.
[18]	BERNSTEIN B. Class,Codes and Control:Theoretical studies towards a sociology of language [M]. London:Routledge and Kegan Paul,1971.
[19]	RICHARDSON J E & WODARK R. Recontextualising fascist ideologies of the past:Right-wing discourses on employment and nativism in Austria and the United Kingdom [J]. Critical Discourse Studies,2009,6(4):251-267. doi: 10.1080/17405900903180996
[20]	LINELL P. Discourse across boundaries:on recontextualisations and the blending of voices in professional discourse [J]. Text,1998,18(2):143-158.
[21]	FLOWERDEW J & RICHARDSON J E. The Routledge Handbook of Critical Discourse Studies [M]. London:Routledge,2018.
[22]	HANSAN R. Towards a paradigmatic description of context:Systems,metafunctions,and semantics [J]. Functional Linguistics,2014(1):1-54.
[23]	HALLIDAY M A K. Introduction:on the “Architecture” of human language [C] // WEBSTER J. On Language and Linguistics. London; New York:Continuum,2003:1-32.
[24]	BALDRY A & THIBAULT P J. Multimodal Transcription and Text Analysis. A Multimedia Toolkit and Coursebook [M]. London:Equinox,2006.
[25]	IEDEMA R. Resemiotization [J]. Semiotica,2001,137(1):23-39.
[26]	IEDEMA R. Multimodality,resemiotisation:Extending the analysis of discourse as multi-semiotic practice [J]. Visual Communication,2003,2(1):29-57. doi: 10.1177/1470357203002001751
[27]	BATEMAN J A. Multimodality and Genre:A Foundation for Systematic Analysis of Multimodal Documents [M]. New York:Palgrave MacMillan,Springer,2008.
[28]	BOWCHER W L & LIANG J Y. GSP and multimodal texts [C] // BOWCHER W L & LIANG J Y. Society in Language, Language in Society. London: Palgrave Macmillan, 2016: 251-274. BOWCHER W L & LIANG J Y. GSP and multimodal texts [C] // BOWCHER W L & LIANG J Y. Society in Language,Language in Society. London:Palgrave Macmillan,2016:251-274.
[29]	HASAN R. The nursery tale as a genre [C] // CLORAN C,BUTT D & WILLIAMS G. Ways of Saying:Ways of Meaning. London:Cassell,1996:51-72.
[30]	BATEMAN J A,DELI J & HENSCHEL R. Mapping the multimodal genres for traditional and electronic newspapers [C] // ROYCE T & BOWCHER W L. New Directions in the Analysis of Multimodal Discourse. Mahwah:Lawrence Erlbaum Associates. 2007:147-172.
[31]	Olympic Charter [Z/OL]. (1996-07-18)[2020-09-01]. https://www.olympic.org/documents/olympic-charter.
[32]	VAN LEEUWEN T. Discourse and Practice:New Tools for Critical Discourse Analysis [M]. New York and London:Oxford University Press,2008.
[33]	LEMEKE J L. Discourse,dynamics,and social change [J]. Cultural Dynamics,1993,6(1-2):243-257. doi: 10.1177/092137409300600107
[34]	张德禄. 文化语境在多模态话语建构中的作用研究[J]. 解放军外国语学院学报,2023(5):54-61,161.

Relative Articles

[1]	FAN Li. Multimodal Low-carbon Discourse：A Case Study of Promotional Materials for National Low Carbon Day (2020−2022)[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2023, 39(6): 739-753. doi: 10.19979/j.cnki.issn10082689.2023020107
[2]	ZENG Lei, ZHU Xinyu. The Study of Resemiotisation System of Projection: Taking Bilingual Picture Book of Children’s Literature as an Example[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2023, 39(5): 554-562. doi: 10.19979/j.cnki.issn10082689.2022100006
[3]	YU Hui, SONG Jin-ge. A Multimodal Approach to the Social Semiotic Construction of Ecological Meanings in Children’s Picture Book[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2022, 38(6): 667-676. doi: 10.19979/j.cnki.issn10082689.2022100067
[4]	WU Yuan-zheng, FENG De-zheng. The Attitudinal Meaning and Value Construction in Multimodal Ecological Discourse[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2022, 38(6): 677-684. doi: 10.19979/j.cnki.issn10082689.2022100139
[5]	LI Zhan-zi, FAN Bing-bing. Construction of Multimodal Corpus and Its Pedagogic Application: A Case Analysis of Text-image Typology in “China-US Focus” Website[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2021, 37(5): 511-517.
[6]	ZENG Lei, ZHU Xin-yu. The Realization of Ideational Meaning of Multimodal Projection in Sinology Comics[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2020, 36(6): 1-9.
[7]	XIAO Hao-zhang, YANG Ying-ying. An Eco-analysis of the Multimodal Effects in China's EFL Education[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2018, 34(6): 1-9.
[8]	TANG Lei. The Application of a Spoken Language Corpus Based on Information Embodied in Conversations: Practical Use and Evaluation of Attributes and Multimodal Information[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2018, 34(6): 64-71.
[9]	ZHANG Chun-yan, CHEN Si-yi-ye. The Construe of Interactive Meaning in Multimodal Electronic Dictionaries—Taking LDOCEP for IOS as an Example[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2016, 32(3): 15-20.
[10]	SUN Yi, ZHOU Jing. Interpretation of National Flag Multimodal Metaphor across the World[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2016, 32(5): 1-7.
[11]	HUANG Li-he. Situated Discourse and Multimodal Studies: Significance, Theory and Methodology[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2016, 32(5): 29-36.
[12]	WANG Jia. The Interaction of Multi-metaphor and Multi-metonymy in Political Cartoons[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2014, 30(6): 20-26.
[13]	CENG Lei. The Study of the Multimodal Discourse in College English Teaching[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2014, 30(5): 9-15.
[14]	CHEN Ming-yao, XIE Li. On Cognitive and Emotional Persuasion of Multimodal Metaphor[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 73-77.
[15]	TANG Qing-ye. The Multimodal Representation of Poverty[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 60-66.
[16]	LI Li. Paraphrasing Humor in Multi-modality[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2012, 28(4): 52-59.
[17]	FENG De-zheng. Digital Technology and Multimodal Discourse Analysis[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 12-17.
[18]	PENG Yi, YANG Hai-yan. The Construal of the Integrative Meaning of Multimodal Commercial Advertising Discourse[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 1-6.
[19]	ZHAO Xiu-feng, SU Hui-yan. Cognitive Meaning Construction of Metaphorical Multimodal Discourse——Blending on the Interplay of Multimetonymy and Multimetaphor[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2010, 26(4): 18-24,30.
[20]	CHEN Yu-min. Multimodal Construal of Appraisal: A Case Study of Olympics Television Public Service Advertising[J]. Journal of University of Science and Technology Beijing ( Social Sciences Edition), 2008, 24(3): 108-114.