基于语料库的多模态供用意义转换层级研究：语境迁移视角

韩艳方

doi:10.19979/j.cnki.issn10082689.2023100109

基于语料库的多模态供用意义转换层级研究：语境迁移视角

doi: 10.19979/j.cnki.issn10082689.2023100109

韩艳方

郑州大学外国语与国际关系学院，郑州 450001

基金项目: 国家哲学社会科学项目“多模态话语建构中模态配置的机制和机理研究”（编号：22BYY002）；河南省高校人文社科一般项目“智媒时代公共生态教育的超模态话语建构及其数字修辞策略”（编号：2024-ZZJH-359）。

详细信息

作者简介:
韩艳方（1989—），男，河南许昌人，郑州大学外国语与国际关系学院讲师

中图分类号: H030
计量
- 文章访问数: 28
- HTML全文浏览量: 210
- PDF下载量: 4
- 被引次数: 0
出版历程
- 收稿日期: 2023-10-31
- 刊出日期: 2024-12-25

Corpus-based Study on the Levels of Multimodal Affordance Transformations: From the Perspective of Context Transfer

HAN Yanfang

School of International Studies, Zhengzhou University, Zhengzhou 450001, China

摘要

摘要: 多模态符号建构作为学界研究热点，尽管理论描写和实践应用不断丰富，但多模态供用意义转换及其层级缺乏关注，亟待讨论。鉴于此，文章在语境迁移视角下，基于奥运会徽符号语料库，从体裁成分、布局、符号选择三个层面，探讨了多模态供用意义转换层级和功能。研究表明，在语境迁移下，体裁成分、布局与具体符号的供用性选择转换呈现较为一致的变化。不同时期多模态供用意义转换的使用具有相关性，语境累积的符号特征迁移到新会徽中，通过维度转换的方式获取最佳选择，实现需要供用意义转换参与的知识塑造、角色定位和连贯强化。从历时看，多模态符号意义转换程度逐渐增强，语境迁移的透过性影响逐渐减弱，供用性转换的抽象程度逐渐上升，表明符号供用意义转换既是符号资源的迁移性语境化过程也是符号意义创新的过程。文章还结合供用系统的选择层次提出了社会实践中的供用意义转换层级，为解析供用意义转换产生的符号现象提供了理据。
- 多模态供用意义 /
- 语境迁移 /
- 转换层级 /
- 多模态语料库
Abstract: Multimodal semiotic construction, a prominent area of academic research, has undergone continuous development in both theoretical understanding and practical applications. However, there is a notable gap in our understanding of multimodal affordance transformation and its hierarchical aspects, which necessitates further exploration. This study, taking a contextual shift perspective and using the Olympic emblem symbol corpus as a foundation, investigates the levels and functions of multimodal affordance meaning transformation, focusing on genre elements, layout, and symbol selection. The research findings suggest that, under the context transfer (1) transformations in genre elements, layout, and semiotic choices display consistent patterns; (2) the use of multimodal affordance meaning transformation across different time periods is interconnected, and the symbolic features accumulated in the context are migrated to the new emblem, with semiotic hallmarks and optimal choices made through dimensional shifts, impacting knowledge shaping, role definition, and coherence; (3) intensifying of the degree of multimodal symbol meaning transformation, diminishing of the transparency of contextual shifts, and increasing of the abstraction level of affordance transformation, indicate that affordance meaning transformation is not just a contextualization process for transferring symbol resources but also a mechanism for symbol meaning innovation. Additionally, this study introduces a framework for understanding affordance transformation in social practices, providing valuable insights for analyzing the phenomena resulting from affordance transformation.
- multimodal affordance /
- context transfer /
- transformation /
- multimodal corpus
注释:

1) 1哈桑^[23]总结GSP的发展特征时，对该术语的运用进行了解释，即“早期采用的是‘general/ized formula’；随后使用了术语‘generic’，指与种类/类型相关的。在巴赫金的‘speech genre’流行时，其又与“schematic structures”相关。与GSP不同，schematic structures是‘单个的’，这符合‘动态视角’，因为其关注的不是类型，而是每个文本的过程。schematic structures与语境配置特征没有明显的关联。因此，术语Generalized Formula一直在‘Generalized Structure Potential’和‘Generic Structure Potential’之间游离。”

2) 2这里的前景化和背景化主要与视觉结构相关，通过某一体裁成分或符号的“越位”或“托底”偏离常规视觉结构，即视觉结构的供用意义转换。借助它们来突出上中下的某一个布局位置或整个上中下布局。

HTML全文

图 1 会徽体裁的供用意义选择系统图

下载: 全尺寸图片幻灯片

图 2 OLC体裁成分历时分布图

下载: 全尺寸图片幻灯片

图 3 OLC体裁布局选择总体分布图

下载: 全尺寸图片幻灯片

图 4 OLC符号选择分布图

下载: 全尺寸图片幻灯片

图 5 多模态供用意义转换图

下载: 全尺寸图片幻灯片

图 6 奥运会徽体裁的供用性原型图

下载: 全尺寸图片幻灯片

图 7 话语实践中的多模态符号供用意义转换层级图

下载: 全尺寸图片幻灯片

参考文献(34)

[1]	BATEMAN J A,WILDFEUER J & HIIPPALA T. Multimodality:Foundations,Research and Analysis:A Problem Oriented Introduction [M]. Berlin:De Gruyter Mouton. 2017.
[2]	张德禄,胡瑞云. 多模态话语建构中的系统、选择与供用特征[J]. 当代修辞学,2019(5):68-79.
[3]	FORCEVILLE C. Visual and Multimodal Communication:Applying Relevance Principle [M]. New York and London:Oxford University Press,2020.
[4]	张德禄. 多模态话语中符号意义构成研究[J]. 当代修辞学,2023(3):24-36.
[5]	GIBSON J J. The Ecological Approach to Visual Perception (Classic Edition) [M]. Psychology Press:New York and London,2014.
[6]	NORMAN D A. The Design of Everyday Things [M]. Doubleday:New York,1988.
[7]	NORMAN D A. Affordances,conventions and design [J]. Interactions,1999,6(3):38-43. doi: 10.1145/301153.301168
[8]	KRESS G. Against arbitrariness:The social production of the sign as a foundational issue in critical discourse analysis [J]. Discourse & Society,1993,4(2):169-191.
[9]	HALLIDAY M A K & MATTHIESSEN C M I M. Construing Experience Through Meaning:A Language-based Approach to Cognition [M]. London:Cotinuum,1999.
[10]	韩艳方. 多模态话语建构中模态协同的多维分析:系统功能视角[J]. 外语学刊,2022(1):35-40.
[11]	NORMAN D A. The Design of Everyday Things(Revised and Expanded Edition) [M]. New York:Basic Books,2013.
[12]	ELLESTROM L. Media Transformation:The Transfer of Media Characteristics Among Media [M]. Basingstoke:Palgrave Macmillan,2014.
[13]	ELLESTROM L. Transmedial Narration Narratives and Stories in Different Media [M]. Basingstoke:Palgrave Macmillan,2019.
[14]	KRESS G &VAN LEEUWEN T. Reading Images:the Grammar of Visual Design [M]. London:Routledge,1996.
[15]	张德禄,吴连春. 再语境化在英语阅读课培养学生核心素养中的作用:社会教育学视角[J]. 西安外国语大学学报,2021,29(1):48-53.
[16]	苗兴伟,李珂. 再语境化视角下企业生态身份的话语建构[J]. 外语教学,2021,42(2):1-6.
[17]	曾蕾,朱薪羽. 多模态视角下投射意义的再符号化研究——以儿童文学双语绘本为例[J]. 北京科技大学学报(社会科学版),2023,39(5):554-562.
[18]	BERNSTEIN B. Class,Codes and Control:Theoretical studies towards a sociology of language [M]. London:Routledge and Kegan Paul,1971.
[19]	RICHARDSON J E & WODARK R. Recontextualising fascist ideologies of the past:Right-wing discourses on employment and nativism in Austria and the United Kingdom [J]. Critical Discourse Studies,2009,6(4):251-267. doi: 10.1080/17405900903180996
[20]	LINELL P. Discourse across boundaries:on recontextualisations and the blending of voices in professional discourse [J]. Text,1998,18(2):143-158.
[21]	FLOWERDEW J & RICHARDSON J E. The Routledge Handbook of Critical Discourse Studies [M]. London:Routledge,2018.
[22]	HANSAN R. Towards a paradigmatic description of context:Systems,metafunctions,and semantics [J]. Functional Linguistics,2014(1):1-54.
[23]	HALLIDAY M A K. Introduction:on the “Architecture” of human language [C] // WEBSTER J. On Language and Linguistics. London; New York:Continuum,2003:1-32.
[24]	BALDRY A & THIBAULT P J. Multimodal Transcription and Text Analysis. A Multimedia Toolkit and Coursebook [M]. London:Equinox,2006.
[25]	IEDEMA R. Resemiotization [J]. Semiotica,2001,137(1):23-39.
[26]	IEDEMA R. Multimodality,resemiotisation:Extending the analysis of discourse as multi-semiotic practice [J]. Visual Communication,2003,2(1):29-57. doi: 10.1177/1470357203002001751
[27]	BATEMAN J A. Multimodality and Genre:A Foundation for Systematic Analysis of Multimodal Documents [M]. New York:Palgrave MacMillan,Springer,2008.
[28]	BOWCHER W L & LIANG J Y. GSP and multimodal texts [C] // BOWCHER W L & LIANG J Y. Society in Language, Language in Society. London: Palgrave Macmillan, 2016: 251-274. BOWCHER W L & LIANG J Y. GSP and multimodal texts [C] // BOWCHER W L & LIANG J Y. Society in Language,Language in Society. London:Palgrave Macmillan,2016:251-274.
[29]	HASAN R. The nursery tale as a genre [C] // CLORAN C,BUTT D & WILLIAMS G. Ways of Saying:Ways of Meaning. London:Cassell,1996:51-72.
[30]	BATEMAN J A,DELI J & HENSCHEL R. Mapping the multimodal genres for traditional and electronic newspapers [C] // ROYCE T & BOWCHER W L. New Directions in the Analysis of Multimodal Discourse. Mahwah:Lawrence Erlbaum Associates. 2007:147-172.
[31]	Olympic Charter [Z/OL]. (1996-07-18)[2020-09-01]. https://www.olympic.org/documents/olympic-charter.
[32]	VAN LEEUWEN T. Discourse and Practice:New Tools for Critical Discourse Analysis [M]. New York and London:Oxford University Press,2008.
[33]	LEMEKE J L. Discourse,dynamics,and social change [J]. Cultural Dynamics,1993,6(1-2):243-257. doi: 10.1177/092137409300600107
[34]	张德禄. 文化语境在多模态话语建构中的作用研究[J]. 解放军外国语学院学报,2023(5):54-61,161.