This paper provides quantitative comparison between the text embedding of a text pre-trained model (i.e., RoBERTa) and a multi-modal pre-trained model (i.e., WenLan). Two quantitative comparison methods are proposed, in an embedding space: representing the semantics of a word using the set of ?-nearest words to it and then analyze the semantic changes of the word in the two spaces using the Jaccard similarity of the two sets; forming pairs between each word and its nearest ? words to analyze the relationship. The results show that the multi-modal pre-training brings more semantic changes for more abstract words (e.g., success, love), and the multi-modal pre-trained model can better differentiate antonyms and discover more hypernyms or hyponyms, while text pre-training works better in finding synonyms. Moreover, multi-modal pre-trained model can construct a more extensive associative relationship between words.

为了详细地分析文本单模态预训练模型RoBERTa和图文多模态预训练模型WenLan文本嵌入的差异, 提出两种定量比较方法, 即在任一空间中, 使用距离一个词最近的k近邻词集合表示其语义, 进而通过集合间的Jaccard相似度来分析两个空间中词的语义变化; 将每个词与其k近邻词组成词对, 分析词对之间的关系。实验结果表明, 图文多模态预训练为更抽象的词(如成功和爱情等)带来更多的语义变化, 可以更好地区分反义词, 发现更多的上下义词, 而文本单模态预训练模型更擅长发现同义词。另外, 图文多模态预训练模型能够建立更广泛的词之间的相关关系。