Meta Multi-Task Learning for Sequence Modeling

Meta-knowledge

Metaknowledge of meta-knowledge represents knowledge concerning preselected knowledge.

owing to the varying definitions of knowledge within the subject matter literature, meta-information might be excluded from meta-knowledge. A comprehensive exploration of cognitive, systemic, and epistemic aspects of human knowledge necessitates a clear differentiation among these concepts.

Meta-knowledge serves as a core tool in various research fields such as knowledge engineering and management. It abstracts from local concepts to create unified objects/entities. The scope includes examples like planning strategies or models used for organizing information; also modifications applied to existing domain-specific knowledge. These frameworks must reliably organize different levels of meta-knowledge structures.

Metaknowledge can be automatically extracted from electronic publication archives, aimed at revealing patterns in research, interactions among researchers and institutions, and also identifying conflicting outcomes.

元知识的概念涉及预设知识领域中的信息。

由于不同学科文献对" knowledge"一词定义存在差异性，在这种情况下，元知识内容可能包含或不包含" metadata"。

为了深入理解人类知识的本质及其组织结构（knowledge organization），必须明确区分这些概念之间的区别。

元 knowledge 作为研究和科学领域中的基础工具之一（one of the fundamental tools in research and science），其核心作用在于提供系统性的认知框架。

如用于构建专家系统（expert systems）、数据管理系统（data management systems）或其他涉及 knowledge management 的应用环境。

元 knowledge 被视为一个独立且完整的概念实体（concept entity），基于本地定义的知识体系抽象而来。

第一级个体元 knowledge 具体表现在规划策略制定（strategy formulation）、模型构建过程中的问题识别（problem identification）以及标记技术的应用等方面；此外，在学习过程中域 knowledge 的更新也是其中的重要组成部分。

元 knowledge 可以通过分析电子出版物档案（digital publication archives）自动提取出来（extracted automatically），从而揭示研究模式（research patterns）、研究人员与机构间的关系网络（inter-institutional relationships）以及相互矛盾的结果（contradictory outcomes）。

Multi-task learning

Multi-task learning, as introduced by Caruana (1997), represents a methodology for acquiring multiple related tasks concurrently, thereby substantially enhancing performance compared to independent task learning. Overcoming the main challenge of multi-task learning lies in determining an effective way to establish a shared framework.

LSTM without peep-hole connections

The paper by Graves, A., in 2013, titled 'Generating sequences with recurrent neural networks,' is an arXiv preprint from arXiv:1308.0850.

Text Classification

用softmax来预测

Sequence Tagging

(CRF) conditional random field 作为output layer

Lafferty, J. D., McCallum, A., and Pereira, F. C. N. 2001.
Exploring probabilistic approaches for the segmentation and tagging of sequential data.
At the International Conference on Machine Learning (ICML) in 2001.

Meta learning Lstm

https://github.com/catnlp/metaLSTM

The General Framework of Multi-Task Learning vs. The Advanced System Structure of Meta-Multi-Task Learning

共享层控制私有层输入

共享层控制私有层模型参数

Singular Value Decomposition

visdom报错问题

需要事先激活

    安装： pip install visdom
    启动：python -m visdom.server

RNN & LSTM

<>

LSTM

该sigmoid函数决定了更新内容的选择；该tanh函数生成了更新候选；核心作用体现在细胞状态上；水平线贯穿于图的上方区域。

LSTM 的变体

其中一个流行的LSTM变体是由Gers & Schmidhuber (2000)提出的该变体通过增添peephole连接而被Gers & Schmidhuber (2000)提出的意思是说门层也会接收细胞状态作为输入。