Dataset bias can be transferred into the model during training. This problem is well known to dataset owners, highlighting the importance of documenting training data characteristics and limitations, especially for future users. However, it is difficult for dataset owners to predict and document all potential biases in their datasets, as the relevance of potential biases may depend on the particular model application scenario. Therefore, we propose that data producers enable future users of the datasets to check the data for bias dynamically and according to their needs, rather than just providing a static description of the dataset characteristics, e.g. information on gender distribution. We propose to extend existing documentation approaches, such as datasheets and data statements, to include information about the dataset in the form of linked data, allowing consumers of the dataset to explore how other possible biases might affect their system. In this study, we use a linked data resource to analyze and evaluate bias in CoNLL-2003, a widely used dataset for named entity recognition, to demonstrate the added value of this approach. To determine whether the bias in the data is reflected in the model, we analyzed the stereotyped knowledge in the English version of the dataset and in a model trained with it.     «

Dataset bias can be transferred into the model during training. This problem is well known to dataset owners, highlighting the importance of documenting training data characteristics and limitations, especially for future users. However, it is difficult for dataset owners to predict and document all potential biases in their datasets, as the relevance of potential biases may depend on the particular model application scenario. Therefore, we propose that data producers enable future users of the...     »