Search Results for "encyclopedia"

...The project follows the main RoBERTa training ideas, including removing next sentence prediction, using more diverse data, training longer, increasing batch size, and tuning optimization settings. Its training data includes news, community discussion, encyclopedia content, and other broad Chinese text sources. The repository also describes whole word masking for Chinese and provides examples for loading and fine-tuning models on sentence-pair matching tasks. Overall, it is a useful pretrained model resource for developers who want stronger Chinese BERT-style representations for classification, matching, reading comprehension, and related NLP tasks.