nlp学习笔记

目录

很多入门例子

bert chinese

很多入门例子

https://github.com/lansinuote/Huggingface_Toturials

bert chinese

import torch 
import torch.nn as nn
from transformers import AutoTokenizer, AutoModel, BertModel, TFBertModel, BertTokenizer

# youpath = 'D:/bert-base-chinese'
# tokenizer =AutoTokenizer.from_pretrained(youpath)

tokenizer = BertTokenizer.from_pretrained(
    pretrained_model_name_or_path='bert-base-chinese',
    cache_dir=None,
    force_download=False,
)

model = BertModel.from_pretrained('bert-base-chinese')