医疗问答 Medical Question Answering

通过医疗问答对Finetune完成闭卷问答（Closed-book QA）任务。
Complete Closed-book QA tasks by fine-tuning on Medical Q&A pairs.

下游效果 Performance

我们测试了该模型在未见过的100条QA对上的表现：

We tested the model on 100 unseen QA pairs:

gram	1-gram	2-gram	3-gram	4-gram
bleu score	0.357727	0.2713	0.22304	0.19099

使用 Usage

模型下载地址 Download Address

Huggingface地址：YuyuanQA-GPT2-3.5B

加载模型 Loading Models

from transformers import GPT2Tokenizer,GPT2LMHeadModel

hf_model_path = 'YuyuanQA-GPT2-3.5B'

tokenizer = GPT2Tokenizer.from_pretrained(hf_model_path)
model = GPT2LMHeadModel.from_pretrained(hf_model_path)

使用示例 Usage Examples

fquestion = "What should gout patients pay attention to in diet?"
inputs = tokenizer(f'Question:{question} answer:',return_tensors='pt')

generation_output = model.generate(**inputs,
                                return_dict_in_generate=True,
                                output_scores=True,
                                max_length=150,
                                # max_new_tokens=80,
                                do_sample=True,
                                top_p = 0.6,
                                eos_token_id=50256,
                                pad_token_id=0,
                                num_return_sequences = 5)

for idx,sentence in enumerate(generation_output.sequences):
    print('next sentence %d:\n'%idx,
          tokenizer.decode(sentence).split('<|endoftext|>')[0])
    print('*'*40)

回答问题 Answering the Questions

支持直接用Haggingface或者pytorch-lightning框架调用。由于在finetune的时候，加入了prompt，在问答的时候，输入应该是："Question:your question about medical? answer:",接着模型就回以续写的方式回答你的问题。用huggingface的调用代码可以参考下面的代码：

from transformers import GPT2Tokenizer,GPT2LMHeadModel
model_path = 'pretrained_model_hf/yuyuanQA-v1' # input your own model file path
model = GPT2LMHeadModel.from_pretrained(model_path)
tokenizer = GPT2Tokenizer.from_pretrained(model_path)
model = model.cuda(6) # move your model to the GPU
model.eval() # just do predict

def answering(question):
# question = "What should gout patients pay attention to in diet?"
    inputs = tokenizer(f'Question:{question} answer:',return_tensors='pt').input_ids.to(model.device)
    
    generation_output = model.generate(input_ids = inputs,
                                return_dict_in_generate=True,
                                output_scores=True,
                                max_length=150,
                                # max_new_tokens=80,
                                do_sample=True,
                                top_p = 0.9,
                                eos_token_id=50256,
                                pad_token_id=0,
                                num_return_sequences = 5)
    answers = []
    for idx,sentence in enumerate(generation_output.sequences):
        next_sentence = tokenizer.decode(sentence).split('<|endoftext|>')[0]
        answer = next_sentence.split(sep='answer:',maxsplit=1)[1]
        answers.append(answer)
    return answers
answering('your question?')

演示 Demo

我们用该模型做了一个医疗问答演示。将来，我们会将这款产品做成微信小程序与大家见面。

We made a demo of medical QA system with this model. In the future, we will make this product into a wechat app to meet you.

avatar

引用 Citation

如果您在您的工作中使用了我们的模型，可以引用我们的论文：

If you are using the resource for your work, please cite the our paper:

@article{fengshenbang,
  author    = {Junjie Wang and Yuxiang Zhang and Lin Zhang and Ping Yang and Xinyu Gao and Ziwei Wu and Xiaoqun Dong and Junqing He and Jianheng Zhuo and Qi Yang and Yongfeng Huang and Xiayu Li and Yanghan Wu and Junyu Lu and Xinyu Zhu and Weifeng Chen and Ting Han and Kunhao Pan and Rui Wang and Hao Wang and Xiaojun Wu and Zhongshen Zeng and Chongpei Chen and Ruyi Gan and Jiaxing Zhang},
  title     = {Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence},
  journal   = {CoRR},
  volume    = {abs/2209.02970},
  year      = {2022}
}

也可以引用我们的网站:

You can also cite our website:

@misc{Fengshenbang-LM,
  title={Fengshenbang-LM},
  author={IDEA-CCNL},
  year={2021},
  howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}