117 Star 814 Fork 468

MindSpore / mindformers

 / 详情

Bloom 560m推理,关闭增量推理后答案异常

TODO
Bug-Report
创建于  
2024-05-13 18:49

以下是'use_past': False的答案:
2024-05-13 18:42:33,255 - mindformers[mindformers/models/base_model.py:115] - INFO - weights in ./checkpoint_download/bloom/bloom_560m.ckpt are loaded
2024-05-13 18:42:33,267 - mindformers[mindformers/auto_class.py:453] - INFO - model built successfully!
2024-05-13 18:42:33,276 - mindformers[mindformers/generation/text_generator.py:1103] - INFO - Generation Config is: {'max_length': 100, 'max_new_tokens': None, 'num_beams': 1, 'do_sample': True, 'use_past': False, 'temperature': 1.0, 'top_k': 5, 'top_p': 1, 'repetition_penalty': 1, 'encoder_repetition_penalty': 1.0, 'renormalize_logits': False, 'pad_token_id': 3, 'bos_token_id': 1, 'eos_token_id': 2, '_from_model_config': True}
2024-05-13 18:42:33,277 - mindformers[mindformers/generation/text_generator.py:174] - INFO - The generation mode will be SAMPLE.
[WARNING] DEVICE(40280,ffff98df20b0,python):2024-05-13-18:42:45.899.704 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_executor.cc:1258] RunInitGraph] Can not find init_subgraph.kernel_graph_4 sub graph, don't need data init subgraph in INFER mode.
2024-05-13 18:43:55,975 - mindformers[mindformers/generation/text_generator.py:724] - INFO - total time: 82.69642066955566 s; generated tokens: 94 tokens; generate speed: 1.1366876491016702 tokens/s
what color is the sky? can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can can

以下是'use_past': True的答案:
2024-05-13 18:22:54,168 - mindformers[mindformers/generation/text_generator.py:1103] - INFO - Generation Config is: {'max_length': 100, 'max_new_tokens': None, 'num_beams': 1, 'do_sample': True, 'use_past': True, 'temperature': 1.0, 'top_k': 5, 'top_p': 1, 'repetition_penalty': 1, 'encoder_repetition_penalty': 1.0, 'renormalize_logits': False, 'pad_token_id': 3, 'bos_token_id': 1, 'eos_token_id': 2, '_from_model_config': True}
2024-05-13 18:22:54,169 - mindformers[mindformers/generation/text_generator.py:174] - INFO - The generation mode will be SAMPLE.
[WARNING] DEVICE(40280,ffff98df20b0,python):2024-05-13-18:23:14.456.678 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_executor.cc:1258] RunInitGraph] Can not find init_subgraph.kernel_graph_0 sub graph, don't need data init subgraph in INFER mode.

[WARNING] GE_ADPT(40280,ffff98df20b0,python):2024-05-13-18:23:54.696.766 [mindspore/ccsrc/transform/graph_ir/convert.cc:885] ConvertAllNode] Parameter @kernel_graph_1:param_input_position cannot found in toposort lists.
[WARNING] GE_ADPT(40280,ffff98df20b0,python):2024-05-13-18:23:54.696.819 [mindspore/ccsrc/transform/graph_ir/convert.cc:885] ConvertAllNode] Parameter @kernel_graph_1:param_init_reset cannot found in toposort lists.
[WARNING] DEVICE(40280,ffff98df20b0,python):2024-05-13-18:23:55.021.515 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_graph_executor.cc:1258] RunInitGraph] Can not find init_subgraph.kernel_graph_1 sub graph, don't need data init subgraph in INFER mode.
2024-05-13 18:24:11,567 - mindformers[mindformers/generation/text_generator.py:724] - INFO - total time: 77.39647126197815 s; generated tokens: 2 tokens; generate speed: 0.02584097139558508 tokens/s
what color is the sky? blue

评论 (0)

hql_hql 创建了Question
hql_hql 修改了描述
hql_hql 任务类型Question 修改为Bug-Report
hql_hql 修改了描述
展开全部操作日志

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(1)
8591397 hql hql 1710835433
Python
1
https://gitee.com/mindspore/mindformers.git
git@gitee.com:mindspore/mindformers.git
mindspore
mindformers
mindformers

搜索帮助