eRec/models/rank/dnn/data/get_slot_data.py
错误信息如下:
cat train_data/part-0 | python get_slot_data.py > slot_train_data/part-0
Traceback (most recent call last):
File "get_slot_data.py", line 70, in
d.run_from_stdin()
File "./miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/incubate/data_generator/init.py", line 128, in run_from_stdin
for user_parsed_line in line_iter():
TypeError: 'NoneType' object is not iterable
辛苦帮忙看一下
可以提供一下运行模型的yaml文件吗~尝试复现一下
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
我都是从 gitee 上直接 copy 过来的,我详细说一下:
workspace: "models/rank/dnn"
dataset:
hyper_parameters:
optimizer:
class: Adam
learning_rate: 0.001
strategy: async
sparse_inputs_slots: 27
sparse_feature_number: 1000001
sparse_feature_dim: 9
dense_input_dim: 13
fc_sizes: [512, 256, 128, 32]
distributed_embedding: 0
mode: [single_cpu_train, single_cpu_infer]
runner:
name: single_cpu_train
class: train
epochs: 4
device: cpu
save_checkpoint_interval: 2 # save model interval of epochs
save_inference_interval: 4 # save inference
save_checkpoint_path: "increment_dnn" # save checkpoint path
save_inference_path: "inference" # save inference path
save_inference_feed_varnames: [] # feed vars of save inference
save_inference_fetch_varnames: [] # fetch vars of save inference
print_interval: 10
phases: [phase1]
name: single_cpu_infer
class: infer
epochs: 1
device: cpu
init_model_path: "increment_dnn" # load model path
phases: [phase2]
name: ps_cluster
class: cluster_train
epochs: 2
device: cpu
fleet_mode: ps
save_checkpoint_interval: 1 # save model interval of epochs
save_checkpoint_path: "increment_dnn" # save checkpoint path
init_model_path: "" # load model path
print_interval: 1
phases: [phase1]
name: online_learning_cluster
class: cluster_train
runner_class_path: "{workspace}/online_learning_runner.py"
epochs: 2
device: cpu
fleet_mode: ps
save_checkpoint_interval: 1 # save model interval of epochs
save_checkpoint_path: "increment_dnn" # save checkpoint path
init_model_path: "" # load model path
print_interval: 1
phases: [phase1]
name: collective_cluster
class: cluster_train
epochs: 2
device: gpu
fleet_mode: collective
save_checkpoint_interval: 1 # save model interval of epochs
save_checkpoint_path: "increment_dnn" # save checkpoint path
init_model_path: "" # load model path
print_interval: 1
phases: [phase1]
name: single_multi_gpu_train
class: train
epochs: 1
device: gpu
selected_gpus: "0,1" # 选择多卡执行训练
save_checkpoint_interval: 1 # save model interval of epochs
save_inference_interval: 4 # save inference
save_step_interval: 1
save_checkpoint_path: "increment_dnn" # save checkpoint path
save_inference_path: "inference" # save inference path
save_step_path: "step_save"
save_inference_feed_varnames: [] # feed vars of save inference
save_inference_fetch_varnames: [] # fetch vars of save inference
print_interval: 1
phases: [phase1]
phase:
name: phase1
model: "{workspace}/model.py" # user-defined model
dataset_name: dataloader_train # select dataset by name
thread_num: 1
name: phase2
model: "{workspace}/model.py" # user-defined model
dataset_name: dataset_infer # select dataset by name
thread_num: 1
ls ./train_data_full
ls ./test_data_full
ls ./train_data
ls ./test_data
您好,在run.sh中,我看您处理train_data_full和test_data_full的时候没有报错,这两个是训练和测试的全量数据集,报错的train_data是在downlkoad.sh中从train_data_full中截取出来的小数据集。您可以直接从全量数据集中将part-0,part-1复制过来使用。
登录 后才可以发表评论