×

签到

分享到微信

打开微信,使用扫一扫进入页面后,点击右上角菜单,

点击“发送给朋友”或“分享到朋友圈”完成分享

实验一中cpu没问题,mlu报错CNRT_RET_ERR_STREAM是什么原因? 17000177362020-07-18 15:16:45 回复 7 查看
实验一中cpu没问题,mlu报错CNRT_RET_ERR_STREAM是什么原因?
分享到:

(virtualenv_mlu) root@localhost:/opt/AICSE-demo-student/demo/style_transfer_bcl/src/online_mlu# python power_difference_test_bcl.py
WARNING:tensorflow:From power_difference_test_bcl.py:10: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

CNML: 7.2.1 c8ada41
CNRT: 4.2.1 fa5e44c
CNML: 7.2.1 c8ada41
CNML: 7.2.1 c8ada41
2020-07-18 07:00:11.427192: I tensorflow/stream_executor/mlu/mlu_stream_executor.cc:470][83587] MLU OP Cache enabled
CNML: 7.2.1 c8ada41
2020-07-18 07:00:11.427477: I tensorflow/core/common_runtime/mlu/mlu_device.cc:396][83587] __LOG_MLU__, Current MLU_Visible_Devices Count: 1
2020-07-18 07:00:11.427534: I tensorflow/core/common_runtime/mlu/mlu_device.cc:115][83587] __LOG_MLU__, save_offline_model: 0
2020-07-18 07:00:11.427566: I tensorflow/core/common_runtime/mlu/mlu_device.cc:121][83587] __LOG_MLU__, core_version: MLU270
2020-07-18 07:00:11.427595: I tensorflow/core/common_runtime/mlu/mlu_device.cc:123][83587] __LOG_MLU__, core_num: 1
WARNING:tensorflow:From power_difference_test_bcl.py:11: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

2020-07-18 07:00:11.451233: I tensorflow/core/grappler/optimizers/mlu_control_optimizer.cc:44][83587] MLUControlOptimizer, it may take a while ...
2020-07-18 07:00:11.451450: I tensorflow/core/grappler/optimizers/mlu_control_optimizer.cc:170][83587] MLUControlOptimizer end!
2020-07-18 07:00:11.454122: I tensorflow/compiler/tf2mlu/convert/convert_graph.cc:654][83587] MLU fusion node PowerDifference/my_mlu_op_0 added for segment 0 consisting of 1 nodes succeeded.
2020-07-18 07:00:11.749300: [cnrtError] [83979] [Card : 0] MLU unfinished. cnrtStream fail.
2020-07-18 07:00:11.749418: [cnrtError] [83979] [Card : 0] unknown error2020-07-18 07:00:11.749434: [cnrtError] [83979] [Card : 0] for more information, please use core dump analysis tools.
2020-07-18 07:00:11.749507: E ./tensorflow/stream_executor/mlu/mlu_api/lib_ops/mlu_lib_common.h:268][83979] TF_CNRT_CHECK failed: CNRT_RET_ERR_STREAM
2020-07-18 07:00:11.752307: E ./tensorflow/stream_executor/mlu/mlu_api/lib_ops/mlu_lib_common.h:241][83979] TF_CNRT_CHECK failed: CNRT_RET_ERR_EVENT
2020-07-18 07:00:11.752485: E tensorflow/stream_executor/stream.cc:331][83979] Error recording event in stream: CNRT_RET_ERR_EVENT; not marking stream as bad, as the Event may be at fault. Monitor for further errors.
2020-07-18 07:00:11.794269: [cnrtError] [83989] [Card : 0] MLU unfinished. cnrtStream fail.
2020-07-18 07:00:11.794333: [cnrtError] [83989] [Card : 0] invalid stream.
2020-07-18 07:00:11.794357: [cnrtError] [83989] [Card : 0] for more information, please use core dump analysis tools.
2020-07-18 07:00:11.794408: E ./tensorflow/stream_executor/mlu/mlu_api/lib_ops/mlu_lib_common.h:268][83989] TF_CNRT_CHECK failed: CNRT_RET_ERR_STREAM
2020-07-18 07:00:11.794486: W tensorflow/stream_executor/stream.cc:274][83989] Error blocking host until done in stream destructor: Unknown: CNRT_RET_ERR_STREAM
Traceback (most recent call last):
  File "power_difference_test_bcl.py", line 39, in <module>
    main()
  File "power_difference_test_bcl.py", line 24, in main
    res = power_difference_op(input_x, input_y, input_pow)
  File "power_difference_test_bcl.py", line 15, in power_difference_op
    return sess.run(out, feed_dict = {x: input_x, y: input_y, z: input_pow})
  File "/opt/AICSE-demo-student/env/tensorflow-v1.10/virtualenv_mlu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 950, in run
    run_ data_ptr)
  File "/opt/AICSE-demo-student/env/tensorflow-v1.10/virtualenv_mlu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run
    feed_dict_tensor, options, run_ data)
  File "/opt/AICSE-demo-student/env/tensorflow-v1.10/virtualenv_mlu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
    run_ data)
  File "/opt/AICSE-demo-student/env/tensorflow-v1.10/virtualenv_mlu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python. work.errors_impl.UnknownError: CNRT_RET_ERR_STREAM
         [[{{node PowerDifference/my_mlu_op_0}}]]

版权所有 © 2024 寒武纪 Cambricon.com 备案/许可证号:京ICP备17003415号-1
关闭