×

签到

分享到微信

打开微信,使用扫一扫进入页面后,点击右上角菜单,

点击“发送给朋友”或“分享到朋友圈”完成分享

算子不适配导致RuntimeError: outputs_[i]-&gtuses().empty() INTERNAL ASSERT FAILED at /pytorch/torch/csrc/jit/ir.cpp:1027, please report a bug to PyTorch. (eraseOutput zzlj2024-03-14 09:58:47 回复 1 查看 技术答疑 使用求助
算子不适配导致RuntimeError: outputs_[i]-&gtuses().empty() INTERNAL ASSERT FAILED at /pytorch/torch/csrc/jit/ir.cpp:1027, please report a bug to PyTorch. (eraseOutput
分享到:

算子出现不适配的情况,查看mlu_functions.yaml发现是有addmm和sigmoid_这两个算子的,并且PyTorch 1.3.0也有这两个算子,请问应该怎么做呢?下面是部分log,为了节省篇幅,我省去了部分WARNING的log,省去的都是提示算子不适配的log。

CNML: 7.10.2 0a592c0
CNRT: 4.10.1 a884a9a
mlu
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:173][addmm][thread:140143720093440][process:22299]:
addmm Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:173][addmm][thread:140143720093440][process:22299]:
addmm Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:1936][sigmoid_][thread:140143720093440][process:22299]:
sigmoid_ Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:1936][sigmoid_][thread:140143720093440][process:22299]:
sigmoid_ Op cannot run on MLU device, start running on CPU!
<class 'torch.Tensor'> <class 'torch.Tensor'>
tensor([[18.4220, 17.7609, 18.8722, 17.2082, 16.9452]])
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:173][addmm][thread:140143720093440][process:22299]:
addmm Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:173][addmm][thread:140143720093440][process:22299]:
addmm Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:1936][sigmoid_][thread:140143720093440][process:22299]:
sigmoid_ Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:1936][sigmoid_][thread:140143720093440][process:22299]:
sigmoid_ Op cannot run on MLU device, start running on CPU!
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:41][MLUSupport][thread:140143720093440][process:22299]:
[Fusion Segment] Please check mlu_functions.yaml && Maybe MLU fusion does NOT supports op: addmm
%33 : Float(*, *) = aten::addmm(%16, %input.5, %30, %31, %31), scope: RnnAgent/GRUCell[rnn] # /torch/venv3/pytorch/lib/python3.6/site-packages/torch/nn/modules/rnn.py:1023:0
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:41][MLUSupport][thread:140143720093440][process:22299]:
[Fusion Segment] Please check mlu_functions.yaml && Maybe MLU fusion does NOT supports op: values
%23 : Tensor[] = aten::values(%1)
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:41][MLUSupport][thread:140143720093440][process:22299]:
[Fusion Segment] Please check mlu_functions.yaml && Maybe MLU fusion does NOT supports op: addmm
%43 : Float(*, *) = aten::addmm(%17, %hx, %40, %31, %31), scope: RnnAgent/GRUCell[rnn] # /torch/venv3/pytorch/lib/python3.6/site-packages/torch/nn/modules/rnn.py:1023:0
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:41][MLUSupport][thread:140143720093440][process:22299]:
[Fusion Segment] Please check mlu_functions.yaml && Maybe MLU fusion does NOT supports op: addmm
%43 : Float(*, *) = aten::addmm(%17, %hx, %40, %31, %31), scope: RnnAgent/GRUCell[rnn] # /torch/venv3/pytorch/lib/python3.6/site-packages/torch/nn/modules/rnn.py:1023:0
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:41][MLUSupport][thread:140143720093440][process:22299]:
[Fusion Segment] Please check mlu_functions.yaml && Maybe MLU fusion does NOT supports op: addmm
%33 : Float(*, *) = aten::addmm(%16, %input.5, %30, %31, %31), scope: RnnAgent/GRUCell[rnn] # /torch/venv3/pytorch/lib/python3.6/site-packages/torch/nn/modules/rnn.py:1023:0
[WARNING][/pytorch/catch/torch_mlu/csrc/jit/passes/segment_graph.cpp][line:217][SegmentGraph][thread:140143720093440][process:22299]:
Graph is segmented into 4 subgraphs.
Please check the above WARNING logs which include "[Fusion Segment]" to see whether there are MLU unsupported ops exists in pytorch model.
Traceback (most recent call last):
  File "test_mlu_copy.py", line 86, in <module>
    x, h = traced_model(x_mlu)
  File "/torch/venv3/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
RuntimeError: outputs_[i]->uses().empty() INTERNAL ASSERT FAILED at /pytorch/torch/csrc/jit/ir.cpp:1027, please report a bug to PyTorch.  (eraseOutput at /pytorch/torch/csrc/jit/ir.cpp:1027)
#0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0x57 (0x7f75baaa3237 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libc10.so)
#1: torch::jit::Node::eraseOutput(unsigned long) + 0x1eb (0x7f7568d8fa2b in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#2: torch::jit::Node::destroy() + 0x24 (0x7f7568d90524 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#3: torch::jit::SubgraphUtils::mergeNodeIntoSubgraph(torch::jit::Node*, torch::jit::Node*) + 0x14c5 (0x7f7568e9edc5 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#4: torch_mlu::jit::buildSubgraph(std::shared_ptr<torch::jit::Graph>&, std::vector<torch::jit::Node*, std::allocator<torch::jit::Node*> >, std::vector<torch_mlu::jit::group, std::allocator<torch_mlu::jit::group> >) + 0x1d3 (0x7f7521e52283 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch_mlu/csrc/lib/libjit_mlu.so)
#5: torch_mlu::jit::SegmentGraph(std::shared_ptr<torch::jit::Graph>&) + 0x253 (0x7f7521e58493 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch_mlu/csrc/lib/libjit_mlu.so)
#6: <unknown function> + 0x2cc0d30 (0x7f7568d49d30 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#7: <unknown function> + 0x2cca302 (0x7f7568d53302 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#8: <unknown function> + 0x2ccb3a5 (0x7f7568d543a5 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#9: <unknown function> + 0x2ccb57b (0x7f7568d5457b in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#10: <unknown function> + 0x2cc2a79 (0x7f7568d4ba79 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#11: torch::jit::Function::run(std::vector<c10::IValue, std::allocator<c10::IValue> >&) + 0x63 (0x7f756900d4a3 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch.so)
#12: <unknown function> + 0x560b80 (0x7f75bb7bbb80 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
#13: <unknown function> + 0x561455 (0x7f75bb7bc455 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
#14: <unknown function> + 0x532414 (0x7f75bb78d414 in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
#15: <unknown function> + 0x1dce3a (0x7f75bb437e3a in /torch/venv3/pytorch/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
<omitting python s>
#39: __libc_start_main + 0xf0 (0x7f75bfd6b840 in /lib/x86_64-linux-gnu/libc.so.6)



版权所有 © 2024 寒武纪 Cambricon.com 备案/许可证号:京ICP备17003415号-1
关闭