谢谢您,这样确实可以,所以训练时的量化与推理时的量化是两回事是吗?自适应量化模型保存成原生模型会对精度造成影响吗?展开
您好,对自适应量化层的量化应该怎么做?下面的代码显然键值是不对应的,我试着将模型结构转化成自适应量化模型再加载,但执行的时候会报RuntimeError: Didn't find kernel to dispatch to for operator 'torch_mlu::quantify_convolution'. Tried to look up kernel for dispatch key 'CPUTensorId'. Registered dispatch keys are: [MLUTensorId] (lookup_ at /data/v1.7.0/PyTorch/src/cambricon_pytorch/pytorch/src/pytorch/aten/src/ATen/core/dispatch/DispatchTable.h:249)的错误,如果把这个过程搬到MLU上执行,又会报[WARNING][/data/v1.7.0/PyTorch/src/cambricon_pytorch/pytorch/src/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:3185][quantify_convolution][thread:140488953952000][process:96698]: quantify_convolution Op cannot run on MLU device, start running on CPU!ValueError: To do for CPU这样的错误。[代码]展开
建议把训练出的模型先通过dequantize()接口保存为原始模型,使用原始的模型再使用图中的代码进行量化和融合。
请问是270上训练吗?推理时,不需要再转成原生模型,但需要量化,融合
您好,对自适应量化层的量化应该怎么做?下面的代码显然键值是不对应的,我试着将模型结构转化成自适应量化模型再加载,但执行的时候会报
RuntimeError: Didn't find kernel to dispatch to for operator 'torch_mlu::quantify_convolution'. Tried to look up kernel for dispatch key 'CPUTensorId'. Registered dispatch keys are: [MLUTensorId] (lookup_ at /data/v1.7.0/PyTorch/src/cambricon_pytorch/pytorch/src/pytorch/aten/src/ATen/core/dispatch/DispatchTable.h:249)
的错误,如果把这个过程搬到MLU上执行,又会报
[WARNING][/data/v1.7.0/PyTorch/src/cambricon_pytorch/pytorch/src/catch/torch_mlu/csrc/aten/operators/op_methods.cpp][line:3185][quantify_convolution][thread:140488953952000][process:96698]:
quantify_convolution Op cannot run on MLU device, start running on CPU!
ValueError: To do for CPU
这样的错误。
def generate_qua(): model = D2Net(model_file=None, use_cuda=False, use_relu=True) # model = mlu_quantize.adaptive_quantize(model,20000) state_dict = torch.load('models/d2-net-3-18-.pth', map_location='cpu') model.load_state_dict(state_dict) model.eval() qconfig = {'iteration':2,'firstconv':False} quantized_model = mlu_quantize.quantize_dynamic_mlu(model, qconfig, dtype='int8', gen_quant=True) input_tensor = torch.zeros((1,3,480,640)) # input_tensor, quantized_model = input_tensor.to(ct.mlu_device()), quantized_model.to(ct.mlu_device()) quantized_model(input_tensor) torch.save(quantized_model.state_dict(),'d2-net-3-18-8.pt') print('quantize finished!')
请登录后评论