Repository navigation
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25183
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 19b0199 with merge base 436abf7 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@rezaasjd has exported this pull request. If you are a Meta employee, you can view the originating Diff in D132375392. |
This PR needs a
|
…port (pytorch#25183) Summary: Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export. annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, so it also works with lifted placeholders. Differential Revision: D132375392
0d410cb to
d9b6327
Compare
…port (pytorch#25183) Summary: Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export. annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, so it also works with lifted placeholders. Differential Revision: D132375392
…port (pytorch#25183) Summary: Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export. annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, so it also works with lifted placeholders. Differential Revision: D132375392
d9b6327 to
19b0199
Compare
…port (pytorch#25183) Summary: Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export. annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, so it also works with lifted placeholders. Differential Revision: D132375392
Summary:
Fusion decides which ops to fuse from the quantizer's annotations, but torch.export does not carry quantization_annotation over to the exported graph: it copies only a fixed set of node metadata keys, and the annotation holds references to nodes of the original graph. Fusion also resolved per-channel qparams as get_attr constants, which export lifts to placeholders. So fusion could only run on the converted graph module, before export.
annotate_input_qspec_map and annotate_output_qspec in the fused_quant quantizer now wrap the torchao helpers and, when the qspec is live, also set QUANTIZED_KEY in node.meta["custom"], which export keeps. OpQuantizer.fuse gates on that flag instead of the annotation, so it gives the same answer on the converted graph and on a program exported from it. The per-channel qparams path in fuse_aten reads shapes from meta["val"] instead of resolving get_attr tensors, so it also works with lifted placeholders.
Differential Revision: D132375392