Home
last modified time | relevance | path

Searched full:quantized (Results 1 – 25 of 2369) sorted by relevance

12345678910>>...95

/aosp_15_r20/external/pytorch/aten/src/ATen/native/quantized/
H A Dlibrary.cpp12 TORCH_LIBRARY(quantized, m) { in TORCH_LIBRARY() argument
19 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add(Tensor qa, Tensor qb, float scale, int zero_point) ->… in TORCH_LIBRARY()
20 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add.out(Tensor qa, Tensor qb, Tensor(a!) out) -> Tensor(a… in TORCH_LIBRARY()
21 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add.Scalar(Tensor qa, Scalar b) -> Tensor qc"), {at::Tag:… in TORCH_LIBRARY()
22 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add.Scalar2(Scalar b, Tensor qa) -> Tensor qc"), {at::Tag… in TORCH_LIBRARY()
23 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add.Scalar_out(Tensor qa, Scalar b, Tensor(a!) out) -> Te… in TORCH_LIBRARY()
24 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add_relu(Tensor qa, Tensor qb, float scale, int zero_poin… in TORCH_LIBRARY()
25 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add_relu.Scalar(Tensor qa, Scalar b) -> Tensor qc"), {at:… in TORCH_LIBRARY()
26 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add_relu.Scalar2(Scalar b, Tensor qa) -> Tensor qc"), {at… in TORCH_LIBRARY()
27 …m.def(TORCH_SELECTIVE_SCHEMA("quantized::add_relu.out(Tensor qa, Tensor qb, Tensor(a!) out) -> Ten… in TORCH_LIBRARY()
[all …]
H A DREADME.md1 The quantized folder holds the implementation of the low-level quantized kernel.
2 The kernels are registered in `torch::_ops` namespace, and operate on the quantized `at::Tensor` da…
3 …arn more about the quantized tensors in the [quantized tensor API wiki](https://github.com/pytorch…
5 This document serves as an entry point for quantized kernel implementation.
7 ## Implementing native quantized ops
9 The new quantized ops are almost always located under the `ATen/native/quantized/cpu` folder. For
10 the sake of an example, let us implement an element-wise quantized [logical XAND](https://en.wiktio…
11 operation under `ATen/native/quantized/cpu/qxand.cpp`.
13 ### Step 0. Implement the quantized function
15 Before writing the quantized kernel and registering it, let us implement a quantized function.
[all …]
H A Dqconv_unpack.cpp17 #include <ATen/native/quantized/cpu/fbgemm_utils.h>
18 #include <ATen/native/quantized/cpu/QnnpackUtils.h>
19 #include <ATen/native/quantized/cpu/OnednnUtils.h>
20 #include <ATen/native/quantized/cpu/QuantUtils.h>
21 #include <ATen/native/quantized/PackedParams.h>
67 "quantized::conv2d_unpack (qnnpack): QNNPACK only supports Conv2d " in run()
81 "Didn't find engine for operation quantized::conv2d_unpack ", in run()
122 "Didn't find engine for operation quantized::conv1d_unpack ", in run()
198 TORCH_LIBRARY_IMPL(quantized, CatchAll, m) { in TORCH_LIBRARY_IMPL() argument
202 m.impl(TORCH_SELECTIVE_NAME("quantized::conv_unpack"), TORCH_FN(QConvUnpackWeightsInt8<2>::run)); in TORCH_LIBRARY_IMPL()
[all …]
/aosp_15_r20/external/pytorch/torch/ao/nn/quantized/
H A Dfunctional.py2 r""" Functional interface (quantized)."""
59 See :class:`~torch.ao.nn.quantized.AvgPool2d` for details and output shape.
62 input: quantized input tensor :math:`(\text{minibatch} , \text{in\_channels} , iH , iW)`
77 raise ValueError("Input to 'quantized.avg_pool2d' must be quantized!")
106 input: quantized input tensor :math:`(\text{minibatch} , \text{in\_channels} , iH , iW)`
121 raise ValueError("Input to 'quantized.avg_pool3d' must be quantized!")
135 Applies a 2D adaptive average pooling over a quantized input signal composed
136 of several quantized input planes.
140 See :class:`~torch.ao.nn.quantized.AdaptiveAvgPool2d` for details and output shape.
148 "Input to 'quantized.functional.adaptive_avg_pool2d' must be quantized!"
[all …]
/aosp_15_r20/external/pytorch/test/quantization/eager/
H A Dtest_numeric_suite_eager.py6 import torch.ao.nn.quantized as nnq
98 r"""Compare the weights of float and static quantized conv layer"""
100 qengine = torch.backends.quantized.engine
108 self.assertTrue(v["float"].shape == v["quantized"].shape)
120 r"""Compare the weights of float and static quantized linear layer"""
122 qengine = torch.backends.quantized.engine
130 self.assertTrue(v["float"].shape == v["quantized"].shape)
142 r"""Compare the weights of float and dynamic quantized linear layer"""
144 qengine = torch.backends.quantized.engine
152 self.assertTrue(len(v["float"]) == len(v["quantized"]))
[all …]
/aosp_15_r20/external/pytorch/torch/csrc/jit/passes/quantization/
H A Dquantization_patterns.h136 // quant fusion for ops like `quantized::add_scalar`, `quantized::mul_scalar`
290 %w_quant : Tensor, %b : Tensor? = quantized::conv1d_unpack(%packed_params) in quant_fusion_pattern_and_replacements()
300 %w_quant : Tensor, %b : Tensor? = quantized::conv1d_unpack(%packed_params) in quant_fusion_pattern_and_replacements()
311 %w_quant : Tensor, %b : Tensor? = quantized::conv1d_unpack(%packed_params) in quant_fusion_pattern_and_replacements()
318 // quantized::conv1d in quant_fusion_pattern_and_replacements()
321 %r_quant = quantized::conv1d(%a_quant, %packed_params, %r_scale, %r_zero_point) in quant_fusion_pattern_and_replacements()
324 // quantized::conv1d_relu in quant_fusion_pattern_and_replacements()
327 %r_quant = quantized::conv1d_relu(%a_quant, %packed_params, %r_scale, %r_zero_point) in quant_fusion_pattern_and_replacements()
334 %w_quant : Tensor, %b : Tensor? = quantized::conv2d_unpack(%packed_params) in quant_fusion_pattern_and_replacements()
344 %w_quant : Tensor, %b : Tensor? = quantized::conv2d_unpack(%packed_params) in quant_fusion_pattern_and_replacements()
[all …]
/aosp_15_r20/external/pytorch/docs/source/
H A Dquantization.rst16 tensors at lower bitwidths than floating point precision. A quantized model
24 speed up inference and only the forward pass is supported for quantized
35 At lower level, PyTorch provides a way to represent quantized tensors and
51 (1). Programmable API for configuring how a model is quantized that can scale to many more use cases
53 …reference quantized model representation that can represent quantized computation with integer ope…
105 1. dynamic quantization (weights quantized with activations read/stored in
106 floating point and quantized for compute)
107 2. static quantization (weights quantized, activations quantized, calibration
109 3. static quantization aware training (weights quantized, activations quantized,
156 quantized ahead of time but the activations are dynamically quantized
[all …]
H A Dquantization-support.rst182 Quantized Tensors support a limited subset of data manipulation methods of the
316 then be quantized.
366 torch.ao.nn.intrinsic.quantized
368 .. automodule:: torch.ao.nn.intrinsic.quantized
369 .. automodule:: torch.ao.nn.intrinsic.quantized.modules
372 This module implements the quantized implementations of fused operations
376 .. currentmodule:: torch.ao.nn.intrinsic.quantized
390 torch.ao.nn.intrinsic.quantized.dynamic
392 .. automodule:: torch.ao.nn.intrinsic.quantized.dynamic
393 .. automodule:: torch.ao.nn.intrinsic.quantized.dynamic.modules
[all …]
/aosp_15_r20/external/pytorch/torch/ao/quantization/fx/
H A D_lower_to_native_backend.py7 import torch.ao.nn.intrinsic.quantized as nniq
8 import torch.ao.nn.intrinsic.quantized.dynamic as nniqd
9 import torch.ao.nn.quantized as nnq
10 import torch.ao.nn.quantized.dynamic as nnqd
11 import torch.ao.nn.quantized.reference as nnqr
14 from torch.ao.nn.quantized.modules.utils import WeightedQuantizedModule
32 torch._ops.ops.quantized.hardswish: ["inplace"],
33 torch._ops.ops.quantized.elu: ["inplace"],
34 torch._ops.ops.quantized.dropout: ["inplace"],
35 torch._ops.ops.quantized.instance_norm: [
[all …]
H A DREADME.md23 Quantized Model
44 convert_fx --> qm[Quantized Model]:::nofs
50 …fig, QConfig propagation (might be removed), fused modules, QAT module, quantized modules, QAT mod…
63 Prepared Model --> (2.1 `convert_to_reference`) --> Reference Quantized Model
64 --> (2.2 Lower to Native Backend) --> Quantized Model
207 The end goal for this step is to insert QDQStubs at edges so that we produce a graph of quantized r…
242 There is a mismatch here and (2) is a quantized dtype, so we need to insert QDQStub at the edge.
248 …ook at the target output dtype for qat_linear_relu Node, and if it is a quantized dtype (quint8, q…
283 …t_linear_relu"]["output_activation"], and find that it is float, which is not a quantized dtype, so
323 ### 3.1 Conversion to Reference Quantized Model
[all …]
H A D_decomposed.py12 # Note: decomposed means decomposed quantized tensor, using decomposed so that the
60 from floating point to quantized values
66 quant_min (int): minimum quantized value for output Tensor
67 quant_max (int): maximum quantized value for output Tensor
122 from floating point to quantized values
179 from floating point to quantized values
235 from quantized values to floating point values
239 e.g. (`torch.uint8`), it is a per tensor quantized Tensor if combined with
246 quant_min (int): minimum quantized value for input Tensor (not used in computation,
249 quant_max (int): maximum quantized value for input Tensor (not used in computation,
[all …]
/aosp_15_r20/external/pytorch/torch/ao/ns/
H A D_numeric_suite.py5 import torch.ao.nn.quantized as nnq
6 import torch.ao.nn.quantized.dynamic as nnqd
58 r"""Compare the weights of the float module with its corresponding quantized
60 a dictionary with two keys 'float' and 'quantized', containing the float and
61 quantized weights. This dict can be used to compare and compute the quantization
62 error of the weights of float and quantized models.
73 wt_compare_dict[key]['quantized'].dequantize()
79 quantized_dict: state dict of the quantized model
83 a dictionary with two keys 'float' and 'quantized', containing the float and
84 quantized weights
[all …]
/aosp_15_r20/external/pytorch/test/quantization/core/
H A Dtest_quantized_op.py90 # Reference quantized Linear operator
148 """Helper function to test quantized activation functions."""
155 A test config is a list that contains metadata about the quantized activation
164 quantized_fn: a list of the quantized functions to be tested
177 output_scale/output_zero_point keyword argument when calling quantized op
183 if (X.device.type == 'cuda') and (torch.backends.quantized.engine == 'qnnpack'):
237 # Finds qY using in-place or non-in-place quantized operators.
242 """Tests the correctness of the quantized::relu op."""
278 """Tests the correctness of the quantized::relu6 op."""
283 torch.ops.quantized.relu6,
[all …]
H A Dtest_quantized_module.py6 import torch.ao.nn.intrinsic.quantized as nniq
7 import torch.ao.nn.quantized.reference as nnqr
9 import torch.ao.nn.quantized as nnq
10 import torch.ao.nn.quantized.dynamic as nnqd
47 quantized operator implementations correctly in the user facing APIs, these are
48 not correctness test for the underlying quantized operators. For correctness
72 """test API functionality for nn.quantized.linear"""
81 nnq.Linear, 'QuantizedLinear', torch.ops.quantized.linear, batch_size,
86 """test API functionality for nn.intrinsic.quantized.linear_relu"""
95 nniq.LinearReLU, 'QuantizedLinearReLU', torch.ops.quantized.linear_relu,
[all …]
/aosp_15_r20/external/pytorch/test/mobile/model_test/
H A Dcoverage.yaml662 - quantized::add
663 - quantized::add_relu
664 - quantized::add_scalar
665 - quantized::batch_norm2d
666 - quantized::batch_norm3d
667 - quantized::cat
668 - quantized::conv1d
669 - quantized::conv1d_prepack
670 - quantized::conv1d_relu
671 - quantized::conv1d_unpack
[all …]
/aosp_15_r20/external/ComputeLibrary/arm_compute/core/
H A DQuantizationInfo.h38 using qasymm8_signed_t = int8_t; /**< 8 bit signed quantized asymmetric scalar value */
39 using qasymm8_t = uint8_t; /**< 8 bit quantized asymmetric scalar value */
40 using qsymm16_t = int16_t; /**< 16 bit quantized symmetric scalar value */
41 using qasymm16_t = uint16_t; /**< 16 bit quantized asymmetric scalar value */
216 "quantized type should be either uint8_t or int8_t.");
223 * @return Quantized value
228 const int quantized = support::cpp11::lround(value / qinfo.scale) + qinfo.offset; in quantize() local
229 …_cast<QUANTIZED_TYPE>(arm_compute::utility::clamp<decltype(quantized), QUANTIZED_TYPE>(quantized)); in quantize()
238 * @return Quantized value
248 … const int quantized = arm_compute::round(value / qinfo.scale, rounding_policy) + qinfo.offset; in quantize() local
[all …]
/aosp_15_r20/external/pytorch/torch/csrc/jit/passes/onnx/
H A Dunpack_quantized_weights.cpp3 #include <ATen/native/quantized/PackedParams.h>
24 // Get the scale of the input to quantized op. There are two cases here
27 // 2. For ops with no output scale in op signature (like quantized::relu)
34 "quantized::max_pool2d", in getScaleFromInput()
39 "quantized::nchw2nhwc", in getScaleFromInput()
40 "quantized::nhwc2nchw", in getScaleFromInput()
43 "quantized::cat", in getScaleFromInput()
54 } else if (input_name == "quantized::linear") { in getScaleFromInput()
55 // %r = quantized::linear(%input, %packed_weight, %w_scale, %w_zero_point) in getScaleFromInput()
58 "quantized::linear expected scale to be 3rd input"); in getScaleFromInput()
[all …]
/aosp_15_r20/external/executorch/kernels/quantized/test/
H A Dtargets.bzl6 op_test("op_quantize_test", kernel_name = "quantized")
7 op_test("op_dequantize_test", kernel_name = "quantized")
8 op_test("op_choose_qparams_test", kernel_name = "quantized")
9 op_test("op_add_test", kernel_name = "quantized", deps = [
10 "//executorch/kernels/quantized/cpu:op_dequantize",
11 "//executorch/kernels/quantized/cpu:op_quantize",
12 "//executorch/kernels/quantized/cpu:op_add",
13 "//executorch/kernels/quantized:generated_lib_headers",
18 op_test("op_embedding_test", kernel_name = "quantized", deps = [
19 "//executorch/kernels/quantized/cpu:op_dequantize",
[all …]
/aosp_15_r20/external/pytorch/torch/ao/nn/quantized/modules/
H A Dconv.py2 r"""Quantized convolution modules."""
92 f"'padding_mode' {padding_mode} is not supported by quantized convolution"
327 r"""Create a (fbgemm/qnnpack) quantized module from a reference quantized module
329 … ref_qconv (Module): a reference quantized module, either produced by torch.ao.quantization
355 r"""Applies a 1D convolution over a quantized input signal composed of
356 several quantized input planes.
379 >>> m = nn.quantized.Conv1d(16, 33, 3, stride=2)
437 self._packed_params = torch.ops.quantized.conv1d_prepack(
441 self._packed_params = torch.ops.quantized.conv1d_prepack(
446 w, b = torch.ops.quantized.conv1d_unpack(self._packed_params)
[all …]
/aosp_15_r20/external/pytorch/aten/src/ATen/native/quantized/cpu/
H A Dqmul.cpp8 #include <ATen/native/quantized/cpu/OnednnUtils.h>
9 #include <ATen/native/quantized/cpu/QnnpackUtils.h>
10 #include <ATen/native/quantized/cpu/QuantUtils.h>
11 #include <ATen/native/quantized/cpu/QuantizedOps.h>
12 #include <ATen/native/quantized/cpu/XnnpackUtils.h>
13 #include <ATen/native/quantized/cpu/init_qnnpack.h>
14 #include <ATen/quantized/Quantizer.h>
322 // all variations of `quantized::mul` is merged into `quantized::mul`
337 // all variations of `quantized::mul` is merged into `quantized::mul`
347 TORCH_LIBRARY_IMPL(quantized, QuantizedCPU, m) { in TORCH_LIBRARY_IMPL() argument
[all …]
H A DBinaryOps.cpp7 #include <ATen/quantized/Quantizer.h>
8 #include <ATen/native/quantized/cpu/BinaryOps.h>
9 #include <ATen/native/quantized/cpu/QuantizedOps.h>
10 #include <ATen/native/quantized/cpu/init_qnnpack.h>
11 #include <ATen/native/quantized/cpu/QnnpackUtils.h>
12 #include <ATen/native/quantized/cpu/XnnpackUtils.h>
67 // To implement tensor-scalar addition in quantized space, we simply in _add_scalar_out()
460 // all variations of `quantized::add` is merged into `quantized::add`
468 // all variations of `quantized::add` is merged into `quantized::add`
474 TORCH_LIBRARY_IMPL(quantized, QuantizedCPU, m) { in TORCH_LIBRARY_IMPL() argument
[all …]
H A Dqembeddingbag_unpack.cpp4 #include <ATen/native/quantized/cpu/EmbeddingPackedParams.h>
5 #include <ATen/native/quantized/cpu/fbgemm_utils.h>
6 #include <ATen/native/quantized/cpu/qembeddingbag.h>
117 // packed_weights = torch.ops.quantized.embedding_bag_byte_prepack(weights) in qembeddingbag_byte_unpack_out()
119 // unpacked_weights = torch.ops.quantized.embedding_bag_byte_unpack(packed_weights) in qembeddingbag_byte_unpack_out()
226 std::uint8_t quantized = input_row[col / NUM_ELEM_PER_BYTE]; in _qembeddingbag_nbit_unpack_helper() local
227 quantized >>= (col % NUM_ELEM_PER_BYTE) * BIT_RATE; in _qembeddingbag_nbit_unpack_helper()
228 quantized &= (1 << BIT_RATE) - 1; in _qembeddingbag_nbit_unpack_helper()
229 output_row[col] = scale * quantized + zero_point; in _qembeddingbag_nbit_unpack_helper()
238 // The input is expected to first have quantized values,
[all …]
H A Dqlinear_prepack.cpp5 #include <ATen/native/quantized/cpu/fbgemm_utils.h>
6 #include <ATen/native/quantized/cpu/init_qnnpack.h>
7 #include <ATen/native/quantized/PackedParams.h>
8 #include <ATen/native/quantized/cpu/QnnpackUtils.h>
9 #include <ATen/native/quantized/cpu/OnednnUtils.h>
10 #include <ATen/native/quantized/cpu/QuantUtils.h>
12 #include <ATen/quantized/Quantizer.h>
69 "The weight tensor for quantized::linear_prepack (fbgemm) should" in prepack()
144 "quantized::linear_prepack (qnnpack): Weight tensor rank should be == 2"); in prepack()
156 "quantized::linear_prepack (qnnpack): Given weight of size ", in prepack()
[all …]
/aosp_15_r20/external/pytorch/test/quantization/jit/
H A Dtest_quantize_jit.py13 import torch.jit.quantized
555 # not quantized
563 "fbgemm" in torch.backends.quantized.supported_engines,
564 " Quantized operations require FBGEMM. FBGEMM is only optimized for CPUs"
979 output of all branches are quantized/observed consistently
1009 # since output for both branch are quantized
1010 # the if node is quantized consistently
1057 # make sure the quantized model is executable
1279 FileCheck().check_count("quantized::conv2d(", 4, exactly=True).run(m.graph)
1297 "quantized::linear_prepack"
[all …]
/aosp_15_r20/external/pytorch/torch/ao/quantization/
H A Dquantization_mappings.py8 import torch.ao.nn.intrinsic.quantized as nniq
9 import torch.ao.nn.intrinsic.quantized.dynamic as nniqd
12 import torch.ao.nn.quantized as nnq
13 import torch.ao.nn.quantized.dynamic as nnqd
14 import torch.ao.nn.quantized.reference as nnqr
56 # Default map for swapping float module to reference quantized modules
75 # Default map for swapping float module to quantized ones
179 # Default mapping from floating point function or torch ops to quantized ops
182 F.elu: torch.ops.quantized.elu,
183 F.hardswish: torch.ops.quantized.hardswish,
[all …]

12345678910>>...95