… ceil_mode and count_include_pad are True. (#9835)

* Added the offset[i] for getting the correct  boundary
* Added corresponding test case

This was inadvertently removed by #9554

Co-authored-by: driazati <driazati@users.noreply.github.com>

This is necessary to make the Rust bindings work.

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>

* Update Dockerfile.ci_arm

* Update Dockerfile.ci_cpu

* Update Dockerfile.ci_gpu

* Update Dockerfile.ci_i386

* Update Dockerfile.ci_lint

* Update Dockerfile.ci_qemu

* Update Dockerfile.ci_wasm

* [USMP] Hill Climb allocator

This PR adds HillClimb allocator "tir.usmp.algo.hill_climb"
to the memory allocation algorithm set.

Change-Id: Ib7485df93757eb512da040528ec86c920db8d03b

* requested changes

Change-Id: I6700a24c1608d92f87be7dde33cc24f5de1f7063

* Conda-related linter small fixes

Change-Id: I0dac5c6d75ade8f813b077c8708aad59d2722933

* Moved implementation from greedy.h to greedy.cc

Change-Id: If8ed159eceef32d3f22b51e0252161d09222eb1e

* Integrated into test_tir_usmp_algo.py unit test

Added "hill_climb" into test_tir_usmp_algo.py
Amended sorting to be consistent with "greedy" family

Change-Id: I8e9f5282f15baaab71d6d129aeb9643376b14763

default list

… keep_dim option in fully connected (#9840)

* Support -> Change the output shape calculation based on keep_dim option

* Support -> Change the output shape calculation based on keep_dim option

* Support -> Change the output shape calculation based on keep_dim option

* Support -> Change the output shape calculation based on keep_dim option

* Change the output shape calculation based on keep_dim option in fully connected

* TODO : Need to construct a fc op with (keep_num_dims == True)

* TODO : Need to construct a fc op with (keep_num_dims == True)

…(#9880)

* encode conditional accesses info into block read/write regions

* compare ir after simplify

…t work) (#9898)

* fixed int8 dense offload for cublas

* support OHWI kernel layout in qnn.conv2d

* fixed reduction axis

* add cublas int8 qnn test

* lint

* fix mix up of channels with conv2d-transpose

* add grouped convtranspose tests

* turn off groups for non-llvm test

* [Fix] relay onnx frontend bug when [A, B, M, N] * [1, B, N, K]

* fix line

Co-authored-by: tomoyazhang <tomoyazhang@tencent.com>

* [Caffe Frontend] supporting group > 1 cases for Deconv op

- Handling group > 1 cases, assuming group == output channels
- Simply decomposed into Relay split, conv2d_transposed, and multi-leveled concatenate ops
- Added some test cases

Signed-off-by: zotanika <zotanika@gmail.com>

* [Caffe Frontend] amending a test case for Deconv op

Signed-off-by: zotanika <zotanika@gmail.com>

* explicit importing tvm.testing

* changing split axis to 0, according to PR #9336

* [Caffe Frontend] adding Reduction op

* reformatting Reduction op test script

* reformatting Reduction test script

* [Caffe frontend] Reduction op
- adding more test cases; handling '0 < axis < num_axes - 1' case to give the result equivalent to Caffe framework
- skipping Relay multiplication if coeff is 1

Signed-off-by: zotanika <zotanika@gmail.com>

* linting test script

* linting

* [Caffe Frontend] Supporting multiple grouped(channel-wise) Deconv op

* Handling group > 1 cases, assuming group == output channels
* Decomposed into Relay split, transposed conv, and multi-leveled concatenation.
* Added some test cases.

Signed-off-by: zotanika <zotanika@gmail.com>

* [Caffe Frontend] supporting variable number of inputs for Eltwise

* extra handling of rest inputs for PROD, SUM, MAX operations
* extra testcases

Signed-off-by: zotanika <zotanika@gmail.com>

* formatting fix

* [Caffe Frontend] reverting codes related Reduction for splitting PR

* Revert "[Caffe Frontend] Supporting multiple grouped(channel-wise) Deconv op"

This reverts commit 43e25e552b790ce9a38fdbcfb3ddf2075c253e20.

* instant fix against docker format error

* instant fix against docker format error

* instant fix against docker format error

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* [microNPU] Remove remaining UnsupportedLayout checks

In #9508 the decision was made to remove the UnsupportedLayout exception
and the checks that throw it, this PR is cleaning up some that remained.

Change-Id: I83bfe233381b83af886343c9569db753e33f9059

* fix lint

Change-Id: I67c1a5371f0b2e51b6cd39435ef4073d8d17af51

* [microNPU][2c] Initial Performance Model

* Added the pre-computed performance modelling per block.
* Added the aggregation of cycles given a stripe config.
* Implemented the op-specific performance code for conv2d.
* Created a DeviceConfig class to hold constant performance related data
that is dependent on the accelerator configuration
* Added generation of all valid block configs. This is pre-computed and
given as an argument when constructing EthosuParts.
* Implemented selection of the block config that gives the least amount
of data read given a StripeConfig.

* Add test guards

* Extended block config testing

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

…_shape_from_cudnn` (#9948)

* Introduce SetConvdescriptors to refactor cudnn/conv_forward.cc

* more refactor

* remove cudnn get output

* cpplint

* [microNPU] Add support for scalar values

PR #9515 enabled support for scalar constants, but didn't consider the
case of a scalar value where the underlying constant data does not have
a shape i.e. `constant.shape == []`. See the test case for a visual
differece when the scalar value is 1.

Change-Id: Id7a238cb5bf999dd5a8428c097202f9fb940a5f0

* Fix failing test by removing constant

Before this PR scalar constants were handled differently so this test
was able to pass. Now that scalar constants are handled in the same
manner as tensor constants, the test fails since unexpected tir is
produced in the compilation pipeline. Since the relay used in this test
case is not expected to be produced by higher levels of the compiler,
removing this constant for now.

Change-Id: I4ea5155778809041339e6faac05af3f72c3e3ea5

* clean up finding tensor from inputs

Change-Id: Ideccf84f8c9149148ff23e2406229cf637c982a3

These are required for running the demos under ci_qemu in combination with Zephyr

… modules parameters map. (#9846)

* [Runtime][Pipeline executor] Global parameters group name and runtime
modules parameters map.

Solution:
To support on the fly parameters setting for each runtime module
in pipeline executor, we create a feature that use global parameters
group name to map the runtime module parameter, after such map
relation get created user can do the on the fly parameters setting
by using the parameters group name.

trigger build.

fix ut issue.

polish comments.

Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update src/runtime/pipeline/pipeline_executor.h

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update src/runtime/pipeline/pipeline_struct.h

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

address review comments.

* Update python/tvm/contrib/pipeline_executor.py

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

* fix plint issue.

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

* jenkinsfile and one test

* formatting

* swtich to proper repo for docker

* fix missing - with _

* jostle

* upgrade to latest images

* jenkinsfile and one test

* formatting

* swtich to proper repo for docker

* fix missing - with _

* upgrade to latest images

* jostle ci

* update with official images

* jostle ci

Appended fused operations in cov2d for int8 were computed in a separate
loop from the main conv2d computation:
```
for i in ... parallel
  for j in ...
    accumulator = 0
    for k in ..
      vectorized_multiply_add(accumulator, data, kernel)
    out = accumulator
  for k in ..
    out = out + fused subsequent ops
```
This patch moves the fused ops one more loop nesting inwards to get
```
for i in ... parallel
  for j in ...
    accumulator = 0
    for k in ..
      vectorized_multiply_add(accumulator, data, kernel)
    out = accumulator + fused subsequent ops
```
On quantized mobilenetv2, this results in approximately a 30% speedup.

Type annotations don't do anything, the type conversion needs to be
explicit.

This fixes compilation error with libstdc++.

As the following parameters are both Expr.

```
zero_point : tvm.relay.Expr
scale : tvm.relay.Expr
```

It's really get me confused when I followed the python arg hints to create a relay and without success.

In case that SSH_AUTH_SOCK is defined, two items will be added to the
docker run command:
1) A propageted ssh authentication socket value , to support
   an underlying ssh calls inside the running contianer.
2) A mounted volume for ssh channel.

Co-authored-by: Samuel Panijel <samuel.panijel@arm.com>

This commit integrates USMP with the AoT executor codegen. Additionally, this commit introduces two PassContext options to disable_usmp and disable_storage_rewrite.

Moved PrintType from codegen_c.cc to codegen_source_base.cc to be accessible by source_module.cc

Moved runtime::metadata to be ExecutorCodegeMetadata as it contains metadata produced by ExecutorCodegen for actual code generation (not a runtime component).

* Disallow copy to/from external HexagonBuffer

* change nbytes -> allocation_nbytes for clarity

* retrigger ci

* fix to stablize the var orders when solve bounds in region analysis

* change to std::find_if since num of vars is generally small

* Add contribute page about CI

This adds some docs with a description of the TVM CI and some usage instructions to both make contributing more friendly and educate existing developers about how CI runs. Bikeshedding on content is welcome

Note: The TODOs are blocked on some other PRs and will be done before landing

* docker instructions

* Comments

* Rebase

Co-authored-by: driazati <driazati@users.noreply.github.com>

* upd

* lint

* python plumbing

* add cpp def

* legalize worked

* clean up

* layout conversion doesnt work

* extract wgrad body

* fix convert layout

* black

* fix kernel size

* revert irrelevant change

* add doc, clarify the meanings of parameters

* update layout convert

* test passed

* fixed layout conversion

* update convert layout

* remove print

* remove layout convert for now

* minor fix

* removed unused import

* add wgrad python reference

* add test stub

* add doc

* test other stride and pad

* tweak

* more pylint filter

* fix typo in doc

* swap arg order (data, grad) to be consistent with conv2d_transpose(dgrad)

* add rfactor

* format

* fix ci

* Add action to label mergeable PRs

Developers often have to ping a committer once their PRs are both passing in CI and are approved. This helps facilitate this process by marking such PRs with a label `ready-for-merge` so committers can easily filter for outstanding PRs that need attention.

* Fix lint and add tests

* Add Action to add cc'ed people as reviewers

This provides a mechanism for non-triager/reviewer/committer PR authors to request reviews through GitHub. Anyone that is referenced by `cc @username` in a PR body will be added as a reviewer (GitHub will limit the reviewers to those with actual permissions to leave reviews so the script to add can be simple).

* remove merge bot stuff

* Fix target triggers

Co-authored-by: driazati <driazati@users.noreply.github.com>

…sources which require a .cpp/.cc file extension (#9243)

* Allow export of C++ kernels using correct file extension

* [WIP] Set module_key=c for CSourceCrtMetadataModuleNode to temporarily fix failing tests

I realized that the module format `cc` is currently already used by the `CSourceCrtMetadataModuleNode` declared in `src/target/source/source_module.cc`.
This needs to be discussed first to decide if either the module_key should be changed or the test cases expecting the systemlib kernel (e.g. `default_lib0.c`) to have a `.c` extension.

* Update Makefiles used by tests/python/relay/aot/ to support C++ file extensions

AOT: Add c++ support to aot_test.mk
AOT: Add c++ support to corstone300.mk

* Add missing definition of GetFormat to cmsisnn and ethosn codegens (WIP)

* Resolve PR comments

* lint python/tvm/runtime/module.py

* fix EthosUModuleNode for CI

* Fix: detect empty module.format

* Add error message to assertion

* Lint python/tvm/runtime/module.py

* finish rpc and shape_dict in tut

* added more to rpc

* tutorial edits

* added tutorial to docs in howto

* accidentally had two copies of tutorial

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Apply suggestions from code review

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* Update gallery/how_to/use_tvms_python_api/tvmc_python.py

Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

* added Leandro's suggestions

* added example model at top

* added example model, blacked it

* trying to get docs to build

* underline too short for title

* forgot Jetson info, added Chris H comments

* reformatting text

* black

* hitting code block issue, trying to debug

* added spaces after the python codeblock

* black

* changing formatting

* touching up more edits'

* more touchups

* changed location of file to tutorial section

* changing doc location

* broke the order of the docs somehow

* fixed it yayy

* added additional indentation

* black'd

Co-authored-by: CircleSpin <jocelyn@pop-os.localdomain>
Co-authored-by: Leandro Nunes <leandro.nunes@arm.com>

…D (#9963)

* [microNPU][2d] Add more Part matchers for the cascader

Adds Part matchers for ethosu_depthwise_conv2d,
ethosu_pooling and ethosu_binary_elementwise. Also
adds additional testing for the CascaderGraph
creation.

Co-authored-by: Jacob Bohlin <jacob.bohlin@arm.com>

* Extended testing for block config

* Add test guards

Co-authored-by: Matthew Barrett <matthew.barrett@arm.com>

…ey (#9831)

are running

Moves LayoutOptimizer and LUTOptimizer passes to be a module pass,
rather than a function pass. This is because it was found that these
passes were not running in the NPU compilation flow. In addition, a
test for both LayoutOptimizer and LUTOptimizer has been added to check
that the passes are running in the compilation pipeline of the NPU.

Change-Id: I5145c6f02eeb0daea3cdba56198e0804ec32f351

This fixes the Rust path to ensure `cargo` is accessible in the
container.

As ci_qemu is targetted at microTVM it likely won't make use of wasmtime
as a dependency - it's used instead for the JS and WASM standalone applications as
far as I can see.

* [Relay] Fix a bug in tensor_array_scatter

  tensor_array_scatter constructs helper functions according to dtype
  and shape of element. When there are multiple scatter operations with
  same dtype and element shape but different indicies_shape, there will
  be name conflict in prelude.

* Refine get_name

* Fix typos.

* Missed funtion -> function.

* [skip ci] Make cc bot skip errors

This adds some better logging and ignores errors (this job shouldn't ever show up as a PR failure) so we can diagnose things like https://github.com/apache/tvm/runs/4873810315?check_suite_focus=true

* Submit cc'ed reviewers one at a time

Co-authored-by: driazati <driazati@users.noreply.github.com>

* Don't use std::move in WithFields

* lint

* Revert LLVM header change

* Trigger

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* First pass on modifying Keras importer to handle TimeDistributed

* Use squeeze inside TimeDistributed, add tests

* linter fixes

* More linting

* Even more linting

* Fix unused argument annotations

* Forgot one pylint annotation

* Forgot to set up data layout in _convert_activation

* Decouple data_layout from etab

* Linting fix

* Forgot to set data_layout argument

* Missed an etab.data_format, also test_conv1d was not in the test file's main

* Rebase fixes

* Linting fix

* _convert_lambda needs a data layout argument too

* linting fix too

* Lint the test file too

* Redundant variables

* Simplify further

* Another simplification

Co-authored-by: Steven Lyubomirsky <slyubomirsky@octoml.ai>

Some platforms (e.g. FreeBSD) use clang as the default OS compiler,
and there is no g++.

Fix a issue that user name like aaa.bb can't be added to docker container

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

…021)

* [TOPI,CUDA] Don't enable cudnn conv2d kernel if is not supported

Specifically, check that layout is not NCHW if datatype is int8.

* remove all conv2d_cudnn int8 support

…ith predicate (#10016)

A spin off of #9872, this adds an env variable `TVM_BACKTRACE_LIMIT` which can be set to an integer to limit the frames printed out on errors. This can make it easier to run interactive TVM scripts with errors since the stack traces are often long (70+ frames).

```bash
export TVM_BACKTRACE_LIMIT=5
python some_code_with_an_error.py
```

cc @tkonolige

Co-authored-by: driazati <driazati@users.noreply.github.com>

* [MetaSchedule] disallow_dynamic_loop

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* Update src/meta_schedule/postproc/disallow_dynamic_loop.cc

Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* Dgrad nchw, nhwc, fp16 working

commit 426e5dca446a27da49270f45171b58f1bfa21fa9
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 11:48:53 2022 +0900

    black

commit 211a58b80f4d0f0b5b0230720e41f35e50cb1eaf
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 11:43:52 2022 +0900

    fp16 also works

commit c2a34d473b063873628bff00e51a44cd8e4d0e4f
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 11:36:36 2022 +0900

    nhwc test also worked

commit c0609ab147fef30c230a94d16b6c1ba35f7dd9c0
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 11:21:23 2022 +0900

    nchw test worked

commit 2bf68c72763708151e9f49f09916a210b2547be8
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 10:41:35 2022 +0900

    add test stub

commit c86b1288d5e371f12cba4e1b1866966cb9264401
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 10:32:09 2022 +0900

    add python definition stub

commit 3166952f9673376801bf4b5b39eeb6f89452f30a
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 06:57:18 2022 +0900

    bwd filter compiled

commit e311ba3d05c5f9424ecb952cb5a520ce81a0828a
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 06:27:55 2022 +0900

    dgrad compiled

commit 47f35beb5eeeb7cbf9f6ec7cf8f5c80c65e8da46
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Tue Jan 18 06:16:43 2022 +0900

    add dgrad stub

commit ebed032d15b1c3895f541c46ce5d80b6dd769034
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Mon Jan 17 17:01:56 2022 +0900

    cpplint

commit 834f54a8c13512130e7d91ca0f54268dc06c5481
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Mon Jan 17 16:55:58 2022 +0900

    remove cudnn get output

commit dcbd9c95fdb8ffef9db9c2350430b270461a31c3
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Mon Jan 17 16:28:07 2022 +0900

    more refactor

commit 146464e8496fff972bdb1687c4e9d432fe3278d5
Author: Masahiro Masuda <masahi129@gmail.com>
Date:   Mon Jan 17 15:57:35 2022 +0900

    Introduce SetConvdescriptors to refactor cudnn/conv_forward.cc

* add python function for cudnn wgrad

* adding wgrad test

* black

* wgrad nchw and nhwc worked

* remove bwd algo name stuff

* compute output shape properly

* swap arg order in wgrad

* add kernel size arg in test

* black

* cleanup

* more fix

* fix dgrad test

* support running relay conv2d_backward_weight directly with cudnn

* black

* refactor reference function to support nhwc

* removed unused function

* lint

* enable offloading conv2d_transpose to cudnn dgrad

* relax tol

* name fix, remove print

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* [microNPU] Add support for requantize

Adds support for stand-alone requantize operation which is legalized to
an identity operation on the NPU.

Change-Id: Ie2450c5fc72f405eddf517593236074aa4716c3b

* fix concatenate tests failing due to not being bit exact

Since requantize is now offloaded, concatenate tests were failing
due a reference not being used.

Change-Id: I44b26b5daecfefb776ca19e6646f3690f5570f52

* test multiple requantize offload

Change-Id: I60a3283461a7a7083c05289e84f570698388077b

* address comments

Change-Id: I7196a0fa468eb7c6a96f2b8a68f3a2dcf5a5693c

…979)

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

Change-Id: I6a3ba9db3e3cb2bd7c10383ebd52f9a1cdad74d0

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

* Addressing comments

Change-Id: I98fcdf95bf408700968827e1abd084a916b3b21c

* [CMSIS-NN] Update microNPU demo to include offloading to CMSIS-NN

    * Addressing comments
    * Remove build folder before running demo to address #10020

Change-Id: Ifa7ad3ff431f427f8afb8b3c9f06711b3b59ad62

* Correctly filter tvmc Targets

Fixed logic to check for >2 TVM Target to be based on none-hybrid
Targets only

Co-authored-by: Chris Sidebottom <chris.sidebottom@arm.com>

* Add qnn.rsqrt op

* Add comment

* [Hexagon] Do not auto-build apps when building TVM

The Hexagon cmakes have recently become unwieldy due to a complex
network of dependencies between various automatically built components.
This was in large part because of trying to automatically build some
apps, which then tried to build TVM runtimes again, but with their
own configurations.

This patch removes the ability to automatically build any Hexagon-
-related apps from the main TVM build. The following cmake options
are now deprecated:
  - `USE_HEXAGON_LAUNCHER`
  - `USE_HEXAGON_PROXY_RPC`

In order to build the binaries needed for HexagonLauncher from
tvm.contrib.hexagon:
  - Build TVM+runtime for x86, with codegen for Hexagon enabled.
    This can be done via `USE_HEXAGON_DEVICE=sim` or `target`.
  - Build Android runtime and tvm_rpc with `-DUSE_RPC=ON`,
    `-DUSE_CPP_RPC=ON`, and `-DUSE_HEXAGON_RPC=ON`.
  - Build Hexagon runtime with `-DUSE_HEXAGON_RPC=ON`, and
    `-DBUILD_STATIC_RUNTIME=ON`.

* Add README.md

* Restart CI

* Add optional variable to set output directory

Adding interfaces into Pipeline Executor to "run", "stop","set input",
and "get input" from the pipeline executor,

In this patch, we also implemented the "BackendRuntime" structure to
wrap the graph runtime interface in order to support  pipeline executor
interface and implement data copy method. This method is used to
transfer data between two backend runtimes.

…ial (#10067)

…036)

* introduce profile_all_alignments option

* add profile_all_alignment option to API

* wip

* fixed dynamic case

* black

* update gen_gemm too

* minor improvement

* fix

* all tests work

* add doc

* fixed for sm = 75 case

* fix typo

* remove unused import

* profile_all -> find_first_valid

* fix

* Add ApplyHisotryBest.

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>

* Retrigger CI.

* Update integration.py

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>
Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>

* mutate-unroll

* mutate-unroll

* WIP

* WIP

* WIP

* test cases

* add examples

* lint

* Amend co-authors information

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>

* WIP

* address comments and changed tensorized comparator

* update

* nit

* fix example

* lint

* lint

* lint

* remove unused

* trigger ci

* clang-format

* fix

* rebase

Co-authored-by: Siyuan Feng <Hzfengsy@sjtu.edu.cn>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>

…ardware (#9993)

* Add env variable to micro tflite tutorial

* Address @gromero comments

* address @areusch comment

* fix scope

* trigger

* trigger

This commit introduces BaseAddress ObjectRef to determine
base addresses in the codegen for microNPU. This is
required when multiple memory pools become available. Thus,
base addresses could not be statically determined in the
source module.

…ets with (#9637)

sse4.1 support

… than call node (#10069)

Co-authored-by: pranav jonnalagadda-SJ1 Eng_ML <pjonnalagadd@sj1mach1.caveonetworks.com>

…(#9999)

Co-authored-by: wangjiuyang <wang.jiuyang@intellif.com>

* multi level tiling

* remove tensor core related code

* pylint

* fix

Co-authored-by: Junru Shao <junrushao1994@gmail.com>

…#10072)

Because of the failure of LSTM conversion from Pytorch

…ort tensoflow 2.6 (#9978)

On tensorflow 2.4 the test is expected to fail as the generated graph is not forzen.
On tensorflow 2.6 the generated graph is identified as frozen, therefore the test is not needed

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

Co-authored-by: Junru Shao <junrushao1994@gmail.com>
Co-authored-by: Xiyou Zhou <xiyou@octoml.ai>
Co-authored-by: Bohan Hou <32121147+spectrometerHBH@users.noreply.github.com>
Co-authored-by: Ruihang Lai <lairuihangdongdong@qq.com>
Co-authored-by: Hongyi Jin <3231950289@qq.com>
Co-authored-by: Wuwei Lin <wuwei@apache.org>

* Fix warning "unsafe mix of type 'const int64_t' and type 'bool' in
operation" occurring in tvm::tir::HasAnn

* Suppress warning "destructor never returns, potential memory leak"
occurring in tvm::runtime::detail::LogFatal::~LogFatal

The top-level makefile should delegate `make clean` to the cmake
folder of each enabled build, similar to the existing delegation of
`make all` and `make runtime`.

…s (#10047)

* fix after cr

* fix after cr 2

* emptycommit

* emptycommit 2nd try

…052)

Setting Android toolchain via CMAKE_TOOLCHAIN_FILE also causes necessary
flags to be added. Also, newer versions of the Android NDK no longer ship
libgcc.a, so this takes care of that as well.