Skip to content

add bazel tools to generate tao build version info#5

Open
Orion34-lanbo wants to merge 1 commit intofeatures/aicompiler_rebase_20220216from
dev/bazel_tools
Open

add bazel tools to generate tao build version info#5
Orion34-lanbo wants to merge 1 commit intofeatures/aicompiler_rebase_20220216from
dev/bazel_tools

Conversation

@Orion34-lanbo
Copy link

@Orion34-lanbo Orion34-lanbo commented Mar 22, 2022

No description provided.

tanyokwok
tanyokwok reviewed Mar 22, 2022
View reviewed changes
tensorflow/workspace2.bzl
# Import third party config rules.
load("//tensorflow:version_check.bzl", "check_bazel_version_at_least")
load("//third_party/gpus:cuda_configure.bzl", "cuda_configure")
load("//third_party/blade_helper:blade_helper_configure.bzl", "blade_helper_configure")
Copy link

@tanyokwok tanyokwok Mar 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blade->baldedisc?

qiuxiafei
qiuxiafei approved these changes Mar 22, 2022
View reviewed changes
Copy link

@qiuxiafei qiuxiafei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qiuxiafei qiuxiafei self-requested a review March 22, 2022 14:04
@qiuxiafei
Copy link

qiuxiafei commented Mar 22, 2022

After taking a second look, I wonder the necessity to add these configurations. Ideally, tao bridge should be compiled without anything in this repo (aka tf_community), even though it contains code of RAL lned to tf_community directory.

@qiuxiafei
Copy link

qiuxiafei commented Mar 22, 2022

How about there two options:

  1. Remove dependencies to TF bazel ruls in tensorflow/compiler/mlir/xla/ral/BUILD. There're only 4 effective rules imported from tensorflow.bzl: tf_cc_shared_object / tf_cc_test / tf_gpu_kernel_library / tf_gpu_library. If they were replaced with standard bazel rules, the whole BUILD file become neutral to both side.
  2. Move all source RAL source files to another directory and expose filegroup targets only. Both bridge and compiler side reference the files and build with their own rule set. One shortcoming is that two nearly identical BUILD files will be maintained.

@Orion34-lanbo
Copy link
Author

Orion34-lanbo commented Mar 23, 2022

After taking a second look, I wonder the necessity to add these configurations. Ideally, tao bridge should be compiled without anything in this repo (aka tf_community), even though it contains code of RAL lned to tf_community directory.

These bazel rules are for compiler side build... bridge side compilation uses gpu related rules from tf community, using these rules may not be a problem

bladedisc pushed a commit that referenced this pull request Aug 16, 2022
@tensorflower-gardener @mihaimaruseac
Merged commit includes the following changes:
49f97f1 
467408627  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update sqlite version in TF

--
467380418  by A. Unique TensorFlower<gardener@tensorflow.org>:

    compat: Update forward compatibility horizon to 2022-08-13

--
467378663  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update GraphDef version to 1222.

--
467363891  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/b750bc2999cf02abca6ad9eeff0a04ec7bf3b683.

--
467363622  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [xla:runtime] NFC: Move constraints documentation from jitrt to xla/runtime/constraints

--
467362586  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [xla:runtime] NFC: Extract JitCompilationContext library from jitrt and move it to xla/runtime

--
467361314  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/0a042cbb5275e6ff9a3a7c2748c74df6dcede09e.

--
467360160  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [xla:runtime] NFC: Extract calling_convention library from jitrt and move it to xla/runtime

--
467341954  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Op documentation update.
    	update of g3doc/_includes/tf_passes.md

--
467341426  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Refactor SELECT_V2 in preparation for porting to TFLM.

--
467340678  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Create some global stat tracking for CompilationEnvironments. This tracking can be used to help debug cases in which multiple CompilationEnvironments are used to compile a single HloModule (which should not happen).

--
467339870  by A. Unique TensorFlower<gardener@tensorflow.org>:
    Automated rollback of changelist 467224197.

467339756  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/b20ec05d272477fa6223213687bb22145df92674.

--
467339529  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [XLA] Bugfix for gather index parallel partitioning where the sharded non-parallel dims in indices are not handled.

--
467337900  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [XLA] Minor renamings, refactorings, checks.

--
467337622  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Remove unneeded dependency.

--
467337170  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Integrate LLVM at llvm/llvm-project@2c3ca3b684bb

    Updates LLVM usage to match
    [2c3ca3b684bb](llvm/llvm-project@2c3ca3b684bb)

--
467335264  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [SavedModel Fingerprinting] Add hash #5, which represents the checkpoint.

    The `checkpoint_hash` is a hash of the serialized .index file, which is the metadata file of the TensorBundle containing a string-string table
    of the name of a tensor to its serialized BundleEntryProto. The BundleEntryProto contains a crc32 hash of the tensor contents, but not the contents of the tensor itself.

    RFC: tensorflow/community#415

--
467334010  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/76b3fea4cc9d5e7cb8a85798e41a61a55c301578.

--
467332094  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [xla:runtime] NFC: Extract executable library from jitrt and move it to xla/runtime

--
467324078  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467322782  by A. Unique TensorFlower<gardener@tensorflow.org>:

    PR #57137: [oneDNN] Skip appending kernel registration to log message for MKL ops

    Imported from GitHub PR tensorflow/tensorflow#57137

    This PR skips printing kernel registrations for MKL ops since it leads to performance drop for some eager models caused by this commit tensorflow/tensorflow@c04f65d This is a temporary fix and the condition will be removed when support for block format is removed as a more permanent fix.
    Copybara import of the project:

    --
    89c4c20fd138a0e992599ac333e1b41417bdd7a5 by Kanvi Khanna <kanvi.khanna@intel.com>:

    [oneDNN] Skip appending kernel registration to log message for MKL ops

    Merging this change closes #57137

--
467322425  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467321561  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467321058  by A. Unique TensorFlower<gardener@tensorflow.org>:

    PR #57089: [TF-TRT] Adjusting Conv2D Test Tolerance

    Imported from GitHub PR tensorflow/tensorflow#57089

    This PR adjusts & fixes the unittest tolerance for the test `Conv2DStridedNCHWTest` in INT8 mode.

    Copybara import of the project:

    --
    13e4bff8394aaa58f2cdf07b6495f6793a1d2324 by DEKHTIARJonathan <contact@jonathandekhtiar.eu>:

    [TF-TRT] Adjusting Conv2D Test Tolerance

    Merging this change closes #57089

--
467320826  by A. Unique TensorFlower<gardener@tensorflow.org>:

    PR #55804: [TF-TRT] Various Cleanups & Python Debugging Assertion Improvements

    Imported from GitHub PR tensorflow/tensorflow#55804

    This PR cleans a few spots in the code base, improves the debuggability of assertion messages in unittests. And replace `distutils.version.LooseVersion` (deprecated) with `packaging.version.Version` (new recommended API).
    Copybara import of the project:

    --
    a4d15ef2cbdcefbf634228b9a5a58a559a5206e5 by DEKHTIARJonathan <contact@jonathandekhtiar.eu>:

    [TF-TRT] Various Cleanups & Python Debugging Assertion Improvements

    Merging this change closes #55804

--
467320083  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <ostream> for 'std::ostream'

--
467319094  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467318151  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <iterator> for 'std::back_inserter'

--
467316931  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    using decl 'IsSubsetOf' is unused

--
467316097  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Move passes under tensorflow/compiler/mlir/tensorflow/.

--
467315812  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467314236  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <memory> for 'std::unique_ptr'

--
467313254  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'
    missing #include <memory> for 'std::make_unique'

--
467312293  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    using decl 'RangeSquareDataset' is unused

--
467311309  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467310637  by A. Unique TensorFlower<gardener@tensorflow.org>:

    PR #57013: [TF-TRT] Add LogSoftmax Support for TF-TRT

    Imported from GitHub PR tensorflow/tensorflow#57013

    This PR adds TF-TRT support to `tf.nn.log_softmax` operation.  This is performed using the formula `logsoftmax = logits - log(reduce_sum(exp(logits), axis=-1))` .  The implemented TRT layers are fused into a single op.

    @DEKHTIARJonathan @tfeher : Please review the changes.
    Copybara import of the project:

    --
    1a8eb9a28dc919f815cbf9c58e0c383992faa08b by Pavani Majety <pmajety@nvidia.com>:

    Add LogSoftmax conversion

    Fix Softmax comments

    [TF-TRT] Move LogSoftmax to use OpConverterBase

    Fix compiler errors

    clang-format

    Undo changes to convert_nodes.cc

    Fix comments

    Merging this change closes #57013

--
467310335  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <array> for 'std::array'

--
467310313  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update test config in cross device ops

--
467309032  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/eba528ef667653c3554984e5c05573b152c9893b.

--
467308765  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467307702  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <vector> for 'std::vector'

--
467306473  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467306092  by A. Unique TensorFlower<gardener@tensorflow.org>:

    PR #56771: Add return_index_map argument in ssim()

    Imported from GitHub PR tensorflow/tensorflow#56771

    Closes #53115
    Copybara import of the project:

    --
    8f5a1b1e8c10ae0ff705d95f605df41e674b9102 by CohenAriel <ariel17112005@gmail.com>:

    Add return_index_map argument in ssim()

    Merging this change closes #56771

--
467305190  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    missing #include <unordered_map> for 'std::unordered_map'
    missing #include <vector> for 'std::vector'
    missing #include <memory> for 'std::shared_ptr'

--
467304747  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [tfrt:jitrt] NFC: Remove Executable::KernelContext

    It was added before runtime::KernelContext and is not used anywhere. Remove it to avoid confusion. In the future we should reuse runtime::KernelContext as an extension point for user-defined memory allocation etc.

--
467303335  by A. Unique TensorFlower<gardener@tensorflow.org>:

    #tf-data-service #codehealth Clean up clang-tidy report.

    'int64' is deprecated: Use int64_t instead.

--
467301808  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Changes all local `State` or `TaskState` enum in coordination service into `CoordinatedTaskState` enum in proto.

--
467300580  by A. Unique TensorFlower<gardener@tensorflow.org>:

    lite: enable variable freezing in tf_tfl_translate tester

--
467298890  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Update TFRT dependency to use revision
    http://github.com/tensorflow/runtime/commit/9bb23f7d1ee0e9a55d26c7168790667e5266a74c.

--
467292686  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [xla:runtime] NFC: Move execution_engine library from jitrt to xla/runtime

--
467280901  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [GML] Add tests for concat in the GML tiling and fusion pipeline

--
467276349  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [GML] Implement dim-based shape reification for concat

--
467273958  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Change mutexes under stream_executor/gpu to use absl::Mutex and absl::MutexLock instead of tensorflow::mutex and tensorflow::mutex_lock. Change instance of absl::make_unique to std::make_unique

--
467272897  by A. Unique TensorFlower<gardener@tensorflow.org>:

    [tf.data] Prepend `/bufferedio/` for all paths passed to LoadDataset op.

--

PiperOrigin-RevId: 467408627
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qiuxiafei qiuxiafei Awaiting requested review from qiuxiafei

1 more reviewer

@tanyokwok tanyokwok tanyokwok left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Orion34-lanbo @qiuxiafei @tanyokwok