feat: add py_zipapp_binary and test rules for zipapp support #3539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

rickeylev merged 46 commits into bazel-contrib:main from rickeylev:refactor.pyzipapp

Feb 1, 2026

.bazelci/presubmit.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -34,6 +34,7 @@ buildifier: @@
       build_flags:
         - "--keep_going"
         - "--build_tag_filters=-integration-test"
+        - "--verbose_failures"
       test_targets:
         - "--"
         - "..."
@@ Expand Down @@

CHANGELOG.md

-Original file line number
+Diff line change
@@ Expand Up / @@ -82,6 +82,17 @@ END_UNRELEASED_TEMPLATE @@
       `true`, a `py_*` target's `pyi_srcs` attribute will be set if any `.pyi` files
       that are associated with the target's `srcs` are present.
       ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)).
+    * (zipapp) {obj}`py_zipapp_binary` and {obj}`py_zipapp_test` rules added. These
+      will replace `--build_python_zip` and the zip output group of
+      `py_binary/py_test`. The zipapp rules support more functionality, correctness,
+      and have better build performance.
+    * (toolchains) Added {obj}`PyExecToolsInfo.exec_runtime` for more easily
+      getting an RBE-compatible runtime to use for build actions.
+    * (providers) {obj}`PyExecutableInfo` has several new fields to aid packaging
+      of binaries: {obj}`PyExecutableInfo.app_runfiles`,
+      {obj}`PyExecutableInfo.interpreter_args`,
+      {obj}`PyExecutableInfo.stage2_bootstrap`, and
+      {obj}`PyExecutableInfo.venv_python_exe`.
     {#v1-8-3}
     ## [1.8.3] - 2026-01-27
@@ Expand Down @@

docs/BUILD.bazel

-Original file line number
+Diff line change
@@ Expand Up / @@ -125,10 +125,13 @@ sphinx_stardocs( @@
             "//python/private/pypi:pkg_aliases_bzl",
             "//python/private/pypi:whl_config_setting_bzl",
             "//python/private/pypi:whl_library_bzl",
+            "//python/private/zipapp:py_zipapp_rule_bzl",
             "//python/uv:lock_bzl",
             "//python/uv:uv_bzl",
             "//python/uv:uv_toolchain_bzl",
             "//python/uv:uv_toolchain_info_bzl",
+            "//python/zipapp:py_zipapp_binary_bzl",
+            "//python/zipapp:py_zipapp_test_bzl",
         ] + ([
             # This depends on @pythons_hub, which is only created under bzlmod,
             "//python/extensions:pip_bzl",
@@ Expand Down @@

python/BUILD.bazel

-Original file line number
+Diff line change
@@ Expand Up / @@ -47,6 +47,7 @@ filegroup( @@
             "//python/runfiles:distribution",
             "//python/runtime_env_toolchains:distribution",
             "//python/uv:distribution",
+            "//python/zipapp:distribution",
         ],
         visibility = ["//:__pkg__"],
     )
@@ Expand Down @@

python/private/BUILD.bazel

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -13,7 +13,7 @@
  
    # limitations under the License.

    load("@bazel_skylib//:bzl_library.bzl", "bzl_library")

    load("@bazel_skylib//rules:common_settings.bzl", "bool_setting")

    load("@bazel_skylib//rules:common_settings.bzl", "bool_flag")

    load("//python:py_binary.bzl", "py_binary")

    load("//python:py_library.bzl", "py_library")

    load(":bazel_config_mode.bzl", "bazel_config_mode")

    @@ -38,6 +38,7 @@ filegroup(
  
            "//python/private/cc:distribution",

            "//python/private/pypi:distribution",

            "//python/private/whl_filegroup:distribution",

            "//python/private/zipapp:distribution",

            "//tools/build_defs/python/private:distribution",

        ],

        visibility = ["//python:__pkg__"],

    @@ -77,7 +78,9 @@ bzl_library(
  
            ":py_info_bzl",

            ":py_internal_bzl",

            ":reexports_bzl",

            ":rule_builders_bzl",

            ":rules_cc_srcs_bzl",

            "@bazel_skylib//lib:dicts",

            "@bazel_skylib//rules:common_settings",

        ],

    )

    @@ -366,7 +369,7 @@ bzl_library(
  
        name = "py_cc_toolchain_rule_bzl",

        srcs = ["py_cc_toolchain_rule.bzl"],

        deps = [

            ":common_labels.bzl",

            ":common_labels_bzl",

            ":py_cc_toolchain_info_bzl",

            ":rules_cc_srcs_bzl",

            ":sentinel_bzl",

    @@ -460,7 +463,10 @@ bzl_library(
  
    bzl_library(

        name = "py_interpreter_program_bzl",

        srcs = ["py_interpreter_program.bzl"],

        deps = ["@bazel_skylib//rules:common_settings"],

        deps = [

            ":sentinel_bzl",

            "@bazel_skylib//rules:common_settings",

        ],

    )

    bzl_library(

    @@ -740,8 +746,8 @@ bzl_library(
  
        srcs = ["venv_runfiles.bzl"],

        deps = [

            ":common_bzl",

            ":py_info.bzl",

            ":py_internal.bzl",

            ":py_info_bzl",

            ":py_internal_bzl",

            "@bazel_skylib//lib:paths",

        ],

    )

    @@ -786,14 +792,6 @@ filegroup(
  
        visibility = ["//visibility:public"],

    )

    filegroup(

        name = "zip_main_template",

        srcs = ["zip_main_template.py"],

        # Not actually public. Only public because it's an implicit dependency of

        # py_runtime.

        visibility = ["//visibility:public"],

    )

    filegroup(

        name = "site_init_template",

        srcs = ["site_init_template.py"],

    @@ -845,7 +843,7 @@ bazel_config_mode(name = "bazel_config_mode")
  
    # This should only be set by analysis tests to expose additional metadata to

    # aid testing, so a setting instead of a flag.

    bool_setting(

    bool_flag(

        name = "visible_for_testing",

        build_setting_default = False,

        # This is only because it is an implicit dependency by the toolchains.

python/private/common.bzl

-Original file line number
+Diff line change
@@ Expand Up / @@ -16,6 +16,10 @@ @@
     load("@bazel_skylib//lib:paths.bzl", "paths")
     load("@rules_cc//cc/common:cc_common.bzl", "cc_common")
     load("@rules_cc//cc/common:cc_info.bzl", "CcInfo")
+    load("@rules_python_internal//:rules_python_config.bzl", "config")
+    load("//python/private:py_interpreter_program.bzl", "PyInterpreterProgramInfo")
+    load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE")
+    load(":builders.bzl", "builders")
     load(":cc_helper.bzl", "cc_helper")
     load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo")
     load(":py_info.bzl", "PyInfo", "PyInfoBuilder")
@@ Expand All / @@ -41,6 +45,17 @@ PYTHON_FILE_EXTENSIONS = [ @@
         "so",  # Python C modules, usually Linux
     ]
+    BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [
+        "//command_line_option:build_python_zip",
+    ]
+    def maybe_builtin_build_python_zip(value, settings = None):
+        settings = settings or {}
+        if not config.bazel_10_or_later:
+            settings["//command_line_option:build_python_zip"] = value
+        return settings
     def create_binary_semantics_struct(
             *,
             get_native_deps_dso_name,
@@ Expand Down Expand Up / @@ -463,3 +478,136 @@ def collect_deps(ctx, extra_deps = []): @@
             deps = list(deps)
             deps.extend(extra_deps)
         return deps
+    def maybe_create_repo_mapping(ctx, *, runfiles):
+        """Creates a repo mapping manifest if bzlmod is enabled.
+        There isn't a way to reference the repo mapping Bazel implicitly
+        creates, so we have to manually create it ourselves.
+        Args:
+            ctx: rule ctx.
+            runfiles: runfiles object to generate mapping for.
+        Returns:
+            File object if the repo mapping manifest was created, None otherwise.
+        """
+        if not py_internal.is_bzlmod_enabled(ctx):
+            return None
+        # We have to add `.custom` because `{name}.repo_mapping` is used by Bazel
+        # internally.
+        repo_mapping_manifest = ctx.actions.declare_file(ctx.label.name + ".custom.repo_mapping")
+        py_internal.create_repo_mapping_manifest(
+            ctx = ctx,
+            runfiles = runfiles,
+            output = repo_mapping_manifest,
+        )
+        return repo_mapping_manifest
+    def actions_run(
+            ctx,
+            *,
+            executable,
+            toolchain = None,
+            **kwargs):
+        """Runs a tool as an action, supporting py_interpreter_program targets.
+        This is wrapper around `ctx.actions.run()` that sets some useful defaults,
+        supports handling `py_interpreter_program` targets, and some other features
+        to let the target being run influence the action invocation.
+        Args:
+            ctx: The rule context. The rule must have the
+                `//python:exec_tools_toolchain_type` toolchain available.
+            executable: The executable to run. This can be a target that provides
+                `PyInterpreterProgramInfo` or a regular executable target. If it
+                provides `testing.ExecutionInfo`, the requirements will be added to
+                the execution requirements.
+            toolchain: The toolchain type to use. Must be None or
+                `//python:exec_tools_toolchain_type`.
+            **kwargs: Additional arguments to pass to `ctx.actions.run()`.
+                `mnemonic` and `progress_message` are required.
+        """
+        mnemonic = kwargs.pop("mnemonic", None)
+        if not mnemonic:
+            fail("actions_run: missing required argument 'mnemonic'")
+        progress_message = kwargs.pop("progress_message", None)
+        if not progress_message:
+            fail("actions_run: missing required argument 'progress_message'")
+        tools = kwargs.pop("tools", None)
+        tools = list(tools) if tools else []
+        action_arguments = []
+        action_env = {
+            "PYTHONHASHSEED": "0",  # Helps avoid non-deterministic behavior
+            "PYTHONNOUSERSITE": "1",  # Helps avoid non-deterministic behavior
+            "PYTHONSAFEPATH": "1",  # Helps avoid incorrect import issues
+        }
+        default_info = executable[DefaultInfo]
+        action_inputs = builders.DepsetBuilder()
+        action_inputs.add(kwargs.pop("inputs", None) or [])
+        if PyInterpreterProgramInfo in executable:
+            if toolchain and toolchain != EXEC_TOOLS_TOOLCHAIN_TYPE:
+                fail(("Action {}: tool {} provides PyInterpreterProgramInfo, which " +
+                      "requires the `toolchain` arg be " +
+                      "None or {}, got: {}").format(
+                    mnemonic,
+                    executable,
+                    EXEC_TOOLS_TOOLCHAIN_TYPE,
+                    toolchain,
+                ))
+            exec_runtime = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools.exec_runtime
+            if exec_runtime.interpreter:
+                action_exe = exec_runtime.interpreter
+                action_inputs.add(exec_runtime.files)
+            elif exec_runtime.interpreter_path:
+                action_exe = exec_runtime.interpreter_path
+            else:
+                fail(("Action {}: PyRuntimeInfo from exec tools toolchain is " +
+                      "malformed: requires one of `interpreter` or " +
+                      "`interpreter_path` set").format(
+                    mnemonic,
+                ))
+            program_info = executable[PyInterpreterProgramInfo]
+            interpreter_args = ctx.actions.args()
+            interpreter_args.add_all(program_info.interpreter_args)
+            interpreter_args.add(default_info.files_to_run.executable)
+            action_arguments.append(interpreter_args)
+            action_env.update(program_info.env)
+            tools.append(default_info.files_to_run)
+            toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE
+        else:
+            action_exe = executable[DefaultInfo].files_to_run
+        execution_requirements = {}
+        if testing.ExecutionInfo in executable:
+            execution_requirements.update(executable[testing.ExecutionInfo].requirements)
+        # Give precedence to caller's execution requirements.
+        execution_requirements.update(kwargs.pop("execution_requirements", None) or {})
+        # Give precedence to caller's env.
+        action_env.update(kwargs.pop("env", None) or {})
+        # Handle arguments=None
+        action_arguments.extend(list(kwargs.pop("arguments", None) or []))
+        ctx.actions.run(
+            executable = action_exe,
+            arguments = action_arguments,
+            tools = tools,
+            env = action_env,
+            execution_requirements = execution_requirements,
+            toolchain = toolchain,
+            mnemonic = mnemonic,
+            progress_message = progress_message,
+            inputs = action_inputs.build(),
+            **kwargs
+        )

python/private/py_exec_tools_info.bzl

-Original file line number
+Diff line change
@@ Expand Up / @@ -26,7 +26,7 @@ e.g. if all the exec tools are prebuilt binaries. @@
     :::{note}
     this interpreter is really only for use when a build tool cannot use
-    the Python toolchain itself. When possible, prefeer to define a `py_binary`
+    the Python toolchain itself. When possible, prefer to define a `py_binary`
     instead and use it via a `cfg=exec` attribute; this makes it much easier
     to setup the runtime environment for the binary. See also:
     `py_interpreter_program` rule.
@@ Expand All / @@ -39,11 +39,27 @@ toolchain. @@
     :::
     :::{warning}
-    This does not work correctly in case of RBE, please use exec_runtime instead.
+    This does not work correctly with RBE. Use {obj}`exec_runtime` instead.
     Once https://github.com/bazelbuild/bazel/issues/23620 is resolved this warning
     may be removed.
     :::
+    """,
+            "exec_runtime": """
+    :type: PyRuntimeInfo | None
+    The Python runtime to use for the exec configuration.
+    :::{versionadded} VERSION_NEXT_FEATURE
+    In prior versions, the equivalent can be obtained using:
+    ```
+    exec_runtime = (
+        ctx.toolchains["@rules_python//python:exec_tools_toolchain_type"].
+            exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime
+        )
+    ```
+    :::
     """,
             "precompiler": """
     :type: Target | None
@@ Expand Down @@

python/private/py_exec_tools_toolchain.bzl

-Original file line number
+Diff line change
@@ Expand Up / @@ -30,11 +30,17 @@ def _py_exec_tools_toolchain_impl(ctx): @@
         if SentinelInfo in ctx.attr.exec_interpreter:
             exec_interpreter = None
+        exec_runtime = None
+        if exec_interpreter != None and platform_common.ToolchainInfo in exec_interpreter:
+            tc = exec_interpreter[platform_common.ToolchainInfo]
+            exec_runtime = getattr(tc, "py3_runtime", None)
         return [
             platform_common.ToolchainInfo(
                 exec_tools = PyExecToolsInfo(
                     exec_interpreter = exec_interpreter,
                     precompiler = ctx.attr.precompiler,
+                    exec_runtime = exec_runtime,
                 ),
                 **extra_kwargs
             ),
@@ Expand Down Expand Up / @@ -104,6 +110,12 @@ def _current_interpreter_executable_impl(ctx): @@
         # re-exec. If it's not a recognized name, then they fail.
         if runtime.interpreter:
             executable = ctx.actions.declare_file(runtime.interpreter.basename)
+            # NOTE: Using ctx.actions.symlink() here doesn't always work with RBE
+            # because it's not guaranteed that it will materialize as a symlink, but
+            # we rely on it being a symlink so that Python can find its actual
+            # PYTHONHOME.
+            # See https://github.com/bazelbuild/bazel/issues/23620
             ctx.actions.symlink(output = executable, target_file = runtime.interpreter, is_executable = True)
         else:
             executable = ctx.actions.declare_symlink(paths.basename(runtime.interpreter_path))
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: add py_zipapp_binary and test rules for zipapp support #3539

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

Uh oh!

Uh oh!

Uh oh!