Skip to content

Script fixes for release #628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
elijahbenizzy merged 6 commits into from
Jan 10, 2026
Merged

Script fixes for release #628

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
66b5d62
Updates to clean up upload file names, upload everything
Jan 8, 2026
baa78e3
WIP
Jan 8, 2026
94e32f7
WIP
Jan 10, 2026
035007d
Adds instructions on voter flow
Jan 10, 2026
2f7830e
Adds forgotten license
Jan 10, 2026
504947b
Adds voter flow instructions to README/script
Jan 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions burr/cli/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,8 @@ def cli():
pass


def _build_ui():
def run_build_ui_bash_commands():
"""Execute the bash commands to build UI artifacts."""
cmd = "npm install --prefix telemetry/ui"
_command(cmd, capture_output=False)
cmd = "npm run build --prefix telemetry/ui"
Expand All @@ -146,12 +147,13 @@ def _build_ui():
_command(cmd, capture_output=False)


@cli.command()
@cli.command(name="build-ui")
def build_ui():
"""Build the UI artifacts from source."""
git_root = _get_git_root()
logger.info("UI build: using project root %s", git_root)
with cd(git_root):
_build_ui()
run_build_ui_bash_commands()


BACKEND_MODULES = {
Expand Down Expand Up @@ -246,7 +248,7 @@ def build_and_publish(prod: bool, no_wipe_dist: bool):
git_root = _get_git_root()
with cd(git_root):
logger.info("Building UI -- this may take a bit...")
_build_ui()
build_ui()
logger.info("Built UI!")
if not no_wipe_dist:
logger.info("Wiping dist/ directory for a clean publish.")
Expand Down
19 changes: 19 additions & 0 deletions docs/_templates/page.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

{% extends "!page.html" %}

{% block extrahead %}
Expand Down
30 changes: 30 additions & 0 deletions scripts/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,36 @@ python scripts/apache_release.py all 0.41.0 0 your_apache_id --no-upload

Output: `dist/` directory with tar.gz (archive + sdist), whl, plus .asc and .sha512 files. Install from the whl file to test it out after runnig the `wheel` subcommand.

## For Voters: Verifying a Release

If you're voting on a release, follow these steps to verify the release candidate:

### Quick Start

```bash
# 1. Download and extract the source archive
wget https://dist.apache.org/repos/dist/dev/incubator/burr/{VERSION}-incubating-RC{N}/apache-burr-{VERSION}-incubating.tar.gz
tar -xzf apache-burr-{VERSION}-incubating.tar.gz
cd apache-burr-{VERSION}-incubating/

# 2. Verify signatures and checksums
python scripts/verify_apache_artifacts.py signatures

# 3. Install build dependencies
pip install flit

# 4. Build the wheel from source
python scripts/apache_release.py wheel {VERSION} {RC_NUM}

# 5. Install and run the wheel
pip install "dist/apache_burr-{VERSION}-py3-none-any.whl[learn]"
burr
```

You can run tests, etc... from here

See the "Verification" section below for more detailed verification steps.

## Verification

Validate artifacts before uploading or voting. Checks GPG signatures, SHA512 checksums, archive integrity, and license compliance with Apache RAT. The `list-contents` command is useful for inspecting what's actually packaged in each artifact.
Expand Down
183 changes: 115 additions & 68 deletions scripts/apache_release.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,46 @@ def _print_step(step_num: int, total: int, description: str) -> None:
print("-" * 80)


def _run_command(
cmd: list[str],
description: str,
error_message: str,
success_message: Optional[str] = None,
capture_output: bool = True,
**kwargs,
) -> subprocess.CompletedProcess:
"""Run a subprocess command with consistent error handling and output.

Args:
cmd: Command and arguments as list
description: What we're doing (printed before running)
error_message: Error message prefix if command fails
success_message: Optional success message (printed after if provided)
capture_output: Whether to capture stdout/stderr (default True)
**kwargs: Additional arguments to pass to subprocess.run

Returns:
CompletedProcess instance
"""
if description:
print(f" {description}")

try:
result = subprocess.run(
cmd,
check=True,
capture_output=capture_output,
text=True,
**kwargs,
)
if success_message:
print(f" ✓ {success_message}")
return result
except subprocess.CalledProcessError as e:
error_detail = f": {e.stderr}" if capture_output and e.stderr else ""
_fail(f"{error_message}{error_detail}")


# ============================================================================
# Environment Validation
# ============================================================================
Expand Down Expand Up @@ -200,14 +240,13 @@ def _sign_artifact(artifact_path: str) -> tuple[str, str]:
checksum_path = f"{artifact_path}.sha512"

# GPG signature
try:
subprocess.run(
["gpg", "--armor", "--output", signature_path, "--detach-sig", artifact_path],
check=True,
)
print(f" ✓ Created GPG signature: {signature_path}")
except subprocess.CalledProcessError as e:
_fail(f"Error signing artifact: {e}")
_run_command(
["gpg", "--armor", "--output", signature_path, "--detach-sig", artifact_path],
description="",
error_message="Error signing artifact",
capture_output=False,
)
print(f" ✓ Created GPG signature: {signature_path}")

# SHA512 checksum
sha512_hash = hashlib.sha512()
Expand Down Expand Up @@ -302,26 +341,25 @@ def _create_git_archive(version: str, rc_num: str, output_dir: str = "dist") ->

os.makedirs(output_dir, exist_ok=True)

archive_name = f"apache-burr-{version}-incubating-src.tar.gz"
archive_name = f"apache-burr-{version}-incubating.tar.gz"
archive_path = os.path.join(output_dir, archive_name)
prefix = f"apache-burr-{version}-incubating-src/"

try:
subprocess.run(
[
"git",
"archive",
"HEAD",
f"--prefix={prefix}",
"--format=tar.gz",
"--output",
archive_path,
],
check=True,
)
print(f" ✓ Created git archive: {archive_path}")
except subprocess.CalledProcessError as e:
_fail(f"Error creating git archive: {e}")
prefix = f"apache-burr-{version}-incubating/"

_run_command(
[
"git",
"archive",
"HEAD",
f"--prefix={prefix}",
"--format=tar.gz",
"--output",
archive_path,
],
description="",
error_message="Error creating git archive",
capture_output=False,
)
print(f" ✓ Created git archive: {archive_path}")

file_size = os.path.getsize(archive_path)
print(f" ✓ Archive size: {file_size:,} bytes")
Expand Down Expand Up @@ -359,20 +397,15 @@ def _build_sdist_from_git(version: str, output_dir: str = "dist") -> str:
_remove_ui_build_artifacts()
_check_git_working_tree()

print(" Running flit build --format sdist...")
try:
env = os.environ.copy()
env["FLIT_USE_VCS"] = "0"
subprocess.run(
["flit", "build", "--format", "sdist"],
env=env,
capture_output=True,
text=True,
check=True,
)
print(" ✓ flit sdist created successfully")
except subprocess.CalledProcessError as e:
_fail(f"Failed to build sdist: {e.stderr}")
env = os.environ.copy()
env["FLIT_USE_VCS"] = "0"
_run_command(
["flit", "build", "--format", "sdist"],
description="Running flit build --format sdist...",
error_message="Failed to build sdist",
success_message="flit sdist created successfully",
env=env,
)

# Find and rename sdist
expected_pattern = f"dist/apache_burr-{version.lower()}.tar.gz"
Expand All @@ -383,7 +416,7 @@ def _build_sdist_from_git(version: str, output_dir: str = "dist") -> str:

original_sdist = sdist_files[0]
apache_sdist = os.path.join(
output_dir, f"apache-burr-{version.lower()}-incubating-src-sdist.tar.gz"
output_dir, f"apache-burr-{version.lower()}-incubating-sdist.tar.gz"
)

if os.path.exists(apache_sdist):
Expand All @@ -401,28 +434,46 @@ def _build_sdist_from_git(version: str, output_dir: str = "dist") -> str:


def _build_ui_artifacts() -> None:
"""Build UI artifacts using burr-admin-build-ui."""
"""Build UI artifacts (npm build + copy to burr/tracking/server/build).

This replicates the logic from burr.cli.__main__.run_build_ui_bash_commands()
without requiring burr to be installed.
"""
print("Building UI artifacts...")

ui_source_dir = "telemetry/ui"
ui_build_dir = "burr/tracking/server/build"

# Clean existing UI build
if os.path.exists(ui_build_dir):
shutil.rmtree(ui_build_dir)

# Check for burr-admin-build-ui
if shutil.which("burr-admin-build-ui") is None:
_fail("burr-admin-build-ui not found. Install with: pip install -e .[cli]")
# Install npm dependencies
_run_command(
["npm", "install", "--prefix", ui_source_dir],
description="Installing npm dependencies...",
error_message="npm install failed",
success_message="npm dependencies installed",
)

# Build UI with npm
_run_command(
["npm", "run", "build", "--prefix", ui_source_dir],
description="Building UI with npm...",
error_message="npm build failed",
success_message="npm build completed",
)

# Build UI
env = os.environ.copy()
env["BURR_PROJECT_ROOT"] = os.getcwd()
# Copy build artifacts
print(" Copying build artifacts...")
os.makedirs(ui_build_dir, exist_ok=True)
ui_output = os.path.join(ui_source_dir, "build")

try:
subprocess.run(["burr-admin-build-ui"], check=True, env=env, capture_output=True)
print(" ✓ UI artifacts built successfully")
except subprocess.CalledProcessError as e:
_fail(f"Error building UI: {e}")
shutil.copytree(ui_output, ui_build_dir, dirs_exist_ok=True)
print(" ✓ Build artifacts copied")
except Exception as e:
_fail(f"Failed to copy build artifacts: {e}")

# Verify
if not os.path.exists(ui_build_dir) or not os.listdir(ui_build_dir):
Expand Down Expand Up @@ -505,13 +556,13 @@ def _build_wheel_from_current_dir(version: str, output_dir: str = "dist") -> str
env = os.environ.copy()
env["FLIT_USE_VCS"] = "0"

subprocess.run(
_run_command(
["flit", "build", "--format", "wheel"],
description="",
error_message="Wheel build failed",
success_message="Wheel built successfully",
env=env,
check=True,
capture_output=True,
)
print(" ✓ Wheel built successfully")

# Find the wheel
wheel_pattern = f"dist/apache_burr-{version}*.whl"
Expand All @@ -525,8 +576,6 @@ def _build_wheel_from_current_dir(version: str, output_dir: str = "dist") -> str

return wheel_path

except subprocess.CalledProcessError as e:
_fail(f"Wheel build failed: {e}")
finally:
# Always restore symlinks
if copied:
Expand Down Expand Up @@ -579,7 +628,9 @@ def _collect_all_artifacts(version: str, output_dir: str = "dist") -> list[str]:

artifacts = []
for filename in os.listdir(output_dir):
if f"{version}-incubating" in filename:
# Match both incubating artifacts and wheel (which doesn't have -incubating suffix)
version_match = f"{version}-incubating" in filename or f"{version}" in filename
if version_match:
if any(filename.endswith(ext) for ext in [".tar.gz", ".whl", ".asc", ".sha512"]):
artifacts.append(os.path.join(output_dir, filename))

Expand Down Expand Up @@ -660,18 +711,14 @@ def _generate_vote_email(version: str, rc_num: str, svn_url: str) -> str:
The Git tag to be voted upon is:
{tag}

Release artifacts are signed with your GPG key. The KEYS file is available at:
Release artifacts are signed with the release manager's GPG key. The KEYS file is available at:
https://downloads.apache.org/incubator/{PROJECT_SHORT_NAME}/KEYS

Please download, verify, and test the release candidate.

Some ideas to verify the release:
1. Build from source - see README in scripts/ directory for instructions
2. Install the wheel using pip to test functionality
3. Run license verification using the verify_apache_artifacts.py script or manually check
- Verify checksums and signatures match
- Check LICENSE/NOTICE files are present
- Ensure all source files have Apache headers
For detailed step-by-step instructions on how to verify this release, please see the
"For Voters: Verifying a Release" section in the scripts/README.md file within the
source archive.

The vote will run for a minimum of 72 hours.
Please vote:
Expand Down
Loading