Skip to content

feat: add modular problem-fetching workflow with persistent storage#137

Merged
khoahuynhdev merged 2 commits intokhoahuynhdev:mainfrom
khoahd-anz:feat/modular-problem-fetching-workflow
Feb 12, 2026
Merged

feat: add modular problem-fetching workflow with persistent storage#137
khoahuynhdev merged 2 commits intokhoahuynhdev:mainfrom
khoahd-anz:feat/modular-problem-fetching-workflow

Conversation

@khoahd-anz
Copy link
Contributor

Summary

Refactors the LeetCode problem-fetching logic into a reusable, modular architecture that stores problem metadata persistently in problem.md files.

What's Changed

New Components

  1. Composite Action (.github/actions/fetch-leetcode-problem)

    • ✅ Reusable action for fetching and storing LeetCode problems
    • ✅ Idempotent: skips API call if problem.md already exists
    • ✅ Outputs all necessary metadata for downstream workflows
    • ✅ Handles premium problems, duplicates, and force-refetch
  2. Python Scripts

    • scripts/convert-problem-json-to-md.py: Converts API JSON → markdown with YAML frontmatter
    • scripts/parse-problem-md.py: Parses markdown → JSON for build-prompt.py
  3. Manual Workflow (.github/workflows/fetch-problem.yml)

    • Allows ad-hoc problem fetching via workflow_dispatch
    • Commits problem.md to repository
    • Shows summary in workflow UI
  4. Complete Specification (specs/modular-problem-fetching-workflow.md)

    • Architecture design and implementation guide (1,879 lines)
    • Testing strategy and migration path

Modified Workflows

  • daily-leetcode.yml: Updated to use new composite action
    • Reads from problem.md when it exists (no API call needed!)
    • Maintains full backward compatibility

Key Benefits

🎯 Persistent Storage: Problem metadata stored in git with timestamps
Efficiency: Eliminates redundant API calls for duplicate problems
👀 Transparency: Problem data visible in GitHub UI and PRs
🧩 Modularity: Reusable components for future workflows
📊 Auditability: Track when problems were fetched and changes over time

Technical Details

problem.md Format

Each problem directory will contain a problem.md file with:

---
number: "0001"
frontend_id: "1"
title: "Two Sum"
slug: "two-sum"
difficulty: "Easy"
topics:
  - "Array"
  - "Hash Table"
acceptance_rate: 49.2
is_premium: false
created_at: "2026-02-12T08:30:19Z"
fetched_at: "2026-02-12T08:30:19Z"
link: "https://leetcode.com/problems/two-sum/"
date: "2026-02-12"
---

# 0001. Two Sum

[Clean markdown content converted from HTML]

Dependencies

  • Python 3.11+
  • Optional: html2text, pyyaml (fallback implementations included)
  • GitHub Actions automatically installs these

Testing

✅ Round-trip conversion (JSON→MD→JSON) verified
✅ Scripts work with both libraries and regex fallback
✅ Compatible with existing build-prompt.py
✅ No breaking changes to existing workflows

Migration

No breaking changes. Existing workflows continue to work as-is.

  • First daily run: Fetches from API, creates problem.md
  • Subsequent runs: Reads from problem.md (no API call!)
  • Existing 107 problems can optionally have problem.md backfilled later

How to Test

Test Manual Workflow

  1. Go to ActionsFetch LeetCode Problem
  2. Click Run workflow
  3. Enter a problem URL (e.g., https://leetcode.com/problems/two-sum/)
  4. Watch it create problems/0001-two-sum/problem.md

Test Daily Workflow

The daily workflow will automatically use the new system:

  • If problem.md exists: Read from file ⚡
  • If doesn't exist: Fetch from API and create file 🆕

Files Changed

New Files

  • .github/actions/fetch-leetcode-problem/action.yml
  • .github/workflows/fetch-problem.yml
  • scripts/convert-problem-json-to-md.py
  • scripts/parse-problem-md.py
  • specs/modular-problem-fetching-workflow.md

Modified Files

  • .github/workflows/daily-leetcode.yml

Implementation follows spec: specs/modular-problem-fetching-workflow.md

Ready for review! 🚀

## Summary

Refactors the LeetCode problem-fetching logic into a reusable, modular
architecture that stores problem metadata persistently in problem.md files.

## What's Changed

### New Components

1. **Composite Action** (`.github/actions/fetch-leetcode-problem`)
   - Reusable action for fetching and storing LeetCode problems
   - Idempotent: skips API call if problem.md already exists
   - Outputs all necessary metadata for downstream workflows
   - Handles premium problems, duplicates, and force-refetch

2. **Python Scripts**
   - `scripts/convert-problem-json-to-md.py`: Converts API JSON to markdown
   - `scripts/parse-problem-md.py`: Parses markdown back to JSON format

3. **Manual Workflow** (`.github/workflows/fetch-problem.yml`)
   - Allows ad-hoc problem fetching via workflow_dispatch
   - Commits problem.md to repository
   - Shows summary in workflow UI

4. **Specification** (`specs/modular-problem-fetching-workflow.md`)
   - Complete architecture design and implementation guide
   - Testing strategy and migration path

### Modified Workflows

- **daily-leetcode.yml**: Updated to use new composite action
  - Reads from problem.md when it exists (no API call!)
  - Maintains full backward compatibility

## Key Benefits

- **Persistent Storage**: Problem metadata stored in git with timestamps
- **Efficiency**: Eliminates redundant API calls for duplicate problems
- **Transparency**: Problem data visible in GitHub UI and PRs
- **Modularity**: Reusable components for future workflows
- **Auditability**: Track when problems were fetched and changes over time

## Technical Details

### problem.md Format

Each problem directory contains a `problem.md` file with:
- YAML frontmatter (metadata: number, title, difficulty, topics, etc.)
- Markdown body (converted from HTML for readability)
- Timestamps (created_at, fetched_at)

### Dependencies

- Python 3.11+
- Optional: html2text, pyyaml (fallback implementations included)

### Testing

- Round-trip conversion (JSON→MD→JSON) verified
- Scripts work with both libraries and regex fallback
- Compatible with existing build-prompt.py

## Migration

No breaking changes. Existing workflows continue to work. New problem
fetches will create problem.md files going forward.

---

Implements spec: specs/modular-problem-fetching-workflow.md
@khoahuynhdev
Copy link
Owner

@claude help me review this PR

@claude
Copy link

claude bot commented Feb 12, 2026

Claude encountered an error —— View job

Failed with exit code 128

I'll analyze this and get back to you.

Allow fetching today's daily challenge by leaving problem_url empty.
When no URL is provided, the composite action will fetch the active
daily coding challenge from LeetCode's GraphQL API.
@khoahuynhdev khoahuynhdev merged commit 7a8a635 into khoahuynhdev:main Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants