Skip to content

Conversation

@CharlieFRuan
Copy link
Collaborator

This PR removes excessive loggings:

  • From Harbor, via two new flags suppress_max_turns_warning and suppress_override_warnings
  • Suppresses the per-trial warnings about environment overrides and turn overrides (4 lines per rollout)
  • From LiteLLM (via Harbor), with litellm.suppress_debug_info, set_verbose, and logger level to warning for LiteLLM
    • Suppresses model provider log, trivial per-request log, etc.

In addition, rename max_episodes to max_turns.

Depends on laude-institute/harbor#561

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to reduce excessive logging from Harbor and LiteLLM. It introduces new configuration flags in default.yaml to suppress Harbor warnings and renames max_episodes to max_turns. For LiteLLM, it adds code to terminal_bench_generator.py to suppress debug info and set the log level. The changes in the YAML configuration are straightforward and look correct. However, the Python change introduces module-level logging configuration, which can cause unintended side effects. I've added a comment with a suggestion to improve this by moving the configuration logic. My review also points out a deviation from the PEP 8 style guide regarding the placement of imports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant