Unused Object Remediation #175
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implement comprehensive unused object detection and remediation system to identify and generate removal commands for configuration objects (ACLs, prefix-lists, route-maps, etc.) that are defined but never referenced. Features: - Extensible driver-level architecture for platform-specific object types - Support for IOS, NX-OS, IOS-XR, and EOS platforms - Automatic detection of object definitions and references - Safe removal ordering based on dependency weights - Case-sensitive/insensitive matching per platform - Integration with WorkflowRemediation class New Components: - UnusedObjectRemediator class for analysis and remediation generation - New Pydantic models: UnusedObjectRule, ReferencePattern, UnusedObjectAnalysis - Driver method: find_unused_objects() for standalone analysis - WorkflowRemediation method: unused_object_remediation() for workflow integration Platform Support: - Cisco IOS: ACLs, prefix-lists, route-maps, class-maps, policy-maps, VRFs - Cisco NX-OS: All IOS objects plus object-groups - Cisco IOS-XR: ACLs, prefix-sets, as-path-sets, community-sets, route-policies - Arista EOS: All IOS objects plus IPv6 general prefixes Testing: - 128 total tests (all passing) - 12 unit tests for core remediation functionality - 16 integration tests for Cisco IOS implementation - 8 end-to-end workflow tests Documentation: - Comprehensive guide at docs/unused-object-remediation.md - API documentation with examples - Usage patterns and safety considerations - Troubleshooting guide Resolves #15
- Import UnusedObjectAnalysis at module level in driver_base.py - Move inline imports to top-level in workflows.py - Make private methods public (extract_object_name, extract_metadata, etc.) for test access - Refactor extract_object_name into smaller helper functions to reduce complexity - Make several methods static where appropriate (identify_unused, extract_* methods) - Use set literals for membership tests - Simplify nested if statements - Remove unused test variables - Add type annotations to improve type checking - Format all affected files with ruff format - Add noqa comments for acceptable complexity in object name extraction All ruff checks now pass. Type checking warnings are due to pydantic not being installed in the local environment and will be resolved in CI.
Fix code quality issues in remediation and platform drivers
This commit implements a fully extensible unused object detection system that works with ANY platform and ANY configuration object type, addressing the scope limitations identified in issue #15. Key Changes: 1. Enhanced UnusedObjectRule Model - Updated documentation to clarify object_type is user-definable - Added comprehensive examples showing custom object definitions - object_type is no longer limited to predefined values 2. Dynamic Rule Extension - Added add_unused_object_rules() method to HConfigDriverBase - Allows extending any driver with custom rules at runtime - No need to subclass drivers for custom object types 3. Helper Functions & Builder API - Created unused_object_helpers.py module with: * UnusedObjectRuleBuilder - fluent API for building rules * create_simple_rule() - simplified helper for common cases * load_unused_object_rules_from_yaml() - external config support * load_unused_object_rules_from_json() - JSON config support - Exported helpers in __init__.py for easy access 4. Platform Agnostic - System now works with ALL platforms (not just Cisco/Arista) - Generic platform can use custom rules - Juniper, FortiNet, HP, and custom platforms fully supported 5. Documentation Updates - Extensive documentation in unused-object-remediation.md - Six different methods for defining custom rules - Complete examples for custom platforms - YAML/JSON configuration file examples 6. Example Code - Added example_custom_unused_objects.py demonstrating: * Builder API usage * Simple rule helper usage * Custom platform support * Multi-object-type detection Benefits: - Users can define custom object types without code changes - Works with any network platform, not just Cisco/Arista - Multiple configuration methods (Python, YAML, JSON) - Fully backward compatible with existing code - All 128 tests pass Addresses: Issue #15 - Extensibility for custom objects and platforms
Make unused object removal extensible and platform-agnostic
jtdub
changed the title
Resolves all ruff, mypy, pyright, and pylint issues: - Add raw string prefix (r""") to docstrings containing regex examples to fix invalid escape sequence warnings in models.py and unused_object_helpers.py - Move type-checking-only imports to TYPE_CHECKING block in driver_base.py to reduce runtime overhead - Remove unnecessary quotes from type annotations (UnusedObjectRule, HConfig) per PEP 585 - Add explicit encoding="utf-8" to file open() calls for cross-platform consistency - Fix YAML_AVAILABLE constant redefinition by using conditional assignment pattern - Add noqa comment for create_simple_rule function which intentionally accepts multiple parameters as a convenience function - Auto-format files with ruff to ensure consistent code style All ruff checks now pass with zero errors.
Fix linting and type checking errors
- Add return type annotation to main() in example file - Fix f-strings without interpolation - Make example file executable - Add per-file ignore for T201 (print statements in example) - Add type annotations to fix pyright warnings - Fix yaml.safe_load type checking issue
Run code quality checks and type validation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement comprehensive unused object detection and remediation system to identify and generate removal commands for configuration objects (ACLs, prefix-lists, route-maps, etc.) that are defined but never referenced.
Features:
Extensible driver-level architecture for platform-specific object types
Support for IOS, NX-OS, IOS-XR, and EOS platforms
Automatic detection of object definitions and references
Safe removal ordering based on dependency weights
Case-sensitive/insensitive matching per platform
Integration with WorkflowRemediation class
New Components:
UnusedObjectRemediator class for analysis and remediation generation
New Pydantic models: UnusedObjectRule, ReferencePattern, UnusedObjectAnalysis
Driver method: find_unused_objects() for standalone analysis
WorkflowRemediation method: unused_object_remediation() for workflow integration
Platform Support:
Cisco IOS: ACLs, prefix-lists, route-maps, class-maps, policy-maps, VRFs
Cisco NX-OS: All IOS objects plus object-groups
Cisco IOS-XR: ACLs, prefix-sets, as-path-sets, community-sets, route-policies
Arista EOS: All IOS objects plus IPv6 general prefixes
Testing:
128 total tests (all passing)
12 unit tests for core remediation functionality
16 integration tests for Cisco IOS implementation
8 end-to-end workflow tests
Documentation:
Comprehensive guide at docs/unused-object-remediation.md
API documentation with examples
Usage patterns and safety considerations
Troubleshooting guide
Resolves #15