Skip to content

Added script - #8

Draft
kron-spar wants to merge 5 commits into
mainfrom
ETT-1378
Draft

kron-spar wants to merge 5 commits into
mainfrom
ETT-1378

Conversation

@kron-spar

Copy link
Copy Markdown
Contributor

The python script is designed to adjust all workflow repos and updated the Readme with directions. It takes a directory(s), a given branch name, and a string to find and a string to replace it.

python3 find_and_replace_workdlows.py --root_dir ../ --branch ETT-1378 --find "hathitrust/github_actions/build@v1" --replace "hathitrust/github_actions/build@v2"

@kron-spar
kron-spar requested review from aelkiss and liseli June 23, 2026 15:21
@github-actions

Copy link
Copy Markdown
Report Summary

┌──────────────────────────────────────────────────────────────────────────────────┬────────────┬─────────────────┬─────────┐
│                                      Target                                      │    Type    │ Vulnerabilities │ Secrets │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ ghcr.io/hathitrust/actions_playground:e215e5baa5cb6b6f1cc7feb3d05893097829839e   │   debian   │        0        │    -    │
│ (debian 13.5)                                                                    │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/certifi-2026.6.17.dist-info/METADATA      │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/charset_normalizer-3.4.7.dist-info/METAD- │ python-pkg │        0        │    -    │
│ ATA                                                                              │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/idna-3.18.dist-info/METADATA              │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/pip-26.1.2.dist-info/METADATA             │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/requests-2.32.3.dist-info/METADATA        │ python-pkg │        2        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/urllib3-2.7.0.dist-info/METADATA          │ python-pkg │        0        │    -    │
└──────────────────────────────────────────────────────────────────────────────────┴────────────┴─────────────────┴─────────┘
Legend:
- '-': Not scanned
- '0': Clean (no security findings detected)


Python (python-pkg)
===================
Total: 2 (MEDIUM: 2, HIGH: 0, CRITICAL: 0)

┌─────────────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│       Library       │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                            Title                             │
├─────────────────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ requests (METADATA) │ CVE-2024-47081 │ MEDIUM   │ fixed  │ 2.32.3            │ 2.32.4        │ requests: Requests vulnerable to .netrc credentials leak via │
│                     │                │          │        │                   │               │ malicious URLs                                               │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2024-47081                   │
│                     ├────────────────┤          │        │                   ├───────────────┼──────────────────────────────────────────────────────────────┤
│                     │ CVE-2026-25645 │          │        │                   │ 2.33.0        │ requests: Requests: Security bypass due to predictable       │
│                     │                │          │        │                   │               │ temporary file creation                                      │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2026-25645                   │
└─────────────────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘

Objecting: Update reusable workflow references in multiple git repositories like a massive find and replace
1. Traverses all git repositories under a parent directory.
2. Searches for .github/workflows/*.yml and *.yaml files.
3. Find and replace lines within the workfloew.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: workflow

Comment thread find_and_replace_workflows.py Outdated
print(f"Error processing repository {repo}: {e}")
break # Remove this break to process all repositories

reponse = input(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type: response

Comment thread find_and_replace_workflows.py Outdated
changes = []
for workflow_file in find_workflow_files(repo):
text = workflow_file.read_text(encoding="utf-8")
matches = list(re.compile(args.find).finditer(text))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can define a variable with the pattern re.compile(args.find) at the beginning of this function, instead of have it twice line 97 and line 105

parser.add_argument(
"--replace", help="String to replace in workflow references", required=True)

args = parser.parse_args()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You use args as a global object, so all your functions have access to the args object. I recommend creating variables for the relevant parameters and defining arguments in your functions. e.g. find=args.find and collect_changes(repo, find). This way, you decouple your functions from the CLI parser.

repo,
)
print(f"\nDifferences in repository {repo}:\n{diff}\n")
return True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function always returns True, but since you don't use its result (line 319), you could remove this return statement.

Comment thread find_and_replace_workflows.py Outdated
print(f"Modified files in {repo}:")
for file in modified_files:
print(f"{file}")
# commit_and_push(repo)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uncomment the commit_and_push

print("\nScanning repositories for proposed changes...")
print("--------------------------------------------------\n")

for repo in repos:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kron-spar I recommend you split the logic into the main function into different stages: 1) collect_changes, 2) apply_changes. Right now,

Step 1: collect_changes, display, and ask user - This step won't update anything
Step 2: For each repository, apply the changes, then commit & push, and if something went wrong, you need to clean up.

Something like this

repos = find_repositories(args.root_dir)

# Step 1: collect the changes
for repo in repos:
       changes = collect_changes(repo)
        if changes:
            proposed_changes[repo] = changes
            print(f"\nFound changes in {repo}")
            for change in changes:
                print(f"File: {change['file']}")
                print(f"    Before: {change['before']}")
                print(f"    After:  {change['after']}")
       
      if not proposed_changes:   
          print("\nNo proposed changes found.")                                                                                                                                      
          sys.exit(0)                                                                                                                                                  
                                                                                                                                                                       
      response = input("\nDo you want to apply these changes? (y/n): ")                                                                                                
      if response.lower() != 'y':                                                                                                                                      
          print("Aborting.")                                                                                                                                           
          sys.exit(0)

# Step 2: apply to all repos                                                                                                                                    
for repo in proposed_changes:                                                                                                                                    
          try:                                                                                                                                                         
              prepare_repository(repo)                                                                                                                                 
              modified_files = apply_changes(repo)                                                                                                                     
              commit_and_push(repo)                                                                                                                                    
          except RuntimeError as e:                                                                                                                                    
              print(f"Error processing repository {repo}: {e}")
              cleanup_repository(repo)  # With this restructure flow, you only need to apply cleanup if something went wrong 

@aelkiss aelkiss left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance the logic looks correct. I agree with Lianet's suggested changes.

Since this is general enough that it could be something we re-use in the future, it's probably worth thinking about some testing here. The challenge is that in that case we don't want to actually git commit, etc. The way to deal with that is dependency injection -- we need a class/object that we can provide that can either do the real git operations or can serve as a mock where we can verify that specific functions have been called, but without actually doing the git operations.

I'd suggest making the changes Lianet suggests, then looking at what a basic integration test might look like, and we can look at how to implement that. Let me know what you think.

@aelkiss

aelkiss commented Jun 26, 2026

Copy link
Copy Markdown
Member

After discussing with @kron-spar I think we can wait on automated tests for now since we expect this to be a one-time thing. If we do end up needing to use this again, then I think it's worth thinking more about automated tests at that point.

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Report Summary

┌──────────────────────────────────────────────────────────────────────────────────┬────────────┬─────────────────┬─────────┐
│                                      Target                                      │    Type    │ Vulnerabilities │ Secrets │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ ghcr.io/hathitrust/actions_playground:67c9ce871e0b172a37eaa43c19ef4a4da84aecee   │   debian   │        4        │    -    │
│ (debian 13.5)                                                                    │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/certifi-2026.6.17.dist-info/METADATA      │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/charset_normalizer-3.4.7.dist-info/METAD- │ python-pkg │        0        │    -    │
│ ATA                                                                              │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/idna-3.18.dist-info/METADATA              │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/pip-26.1.2.dist-info/METADATA             │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/requests-2.32.3.dist-info/METADATA        │ python-pkg │        2        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/urllib3-2.7.0.dist-info/METADATA          │ python-pkg │        0        │    -    │
└──────────────────────────────────────────────────────────────────────────────────┴────────────┴─────────────────┴─────────┘
Legend:
- '-': Not scanned
- '0': Clean (no security findings detected)


ghcr.io/hathitrust/actions_playground:67c9ce871e0b172a37eaa43c19ef4a4da84aecee (debian 13.5)
============================================================================================
Total: 4 (MEDIUM: 1, HIGH: 3, CRITICAL: 0)

┌──────────────┬────────────────┬──────────┬────────┬───────────────────┬──────────────────┬──────────────────────────────────────────────────────────┐
│   Library    │ Vulnerability  │ Severity │ Status │ Installed Version │  Fixed Version   │                          Title                           │
├──────────────┼────────────────┼──────────┼────────┼───────────────────┼──────────────────┼──────────────────────────────────────────────────────────┤
│ libssh2-1t64 │ CVE-2026-55199 │ HIGH     │ fixed  │ 1.11.1-1          │ 1.11.1-1+deb13u1 │ libssh2: libssh2: Denial of Service via crafted          │
│              │                │          │        │                   │                  │ SSH_MSG_EXT_INFO message                                 │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-55199               │
│              ├────────────────┤          │        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2026-55200 │          │        │                   │                  │ libssh2: libssh2 - Out-of-Bounds Write via Unchecked     │
│              │                │          │        │                   │                  │ packet_length in transport.c                             │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-55200               │
│              ├────────────────┤          │        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2026-7598  │          │        │                   │                  │ libssh2: integer overflow via large username or password │
│              │                │          │        │                   │                  │ arguments                                                │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-7598                │
│              ├────────────────┼──────────┤        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2025-15661 │ MEDIUM   │        │                   │                  │ libssh2: libssh2: Information disclosure and denial of   │
│              │                │          │        │                   │                  │ service via crafted SFTP response...                     │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2025-15661               │
└──────────────┴────────────────┴──────────┴────────┴───────────────────┴──────────────────┴──────────────────────────────────────────────────────────┘

Python (python-pkg)
===================
Total: 2 (MEDIUM: 2, HIGH: 0, CRITICAL: 0)

┌─────────────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│       Library       │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                            Title                             │
├─────────────────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ requests (METADATA) │ CVE-2024-47081 │ MEDIUM   │ fixed  │ 2.32.3            │ 2.32.4        │ requests: Requests vulnerable to .netrc credentials leak via │
│                     │                │          │        │                   │               │ malicious URLs                                               │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2024-47081                   │
│                     ├────────────────┤          │        │                   ├───────────────┼──────────────────────────────────────────────────────────────┤
│                     │ CVE-2026-25645 │          │        │                   │ 2.33.0        │ requests: Requests: Security bypass due to predictable       │
│                     │                │          │        │                   │               │ temporary file creation                                      │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2026-25645                   │
└─────────────────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Report Summary

┌──────────────────────────────────────────────────────────────────────────────────┬────────────┬─────────────────┬─────────┐
│                                      Target                                      │    Type    │ Vulnerabilities │ Secrets │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ ghcr.io/hathitrust/actions_playground:951ad7336f71823c1471ecf91f8e453b1d2774fe   │   debian   │        4        │    -    │
│ (debian 13.5)                                                                    │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/certifi-2026.6.17.dist-info/METADATA      │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/charset_normalizer-3.4.7.dist-info/METAD- │ python-pkg │        0        │    -    │
│ ATA                                                                              │            │                 │         │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/idna-3.18.dist-info/METADATA              │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/pip-26.1.2.dist-info/METADATA             │ python-pkg │        0        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/requests-2.32.3.dist-info/METADATA        │ python-pkg │        2        │    -    │
├──────────────────────────────────────────────────────────────────────────────────┼────────────┼─────────────────┼─────────┤
│ usr/local/lib/python3.14/site-packages/urllib3-2.7.0.dist-info/METADATA          │ python-pkg │        0        │    -    │
└──────────────────────────────────────────────────────────────────────────────────┴────────────┴─────────────────┴─────────┘
Legend:
- '-': Not scanned
- '0': Clean (no security findings detected)


ghcr.io/hathitrust/actions_playground:951ad7336f71823c1471ecf91f8e453b1d2774fe (debian 13.5)
============================================================================================
Total: 4 (MEDIUM: 1, HIGH: 3, CRITICAL: 0)

┌──────────────┬────────────────┬──────────┬────────┬───────────────────┬──────────────────┬──────────────────────────────────────────────────────────┐
│   Library    │ Vulnerability  │ Severity │ Status │ Installed Version │  Fixed Version   │                          Title                           │
├──────────────┼────────────────┼──────────┼────────┼───────────────────┼──────────────────┼──────────────────────────────────────────────────────────┤
│ libssh2-1t64 │ CVE-2026-55199 │ HIGH     │ fixed  │ 1.11.1-1          │ 1.11.1-1+deb13u1 │ libssh2: libssh2: Denial of Service via crafted          │
│              │                │          │        │                   │                  │ SSH_MSG_EXT_INFO message                                 │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-55199               │
│              ├────────────────┤          │        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2026-55200 │          │        │                   │                  │ libssh2: libssh2 - Out-of-Bounds Write via Unchecked     │
│              │                │          │        │                   │                  │ packet_length in transport.c                             │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-55200               │
│              ├────────────────┤          │        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2026-7598  │          │        │                   │                  │ libssh2: integer overflow via large username or password │
│              │                │          │        │                   │                  │ arguments                                                │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2026-7598                │
│              ├────────────────┼──────────┤        │                   │                  ├──────────────────────────────────────────────────────────┤
│              │ CVE-2025-15661 │ MEDIUM   │        │                   │                  │ libssh2: libssh2: Information disclosure and denial of   │
│              │                │          │        │                   │                  │ service via crafted SFTP response...                     │
│              │                │          │        │                   │                  │ https://avd.aquasec.com/nvd/cve-2025-15661               │
└──────────────┴────────────────┴──────────┴────────┴───────────────────┴──────────────────┴──────────────────────────────────────────────────────────┘

Python (python-pkg)
===================
Total: 2 (MEDIUM: 2, HIGH: 0, CRITICAL: 0)

┌─────────────────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬──────────────────────────────────────────────────────────────┐
│       Library       │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                            Title                             │
├─────────────────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼──────────────────────────────────────────────────────────────┤
│ requests (METADATA) │ CVE-2024-47081 │ MEDIUM   │ fixed  │ 2.32.3            │ 2.32.4        │ requests: Requests vulnerable to .netrc credentials leak via │
│                     │                │          │        │                   │               │ malicious URLs                                               │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2024-47081                   │
│                     ├────────────────┤          │        │                   ├───────────────┼──────────────────────────────────────────────────────────────┤
│                     │ CVE-2026-25645 │          │        │                   │ 2.33.0        │ requests: Requests: Security bypass due to predictable       │
│                     │                │          │        │                   │               │ temporary file creation                                      │
│                     │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2026-25645                   │
└─────────────────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴──────────────────────────────────────────────────────────────┘

@kron-spar
kron-spar marked this pull request as draft July 15, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants