Affiliation Analysis Limited to Email Domains Only

[Ipswitch]

Problem Description

The affiliation analysis plugin currently only checks contributor email domains against organization patterns, which leads to significant false negatives when contributors use personal email addresses for commits while working for flagged organizations. The current affiliation analysis in plugins/affiliation/src/main.rs only examines the email domain from git commit metadata:

if affiliator.is_match(contributor.email.as_str()) {
    // Flag as affiliated
}

This approach has several limitations:

1. Personal Email Usage: Many contributors working for organizations use personal emails (@gmail.com, @outlook.com, etc.) in their git commits
2. No Identity Resolution: There's no connection between git commit authors and their actual GitHub usernames/profiles
3. Inconsistent Identity: The same person might use different emails across commits
4. Missed Affiliations: Company employees using personal emails won't be detected as affiliated

Real-World Example: Hipcheck Repository Analysis

Test Command Used

docker run --rm \
  -e HC_GITHUB_TOKEN=$HC_GITHUB_TOKEN \
  mitre/hipcheck:latest \
  check https://github.com/mitre/hipcheck -t repo -f json 2>/dev/null | \
  awk '/^{/{f=1} f' > hc-output.json

Actual Results (Hipcheck v3.3.1)

When running Hipcheck on its own repository, the affiliation analysis shows 815 flagged contributions but provides minimal context:

{
  "analysis": "Affiliation",
  "value": 815,
  "threshold": 0,
  "concerns": [
    {
      "contributor": "Cal Stepanian",
      "count": 1
    },
    {
      "contributor": "Andrew Lilley Brinker", 
      "count": 1
    },
    {
      "contributor": "Michael Chernicoff",
      "count": 1
    },
    {
      "contributor": "j-lanson",
      "count": 1
    },
    {
      "contributor": "sei-shissam",
      "count": 1
    },
    {
      "contributor": "GitHub",
      "count": 1
    },
    {
      "contributor": "dependabot[bot]",
      "count": 1
    }
  ]
}

Problem Analysis

The output shows contributor names only - no email domains, GitHub usernames, or organizational context. This reveals several issues:

1. Missing Email Context: We can't see what email domains triggered the flags
2. No GitHub Integration: Names like "j-lanson" could be GitHub usernames, but no connection is made
3. Identity Ambiguity: "GitHub" and "dependabot[bot]" suggest some automation, but no clear resolution
4. Limited Actionability: Users can't understand why these contributors were flagged

Key Questions This Raises

• Are these contributors flagged because of email domains like @mitre.org?
• Do any use personal emails (@gmail.com) while working for concerning organizations?
• What are their actual GitHub profiles and organizational affiliations?
• Are we missing contributors who should be flagged but use personal emails?

Generic Example Scenario

A Google employee commits using john.doe@gmail.com instead of john.doe@google.com. The current system would:

• ❌ Not flag this contributor as Google-affiliated
• ❌ Miss a potentially concerning affiliation
• ✅ Would flag if they used john.doe@google.com

Comprehensive GitHub Username Resolution Analysis

Complete Success Rate Findings

Analysis of the complete Hipcheck repository commit history using GitHub GraphQL API reveals:

• 📊 850 total commit instances analyzed (complete repository history)
• ✅ 100% of commits have GitHub user accounts linked
• 👥 22 unique contributors mapped to 16 unique GitHub usernames
• 🎯 Perfect resolution - every commit author has an available GitHub username

GitHub Usernames by Commit Count

Analysis shows that every contributor has a resolvable GitHub username, proving the GitHub GraphQL API provides complete identity resolution:

🐙 GitHub Usernames by Commit Count:
   1. @dependabot[bot]: 225 commits
      Primary: dependabot[bot] (49699333+dependabot[bot]@users.noreply.github.com)
   2. @alilleybrinker: 217 commits
      Primary: Andrew Lilley Brinker (abrinker@mitre.org) (2 emails)
      All emails: abrinker@mitre.org, alilleybrinker@gmail.com
   3. @j-lanson: 203 commits
      Primary: jlanson (110619939+j-lanson@users.noreply.github.com) (2 emails, 3 names)
      All emails: 110619939+j-lanson@users.noreply.github.com, jlanson@mitre.org
      All names: Julian Lanson, j-lanson, jlanson
   4. @mchernicoff: 80 commits
      Primary: Michael Chernicoff (76788795+mchernicoff@users.noreply.github.com) (3 emails)
      All emails: 76788795+mchernicoff@users.noreply.github.com, mchernicoff@mitre.org, mjchernicoff@gmail.com
   5. @patrickjcasey: 48 commits
      Primary: Patrick Casey (patrick.casey1@outlook.com)
   6. @cstepanian: 21 commits
      Primary: Cal Stepanian (61707847+cstepanian@users.noreply.github.com)
   7. @vcfxb: 19 commits
      Primary: Venus Xeon-Blonde (alfriadox@gmail.com) (2 emails)
      All emails: alfriadox@gmail.com, venus@mitre.org
   8. @KirilldogU: 9 commits
      Primary: Kirill Usubyan (kusubyan@mitre.org)
   9. @aamohd: 8 commits
      Primary: Aisha M (amohammed@mitre.org)
  10. @ninaagrawal: 6 commits
      Primary: Nina 'Nino' Agrawal (ninaagrawal@mitre.org)
  11. @bmarr-mitre: 6 commits
      Primary: BMARR (bmarr@mitre.org)
  12. @devin-b-lake: 4 commits
      Primary: Devin Lake (dlake@mitre.org)
  13. @operagxsasha: 1 commit
      Primary: operagxsasha (operagxsasha@gmail.com)
  14. @sei-shissam: 1 commit
      Primary: sei-shissam (shissam@sei.cmu.edu)
  15. @Mariazeigler: 1 commit
      Primary: mzeigler (mzeigler@mitre.org)
  16. @ashleygwilliams: 1 commit
      Primary: Ashley Williams (ashley666ashley@gmail.com)

Key Insights from GitHub Resolution Data

1. Multiple Identity Management: Contributors like @alilleybrinker, @j-lanson, @mchernicoff, and @vcfxb use multiple email addresses but map to the same GitHub account
2. Personal Email Success: Regular emails like operagxsasha@gmail.com, ashley666ashley@gmail.com, and patrick.casey1@outlook.com all successfully resolve to GitHub usernames
3. Complete Coverage: Even single-commit contributors have full GitHub identity resolution
4. Organizational Context: Many contributors use both organizational emails (@mitre.org) and personal emails, but the GitHub username provides the consistent identity

Current System Limitations

This analysis proves that the GitHub GraphQL API provides perfect username resolution for all contributors, but the current Rust implementation fails to utilize this data.

Current Behavior

The git plugin extracts contributor data containing only:

pub struct Contributor {
    pub name: String,    // From git commit author name
    pub email: String,   // From git commit author email
}

Expected/Desired Behavior

The affiliation analysis should be enhanced to:

1. Resolve git commit authors to GitHub usernames when possible
2. Check GitHub profile information for organization affiliations
3. Fall back to email domain matching when GitHub resolution isn't available
4. Provide more comprehensive identity resolution

Impact

This limitation could result in:

• Security blind spots: Missing contributors with concerning organizational affiliations
• Incomplete risk assessment: Underestimating the influence of specific organizations
• False confidence: Believing a repository has no concerning affiliations when it actually does
• Dogfooding Issues: Even Hipcheck's own repository analysis may have false negatives due to this limitation

Reproducing the Issue

1. Run the test command above on any repository (including hipcheck itself)
2. Observe that contributor output shows only names, no email domains or GitHub context
3. Note the high count (815 in hipcheck's case) with minimal explanation
4. Manually check GitHub profiles of flagged contributors to understand actual affiliations
5. Compare git commit emails vs GitHub profiles to identify potential false negatives

Version Information

• Affected Version: Hipcheck 3.3.1 (latest Docker container as of July 2025)
• Container: mitre/hipcheck:latest
• Analysis Date: 2025-07-27

Proposed Solution

Enhance the git plugin to:

1. Add optional GitHub username resolution for contributors
2. Query GitHub API for user/organization information
3. Provide GitHub-enhanced contributor data to the affiliation plugin
4. Maintain backward compatibility with existing email-based analysis

Required GraphQL Query

query CommitAuthors($owner: String!, $repo: String!, $cursor: String) {
    repository(owner: $owner, name: $repo) {
        defaultBranchRef {
            target {
                ... on Commit {
                    history(first: 100, after: $cursor) {
                        pageInfo {
                            hasNextPage
                            endCursor
                        }
                        nodes {
                            oid
                            author {
                                name
                                email
                                user {
                                    login
                                    name
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Proposed Enhanced Identity Resolution

The solution would provide:

1. GitHub Username Resolution: When available, show GitHub usernames like 61707847+cstepanian@users.noreply.github.com
2. Fallback to Email: When GitHub resolution fails, display email addresses
3. Comprehensive Display: Format Name (Email/Username) for better identity tracking
4. Preserved Functionality: Maintain all existing affiliation checking logic

Expected Benefits

1. Enhanced Visibility: Users would see both names and identifying information (email/username)
2. Better Identity Tracking: GitHub usernames would provide clearer identity resolution than names alone
3. Maintained Compatibility: All existing functionality would be preserved while adding new capabilities
4. Comprehensive Analysis: Both traditional email-based and GitHub-based identity information would be available

[alilleybrinker]

Hey @Ipswitch, thanks for opening this. Yes, you're right that the affiliation analysis is fairly limited because it only considers email hosts. This is documented on the plugin's docs page. We'd love to see improvements to it, and if you're interested in contributing for that I'd be happy to mentor.

As you identified, the likely biggest initial wins would be from integrating additional data sources like GitHub.

Integrating commit signing is also a possibility, though it brings in a host of extra complexity for how to 1) get the key material to validate against, 2) how to handle revocation, 3) how to handle integrating with whatever verification tools are used, and more.

Happy to discuss further!