diff --git a/.github/labeler.yml b/.github/labeler.yml index 61d43d1de..d32ba528b 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -28,6 +28,7 @@ contract:change-request: - 'tools/bitbucket/**' - 'tools/change-request/**' - 'tools/github/**' + - 'tools/gitlab/**' - 'tools/jira-patch/**' - 'tools/mail-patch/**' @@ -95,6 +96,7 @@ contract:source-control: - 'tools/asf-svn/**' - 'tools/fossil/**' - 'tools/github/**' + - 'tools/gitlab/**' - 'tools/sourcehut/**' - 'tools/vcs/**' @@ -107,6 +109,7 @@ contract:tracker: - 'tools/github/**' - 'tools/github-body-field/**' - 'tools/github-rollup/**' + - 'tools/gitlab/**' - 'tools/jira/**' - 'tools/sourcehut/**' diff --git a/docs/adapters/registry.md b/docs/adapters/registry.md index d1fe1e6ec..beb644f8a 100644 --- a/docs/adapters/registry.md +++ b/docs/adapters/registry.md @@ -54,7 +54,7 @@ extension point = a documented, labelled slot with a tracking issue. | [`tools/forwarder-relay`](../../tools/forwarder-relay/) | ASF-security ([`tools/gmail/asf-relay.md`](../../tools/gmail/asf-relay.md)) | huntr.com, HackerOne, GHSA relay | | [`tools/scan-format`](../../tools/scan-format/) | ASVS | other scanner formats | | [`tools/vcs`](../../tools/vcs/) | Git, Mercurial, Fossil | Subversion [\#602](https://github.com/apache/magpie/issues/602), Jujutsu [\#603](https://github.com/apache/magpie/issues/603), Perforce [\#605](https://github.com/apache/magpie/issues/605) | -| Forge / tracker | [`github`](../../tools/github/), [`jira`](../../tools/jira/), [`bitbucket`](../../tools/bitbucket/) `partial-read-only` foundation, [`sourcehut`](../../tools/sourcehut/), [`fossil`](../../tools/fossil/) | GitLab [\#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [\#310](https://github.com/apache/magpie/issues/310), Pagure [\#312](https://github.com/apache/magpie/issues/312), deeper Bitbucket/Jira coverage [\#606](https://github.com/apache/magpie/issues/606), Bugzilla [\#302](https://github.com/apache/magpie/issues/302) | +| Forge / tracker | [`github`](../../tools/github/), [`jira`](../../tools/jira/), [`bitbucket`](../../tools/bitbucket/) `partial-read-only` foundation, [`sourcehut`](../../tools/sourcehut/), [`fossil`](../../tools/fossil/), [`gitlab`](../../tools/gitlab/) `partial-read-only` foundation | Forgejo/Gitea [\#310](https://github.com/apache/magpie/issues/310), Pagure [\#312](https://github.com/apache/magpie/issues/312), deeper Bitbucket/Jira coverage [\#606](https://github.com/apache/magpie/issues/606), GitLab [\#305](https://github.com/apache/magpie/issues/305), Bugzilla [\#302](https://github.com/apache/magpie/issues/302) | | Agent harness | Claude Code, [Codex](codex.md) `experimental` ([#313](https://github.com/apache/magpie/issues/313)), [Gemini CLI](gemini.md) `experimental` ([#314](https://github.com/apache/magpie/issues/314)), [Local LLM (Ollama / llama.cpp / vLLM)](local-llm.md) ([#315](https://github.com/apache/magpie/issues/315)), [Cursor](cursor.md) ([#316](https://github.com/apache/magpie/issues/316)), [Goose](goose.md) `guide only` ([#319](https://github.com/apache/magpie/issues/319)), [Aider](aider.md) `guide only` ([#317](https://github.com/apache/magpie/issues/317)), [GitHub Copilot](copilot.md) `guide only` ([#318](https://github.com/apache/magpie/issues/318)) | Amazon Q [#320](https://github.com/apache/magpie/issues/320)–OpenHands [#322](https://github.com/apache/magpie/issues/322) | | Security cross-ref | [`tools/osv`](../../tools/osv/) | — | diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md index 61a8e5ff8..0d42f7224 100644 --- a/docs/labels-and-capabilities.md +++ b/docs/labels-and-capabilities.md @@ -323,6 +323,7 @@ or a contract-free mix of substrates (e.g. `tools/spec-inventory` is | [`tools/bitbucket`](../tools/bitbucket/) | `contract:change-request` + `contract:tracker` | Coverage: `partial`. Bitbucket Cloud and Bitbucket Data Center bridge foundation for repository metadata context, branch restriction context for PR-management decisions, pull-request discovery/fetching, read-only commit fetching, read-only diff fetching, comments-only discussion fetching, read-only review-state fetching, Cloud-only pull-request task listing/fetching, read-only merge-check context fetching, and read-only status fetching, plus narrowly scoped Cloud pull-request comment creation and approve/unapprove actions. Tracker coverage includes Cloud-only issue listing/fetching, issue comment fetching, issue attachment metadata fetching, and confirmed issue-comment creation. The `partial` qualifier means this tool implements named contract operations but does not satisfy the complete contract and must not be counted as a complete/selectable backend. Broader pull-request review/mutation, broader issue writes, and linked Jira handoff coverage remain incomplete. | | [`tools/fossil`](../tools/fossil/) | `contract:tracker` + `contract:source-control` | Fossil SCM forge bridge: integrates local SQLite-backed ticket tracking, wiki, and forum reads with the version-control shim | | [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` + `contract:change-request` | GitHub REST / GraphQL tracker substrate (called by every lifecycle phase) plus the Git source-control binding documented in [`source-control.md`](../tools/github/source-control.md) (runnable backend in [`tools/vcs`](../tools/vcs/)) and the pull-request review/merge gate (`change-request`; the ASF default backend, alongside `tools/jira-patch/` and `tools/mail-patch/` for SVN-first projects) | +| [`tools/gitlab`](../tools/gitlab/) | `contract:tracker` + `contract:source-control` + `contract:change-request` | Coverage: `partial`. GitLab REST API v4 forge bridge foundation for repository metadata context under `contract:source-control`, issue listing/fetching under `contract:tracker`, and merge request discovery, diffs, commits, and CI pipeline status under `contract:change-request`. The `partial` qualifier means this tool implements named contract operations but does not satisfy the complete contract and must not be counted as a complete/selectable backend. Write operations, issue mutation, and merge request mutations remain out of scope for this foundation. | | [`tools/github-body-field`](../tools/github-body-field/) | `contract:tracker` | Read or rewrite one `### Field` section of a GitHub issue body without bringing the body into agent context — substrate helper for the security-sync skills | | [`tools/github-rollup`](../tools/github-rollup/) | `contract:tracker` | Append to (or create) the status-rollup comment on a GitHub issue without bringing the rollup body into agent context — substrate helper for every status-update-emitting skill | | [`tools/gmail`](../tools/gmail/) | `contract:mail-source` + `contract:mail-create` + `contract:mail-archive` | Gmail API substrate — inbound report intake (`mail-source`), thread / archive reads (`mail-archive`), plus outbound courtesy-reply drafting (`mail-create`); read + draft only, never sends | diff --git a/docs/vendor-neutrality.md b/docs/vendor-neutrality.md index 18a8a611c..1e142a97c 100644 --- a/docs/vendor-neutrality.md +++ b/docs/vendor-neutrality.md @@ -358,7 +358,9 @@ declare it under *Tools enabled*; no skill changes). The forge/tracker extension points are open, labelled `good first issue`, not hypothetical: -[GitLab](https://github.com/apache/magpie/issues/305), +[GitLab](https://github.com/apache/magpie/issues/305) (initial +[`tools/gitlab`](../tools/gitlab/) `partial-read-only` bridge; +full write and mutation coverage tracked there), [Codeberg / Gitea / Forgejo](https://github.com/apache/magpie/issues/310), [Pagure](https://github.com/apache/magpie/issues/312) (Fedora / `pagure.io`), @@ -503,7 +505,7 @@ coverage without pretending one team can implement an open-ended set. |---|---|---|---| | LLM backend | ✅ by construction | Claude Code, Ollama, vLLM, Apache-hosted, Bedrock, direct Anthropic | Any endpoint meeting the capability floor + privacy gate | | Agentic harness | ✅ by construction (`AGENTS.md` standard) | Claude Code; OpenCode; [Codex adapter](adapters/codex.md) (experimental); [Gemini adapter](adapters/gemini.md) (experimental); community use under Cursor, Copilot, Kiro | Remaining runtime adapters [#314–#322](https://github.com/apache/magpie/issues?q=is%3Aissue+state%3Aopen+adapter+in%3Atitle) | -| Forge / tracker | ✅ by construction | GitHub, Jira, SourceHut; Bitbucket `partial-read-only` foundation excluded from complete-backend counts; CVE/scan/relay via adapter contracts | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), full Bitbucket tracker/change-request/Jira coverage [#606](https://github.com/apache/magpie/issues/606), Bugzilla [#302](https://github.com/apache/magpie/issues/302) | +| Forge / tracker | ✅ by construction | GitHub, Jira, SourceHut; Bitbucket and GitLab `partial-read-only` foundations excluded from complete-backend counts; CVE/scan/relay via adapter contracts | Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), full Bitbucket tracker/change-request/Jira coverage [#606](https://github.com/apache/magpie/issues/606), GitLab [#305](https://github.com/apache/magpie/issues/305), Bugzilla [#302](https://github.com/apache/magpie/issues/302) | | Communication channels | ✅ by construction | PonyMail / mail-archive reads | mbox [#304](https://github.com/apache/magpie/issues/304), IMAP [#303](https://github.com/apache/magpie/issues/303), Mailman 3 [#306](https://github.com/apache/magpie/issues/306); Discourse [#307](https://github.com/apache/magpie/issues/307), Zulip [#308](https://github.com/apache/magpie/issues/308), Matrix [#309](https://github.com/apache/magpie/issues/309) | | Source control (VCS) | ✅ by construction | **Git (complete)**, **Mercurial (complete)**; ASF SVN surface ([`tools/asf-svn`](../tools/asf-svn/): source control + dist.apache.org + authorization) | Subversion generic VCS binding [\#602](https://github.com/apache/magpie/issues/602) (detected); Jujutsu [\#603](https://github.com/apache/magpie/issues/603), Fossil [\#604](https://github.com/apache/magpie/issues/604), Perforce [\#605](https://github.com/apache/magpie/issues/605) (tracked) | | Project governance | ✅ by construction | ASF + non-ASF adopter profiles | Adopter config (modes, thresholds) | @@ -568,9 +570,9 @@ generated block below. | Capability contract | Neutral? | Class | Backends today | Basis | |---|---|---|---|---| -| `contract:tracker` | ✅ | vendor-backed | Atlassian, Fossil, GitHub, SourceHut | 4 backend vendors: Atlassian, Fossil, GitHub, SourceHut; partial foundation, not counted: bitbucket | -| `contract:source-control` | ✅ | vendor-backed | Fossil, Git, GitHub, SourceHut, Subversion | 5 backend vendors: Fossil, Git, GitHub, SourceHut, Subversion | -| `contract:change-request` | ✅ | vendor-backed | Atlassian, GitHub, email | 3 backend vendors: Atlassian, GitHub, email; partial foundation, not counted: bitbucket | +| `contract:tracker` | ✅ | vendor-backed | Atlassian, Fossil, GitHub, SourceHut | 4 backend vendors: Atlassian, Fossil, GitHub, SourceHut; partial foundation, not counted: bitbucket, gitlab | +| `contract:source-control` | ✅ | vendor-backed | Fossil, Git, GitHub, SourceHut, Subversion | 5 backend vendors: Fossil, Git, GitHub, SourceHut, Subversion; partial foundation, not counted: gitlab | +| `contract:change-request` | ✅ | vendor-backed | Atlassian, GitHub, email | 3 backend vendors: Atlassian, GitHub, email; partial foundation, not counted: bitbucket, gitlab | | `contract:mail-archive` | ✅ | vendor-backed | ASF, Google, SourceHut | 3 backend vendors: ASF, Google, SourceHut | | `contract:mail-source` | ✅ | vendor-backed | ASF, Google, Maildir | 3 backend vendors: ASF, Google, Maildir | | `contract:mail-create` | ✅ | vendor-backed | Google, Maildir | 2 backend vendors: Google, Maildir | diff --git a/pyproject.toml b/pyproject.toml index 7a24df598..647c80eab 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -151,4 +151,5 @@ members = [ "tools/vetted-ops", "tools/fossil", "tools/sourcehut", + "tools/gitlab", ] diff --git a/tools/dev/tests/test_check_duplication.py b/tools/dev/tests/test_check_duplication.py index ac318214e..1531ed38c 100644 --- a/tools/dev/tests/test_check_duplication.py +++ b/tools/dev/tests/test_check_duplication.py @@ -104,11 +104,25 @@ def test_identical_text_inside_generated_regions_is_invisible() -> None: `strip_generated_regions` exists. Reuses a real propagated block from the live tree so the test tracks the real marker text, not a hand-written stand-in.""" - live_skill = next((REPO / "skills").glob("*/SKILL.md")) - text = live_skill.read_text() - match = MOD.PREFLIGHT_RE.search(text) - assert match, "expected the live skill to carry the auto pre-flight block" - block = match.group(0) + block = None + for pattern in ( + "skills/*/SKILL.md", + "plugins/magpie-*/skills/*/SKILL.md", + "plugins/magpie-*/skills/*/*.md", + ): + for path in REPO.glob(pattern): + try: + match = MOD.PREFLIGHT_RE.search(path.read_text(encoding="utf-8")) + if match: + block = match.group(0) + break + except OSError: + continue + if block: + break + if not block: + preflight_content = (REPO / "tools" / "dev" / "preflight-block.md").read_text(encoding="utf-8") + block = f"{MOD.PREFLIGHT_BEGIN}\n{preflight_content}\n{MOD.PREFLIGHT_END}\n" paragraphs = MOD.extract_paragraphs( Path("virtual.md"), text=f"# Heading\n\n{block}\n## Next\n\nSome unrelated text.\n" diff --git a/tools/gitlab/README.md b/tools/gitlab/README.md new file mode 100644 index 000000000..36636f4ac --- /dev/null +++ b/tools/gitlab/README.md @@ -0,0 +1,84 @@ + + + + +**Table of Contents** + +- [GitLab bridge](#gitlab-bridge) + - [Prerequisites](#prerequisites) + - [Configuration](#configuration) + - [Operations](#operations) + - [Usage](#usage) + + + +# GitLab bridge + +**Capability:** contract:tracker + contract:source-control + contract:change-request + +**Coverage:** `partial` + +**Kind:** implementation + +**Vendor:** GitLab + +Read-only client for the GitLab REST API v4. + +This bridge implements a `partial` read-only foundation for repository +metadata context under `contract:source-control`, issue listing and fetching +under `contract:tracker`, and merge request discovery, diffs, commits, and +CI pipeline status under `contract:change-request`. +Partial adapters may implement named contract verbs, but they do not satisfy +the complete contract and must not be advertised as complete/selectable backends. +Write operations and issue/MR mutations remain out of scope for this foundation. + +## Prerequisites + +- **Runtime:** Python 3.11+ via `uv`. +- **CLIs:** `uv`. +- **Credentials / auth:** `GITLAB_TOKEN` (Personal Access Token, OAuth Bearer token) + or `CI_JOB_TOKEN` with API access. Tokens are optional for unauthenticated reads + on public projects. +- **Auth scheme override:** `GITLAB_AUTH_SCHEME` (`PrivateToken`, `Bearer`, `JobToken`) + can be set to override header selection explicitly. +- **Network:** Access to the configured GitLab instance; `GITLAB_INSTANCE_URL` + defaults to `https://gitlab.com`. + +## Configuration + +Set `GITLAB_TOKEN` in your environment: + +```bash +export GITLAB_TOKEN="glpat-..." +``` + +For self-hosted instances (e.g. Debian Salsa, GNOME): + +```bash +export GITLAB_INSTANCE_URL="https://salsa.debian.org" +``` + +To explicitly force an authentication scheme (e.g. OAuth Bearer token vs Private Token): + +```bash +export GITLAB_AUTH_SCHEME="Bearer" +``` + +## Operations + +See [tool.md](tool.md) for the full operations catalogue and contract mapping. + +## Usage + +List open issues for a project: + +```bash +uv run --project tools/gitlab magpie-gitlab issue list +``` + +Get a merge request diff: + +```bash +uv run --project tools/gitlab magpie-gitlab mr diff +``` diff --git a/tools/gitlab/issue-template.md b/tools/gitlab/issue-template.md new file mode 100644 index 000000000..51e83c7a5 --- /dev/null +++ b/tools/gitlab/issue-template.md @@ -0,0 +1,25 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [GitLab Issue Template](#gitlab-issue-template) + - [Markdown Guidelines](#markdown-guidelines) + + + + + +# GitLab Issue Template + +Schema for GitLab issues and merge request bodies. + +## Markdown Guidelines + +GitLab Flavored Markdown (GLFM) is used across all issue and MR descriptions: +- **Checkboxes**: Formatted as `- [ ]` and `- [x]`. +- **References**: Issues are referenced with `#ID` and MRs with `!ID`. +- **Labels**: Scoped labels often use `::` (e.g., `workflow::in-review`). + +Ensure that the adapter correctly parses these specific constructs +when reading issue bodies. diff --git a/tools/gitlab/pyproject.toml b/tools/gitlab/pyproject.toml new file mode 100644 index 000000000..1ad59a9c5 --- /dev/null +++ b/tools/gitlab/pyproject.toml @@ -0,0 +1,71 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "magpie-gitlab" +version = "0.1.0" +description = "GitLab forge, issue tracker, and merge request bridge for Apache Magpie." +readme = "README.md" +requires-python = ">=3.11" +license = { text = "Apache-2.0" } +dependencies = [] + +[project.scripts] +magpie-gitlab = "magpie_gitlab.cli:main" + +[tool.hatch.build.targets.wheel] +packages = ["src/magpie_gitlab"] + +[tool.ruff] +line-length = 110 +target-version = "py311" +src = ["src", "tests"] + +[tool.ruff.lint] +select = ["E", "W", "F", "I", "B", "UP", "SIM", "C4", "RUF"] +ignore = ["E501"] + +[tool.ruff.lint.per-file-ignores] +"tests/**" = ["B", "SIM"] + +[tool.mypy] +python_version = "3.11" +files = ["src", "tests"] +warn_unused_ignores = true +warn_redundant_casts = true +warn_unreachable = true +check_untyped_defs = true +no_implicit_optional = true +disallow_untyped_defs = true +disallow_incomplete_defs = true + +[[tool.mypy.overrides]] +module = "tests.*" +disallow_untyped_defs = false +disallow_incomplete_defs = false + +[tool.pytest.ini_options] +minversion = "8.0" +addopts = "-ra -q" +testpaths = ["tests"] + +[dependency-groups] +dev = ["magpie-dev"] diff --git a/tools/gitlab/src/magpie_gitlab/__init__.py b/tools/gitlab/src/magpie_gitlab/__init__.py new file mode 100644 index 000000000..c8eac1454 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/__init__.py @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""GitLab forge, issue tracker, and merge request bridge.""" + +from __future__ import annotations + +from .cli import main + +__all__ = ["main"] diff --git a/tools/gitlab/src/magpie_gitlab/cli.py b/tools/gitlab/src/magpie_gitlab/cli.py new file mode 100644 index 000000000..ad0f00369 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/cli.py @@ -0,0 +1,127 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import argparse +import json +import sys + +from .client import get_project, load_config +from .issues import get_issue, list_issues +from .merge_requests import get_mr, get_mr_commits, get_mr_diff, list_mrs +from .pipelines import get_pipeline_status, list_mr_pipelines + + +def main() -> int: + parser = argparse.ArgumentParser(description="GitLab CLI for Magpie") + subparsers = parser.add_subparsers(dest="command", required=True) + + # repo + repo_p = subparsers.add_parser("repo") + repo_subs = repo_p.add_subparsers(dest="action", required=True) + repo_get = repo_subs.add_parser("get") + repo_get.add_argument("project") + + # issue + issue_p = subparsers.add_parser("issue") + issue_subs = issue_p.add_subparsers(dest="action", required=True) + issue_list = issue_subs.add_parser("list") + issue_list.add_argument("project") + issue_list.add_argument( + "--state", + choices=["opened", "closed", "all"], + default="opened", + ) + issue_list.add_argument("--limit", type=int, default=None) + issue_get = issue_subs.add_parser("get") + issue_get.add_argument("project") + issue_get.add_argument("issue_iid", type=int) + + # mr + mr_p = subparsers.add_parser("mr") + mr_subs = mr_p.add_subparsers(dest="action", required=True) + mr_list = mr_subs.add_parser("list") + mr_list.add_argument("project") + mr_list.add_argument( + "--state", + choices=["opened", "closed", "locked", "merged", "all"], + default="opened", + ) + mr_list.add_argument("--limit", type=int, default=None) + mr_get = mr_subs.add_parser("get") + mr_get.add_argument("project") + mr_get.add_argument("mr_iid", type=int) + mr_diff = mr_subs.add_parser("diff") + mr_diff.add_argument("project") + mr_diff.add_argument("mr_iid", type=int) + mr_diff.add_argument("--limit", type=int, default=None) + mr_commits = mr_subs.add_parser("commits") + mr_commits.add_argument("project") + mr_commits.add_argument("mr_iid", type=int) + mr_commits.add_argument("--limit", type=int, default=None) + mr_pipelines = mr_subs.add_parser("pipelines") + mr_pipelines.add_argument("project") + mr_pipelines.add_argument("mr_iid", type=int) + mr_pipelines.add_argument("--limit", type=int, default=None) + + # pipeline + pipe_p = subparsers.add_parser("pipeline") + pipe_subs = pipe_p.add_subparsers(dest="action", required=True) + pipe_status = pipe_subs.add_parser("status") + pipe_status.add_argument("project") + pipe_status.add_argument("pipeline_id", type=int) + + args = parser.parse_args() + + try: + config = load_config() + res = None + if args.command == "repo" and args.action == "get": + res = get_project(args.project, config) + elif args.command == "issue": + if args.action == "list": + res = list_issues(args.project, config, state=args.state, limit=args.limit) + elif args.action == "get": + res = get_issue(args.project, args.issue_iid, config) + elif args.command == "mr": + if args.action == "list": + res = list_mrs(args.project, config, state=args.state, limit=args.limit) + elif args.action == "get": + res = get_mr(args.project, args.mr_iid, config) + elif args.action == "diff": + res = get_mr_diff(args.project, args.mr_iid, config, limit=args.limit) + elif args.action == "commits": + res = get_mr_commits(args.project, args.mr_iid, config, limit=args.limit) + elif args.action == "pipelines": + res = list_mr_pipelines(args.project, args.mr_iid, config, limit=args.limit) + elif args.command == "pipeline" and args.action == "status": + res = get_pipeline_status(args.project, args.pipeline_id, config) + + if res is None: + parser.print_help() + return 1 + + print(json.dumps(res, indent=2)) + return 0 + except Exception as e: + print(f"Error: {e}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/gitlab/src/magpie_gitlab/client.py b/tools/gitlab/src/magpie_gitlab/client.py new file mode 100644 index 000000000..e02d4e705 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/client.py @@ -0,0 +1,269 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import json +import math +import os +import sys +import urllib.error +import urllib.parse +import urllib.request +from dataclasses import dataclass, field +from typing import Any + +DEFAULT_TIMEOUT_SECONDS = 30 +DEFAULT_MAX_PAGES = 10 + + +class GitLabError(Exception): + pass + + +@dataclass +class GitLabConfig: + token: str | None + instance_url: str + token_type: str = field(default="bearer") + auth_scheme: str = field(default="") + + +# --------------------------------------------------------------------------- +# URL validation +# --------------------------------------------------------------------------- + + +def validate_instance_url(url: str) -> None: + """Reject non-HTTPS URLs unless they target localhost for local dev.""" + parsed = urllib.parse.urlparse(url) + if parsed.scheme == "https": + return + if parsed.scheme == "http" and parsed.hostname in ("localhost", "127.0.0.1", "::1"): + return + raise GitLabError(f"Insecure instance URL scheme '{parsed.scheme}': HTTPS is required") + + +# --------------------------------------------------------------------------- +# Safe redirect handler -- prevents token leak on cross-origin or +# HTTPS->HTTP redirects (CWE-200 / CWE-319). +# --------------------------------------------------------------------------- + + +class _SafeRedirectHandler(urllib.request.HTTPRedirectHandler): + """Block redirects that would leak credentials to another origin.""" + + def redirect_request( + self, + req: urllib.request.Request, + fp: Any, + code: int, + msg: str, + headers: Any, + newurl: str, + ) -> urllib.request.Request | None: + orig = urllib.parse.urlparse(req.full_url) + resolved_dest = urllib.parse.urljoin(req.full_url, newurl) + dest = urllib.parse.urlparse(resolved_dest) + + # Deny transport downgrade (HTTPS -> HTTP) + if orig.scheme == "https" and dest.scheme != "https": + raise GitLabError("Redirect blocked: HTTPS-to-HTTP downgrade is forbidden") + + # Deny cross-origin redirect (scheme, hostname, port) + orig_origin = (orig.scheme, orig.hostname, orig.port) + dest_origin = (dest.scheme, dest.hostname, dest.port) + if orig_origin != dest_origin: + raise GitLabError( + f"Redirect blocked: cross-origin redirect from {orig_origin} to {dest_origin} is forbidden" + ) + + return super().redirect_request(req, fp, code, msg, headers, resolved_dest) + + +def _build_opener() -> urllib.request.OpenerDirector: + return urllib.request.build_opener(_SafeRedirectHandler) + + +# --------------------------------------------------------------------------- +# Config loading +# --------------------------------------------------------------------------- + + +def load_config() -> GitLabConfig: + """Build a ``GitLabConfig`` from environment variables. + + Prefers ``GITLAB_TOKEN`` (Personal Access Token, sent as ``PRIVATE-TOKEN:`` + when starting with ``glpat-`` or ``Authorization: Bearer`` otherwise). + Falls back to ``CI_JOB_TOKEN`` (sent as ``JOB-TOKEN:``). + Tokens are optional for unauthenticated reads on public projects. + """ + gitlab_token = os.environ.get("GITLAB_TOKEN") + ci_job_token = os.environ.get("CI_JOB_TOKEN") + auth_scheme = os.environ.get("GITLAB_AUTH_SCHEME", "") + + if gitlab_token: + token = gitlab_token + token_type = "private-token" if token.startswith("glpat-") else "bearer" + elif ci_job_token: + token = ci_job_token + token_type = "job_token" + else: + token = None + token_type = "bearer" + + instance_url = os.environ.get("GITLAB_INSTANCE_URL", "https://gitlab.com").rstrip("/") + validate_instance_url(instance_url) + return GitLabConfig( + token=token, + instance_url=instance_url, + token_type=token_type, + auth_scheme=auth_scheme, + ) + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def quote_path(value: str) -> str: + return urllib.parse.quote(value, safe="") + + +def _auth_headers(config: GitLabConfig) -> dict[str, str]: + """Return the correct authentication header for the token type, or none if unauthenticated.""" + headers: dict[str, str] = {"Accept": "application/json"} + if not config.token: + return headers + + if config.auth_scheme: + scheme = config.auth_scheme.strip().lower() + if scheme in ("private-token", "privatetoken"): + headers["PRIVATE-TOKEN"] = config.token + elif scheme == "bearer": + headers["Authorization"] = f"Bearer {config.token}" + elif scheme in ("job-token", "job_token", "jobtoken"): + headers["JOB-TOKEN"] = config.token + else: + raise GitLabError(f"Unsupported GITLAB_AUTH_SCHEME: '{config.auth_scheme}'") + else: + if config.token_type == "job_token": + headers["JOB-TOKEN"] = config.token + elif config.token.startswith("glpat-"): + headers["PRIVATE-TOKEN"] = config.token + else: + headers["Authorization"] = f"Bearer {config.token}" + + return headers + + +# --------------------------------------------------------------------------- +# Core HTTP helpers +# --------------------------------------------------------------------------- + + +def get_json(url: str, config: GitLabConfig) -> Any: + """Fetch a single JSON resource (no pagination).""" + validate_instance_url(url) + headers = _auth_headers(config) + request = urllib.request.Request(url, headers=headers, method="GET") + opener = _build_opener() + try: + with opener.open(request, timeout=DEFAULT_TIMEOUT_SECONDS) as response: + return json.loads(response.read().decode("utf-8")) + except urllib.error.HTTPError as exc: + raise GitLabError(f"HTTP {exc.code}: {exc.reason}") from exc + except GitLabError: + raise + except Exception as exc: + raise GitLabError(f"Request failed: {exc}") from exc + + +def get_paged_json( + url: str, + config: GitLabConfig, + limit: int | None = None, + max_pages: int | None = DEFAULT_MAX_PAGES, +) -> list[Any]: + """Fetch a paginated JSON collection, following ``X-Next-Page`` up to ``limit`` or ``max_pages``.""" + validate_instance_url(url) + headers = _auth_headers(config) + items: list[Any] = [] + separator = "&" if "?" in url else "?" + current_url: str | None = f"{url}{separator}per_page=100" if "per_page=" not in url else url + pages_fetched = 0 + + target_pages: int | None = max_pages + if limit is not None: + target_pages = max(1, math.ceil(limit / 100)) + + opener = _build_opener() + while current_url: + request = urllib.request.Request(current_url, headers=headers, method="GET") + try: + with opener.open(request, timeout=DEFAULT_TIMEOUT_SECONDS) as response: + data = json.loads(response.read().decode("utf-8")) + if isinstance(data, list): + items.extend(data) + else: + if not items: + return [data] + raise GitLabError("Unexpected non-list response during pagination") + + pages_fetched += 1 + next_page = response.headers.get("X-Next-Page") + has_more = isinstance(next_page, str) and bool(next_page.strip()) + + if limit is not None and len(items) >= limit: + items = items[:limit] + break + + if target_pages is not None and pages_fetched >= target_pages: + if has_more and limit is None: + print( + f"[magpie-gitlab] Note: Results capped at {len(items)} items ({pages_fetched} pages); use --limit to fetch more.", + file=sys.stderr, + ) + break + + if has_more and isinstance(next_page, str): + parsed = urllib.parse.urlparse(current_url) + query = urllib.parse.parse_qs(parsed.query) + query["page"] = [next_page.strip()] + new_query = urllib.parse.urlencode(query, doseq=True) + current_url = urllib.parse.urlunparse(parsed._replace(query=new_query)) + else: + current_url = None + except urllib.error.HTTPError as exc: + raise GitLabError(f"HTTP {exc.code}: {exc.reason}") from exc + except GitLabError: + raise + except Exception as exc: + raise GitLabError(f"Request failed: {exc}") from exc + + return items + + +# --------------------------------------------------------------------------- +# High-level resource helpers +# --------------------------------------------------------------------------- + + +def get_project(project: str, config: GitLabConfig) -> Any: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}" + return get_json(url, config) diff --git a/tools/gitlab/src/magpie_gitlab/issues.py b/tools/gitlab/src/magpie_gitlab/issues.py new file mode 100644 index 000000000..5a1465821 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/issues.py @@ -0,0 +1,39 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import urllib.parse +from typing import Any + +from .client import GitLabConfig, get_json, get_paged_json, quote_path + + +def list_issues( + project: str, + config: GitLabConfig, + state: str = "opened", + limit: int | None = None, +) -> list[Any]: + query = urllib.parse.urlencode({"state": state}) + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/issues?{query}" + return get_paged_json(url, config, limit=limit) + + +def get_issue(project: str, issue_iid: int | str, config: GitLabConfig) -> Any: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/issues/{quote_path(str(issue_iid))}" + return get_json(url, config) diff --git a/tools/gitlab/src/magpie_gitlab/merge_requests.py b/tools/gitlab/src/magpie_gitlab/merge_requests.py new file mode 100644 index 000000000..3649f0640 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/merge_requests.py @@ -0,0 +1,60 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import urllib.parse +from typing import Any + +from .client import GitLabConfig, get_json, get_paged_json, quote_path + + +def list_mrs( + project: str, + config: GitLabConfig, + state: str = "opened", + limit: int | None = None, +) -> list[Any]: + query = urllib.parse.urlencode({"state": state}) + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/merge_requests?{query}" + return get_paged_json(url, config, limit=limit) + + +def get_mr(project: str, mr_iid: int | str, config: GitLabConfig) -> Any: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/merge_requests/{quote_path(str(mr_iid))}" + return get_json(url, config) + + +def get_mr_diff( + project: str, + mr_iid: int | str, + config: GitLabConfig, + limit: int | None = None, +) -> list[Any]: + """Fetch MR diff hunks using the paginated /diffs endpoint (GitLab 15.7+).""" + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/merge_requests/{quote_path(str(mr_iid))}/diffs" + return get_paged_json(url, config, limit=limit) + + +def get_mr_commits( + project: str, + mr_iid: int | str, + config: GitLabConfig, + limit: int | None = None, +) -> list[Any]: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/merge_requests/{quote_path(str(mr_iid))}/commits" + return get_paged_json(url, config, limit=limit) diff --git a/tools/gitlab/src/magpie_gitlab/pipelines.py b/tools/gitlab/src/magpie_gitlab/pipelines.py new file mode 100644 index 000000000..4fa5753e3 --- /dev/null +++ b/tools/gitlab/src/magpie_gitlab/pipelines.py @@ -0,0 +1,37 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +from typing import Any + +from .client import GitLabConfig, get_json, get_paged_json, quote_path + + +def get_pipeline_status(project: str, pipeline_id: int | str, config: GitLabConfig) -> Any: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/pipelines/{quote_path(str(pipeline_id))}" + return get_json(url, config) + + +def list_mr_pipelines( + project: str, + mr_iid: int | str, + config: GitLabConfig, + limit: int | None = None, +) -> list[Any]: + url = f"{config.instance_url}/api/v4/projects/{quote_path(project)}/merge_requests/{quote_path(str(mr_iid))}/pipelines" + return get_paged_json(url, config, limit=limit) diff --git a/tools/gitlab/src/magpie_gitlab/py.typed b/tools/gitlab/src/magpie_gitlab/py.typed new file mode 100644 index 000000000..e69de29bb diff --git a/tools/gitlab/tests/__init__.py b/tools/gitlab/tests/__init__.py new file mode 100644 index 000000000..2df32291c --- /dev/null +++ b/tools/gitlab/tests/__init__.py @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations diff --git a/tools/gitlab/tests/conftest.py b/tools/gitlab/tests/conftest.py new file mode 100644 index 000000000..40262c7c6 --- /dev/null +++ b/tools/gitlab/tests/conftest.py @@ -0,0 +1,66 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import json +import urllib.request +from typing import Any +from unittest import mock + +import pytest + + +@pytest.fixture +def mock_urlopen(monkeypatch): + """Patch both urlopen and build_opener so the safe-redirect opener + created inside get_json / get_paged_json goes through the same mock. + """ + mock_open = mock.MagicMock() + mock_opener = mock.MagicMock() + mock_opener.open = mock_open + monkeypatch.setattr(urllib.request, "urlopen", mock_open) + monkeypatch.setattr( + urllib.request, + "build_opener", + lambda *args, **kwargs: mock_opener, + ) + return mock_open + + +def build_mock_response( + json_data: Any, + status: int = 200, + headers: dict[str, str] | None = None, +) -> mock.MagicMock: + body = json.dumps(json_data).encode("utf-8") + resp = mock.MagicMock() + resp.read.return_value = body + resp.status = status + _headers = headers if headers is not None else {} + resp.headers = _headers + resp.__enter__.return_value = resp + resp.__exit__.return_value = None + return resp + + +@pytest.fixture +def mock_env(monkeypatch): + monkeypatch.setenv("GITLAB_TOKEN", "glpat-test123") + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.delenv("CI_JOB_TOKEN", raising=False) + monkeypatch.setenv("GITLAB_INSTANCE_URL", "https://gitlab.example.com") diff --git a/tools/gitlab/tests/test_cli.py b/tools/gitlab/tests/test_cli.py new file mode 100644 index 000000000..20ada80fb --- /dev/null +++ b/tools/gitlab/tests/test_cli.py @@ -0,0 +1,143 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import json + +import pytest + +from magpie_gitlab.cli import main + +from .conftest import build_mock_response + + +def test_cli_issue_get(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response({"id": 1, "title": "CLI Test"}) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "issue", "get", "group/project", "1"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert res["title"] == "CLI Test" + + +def test_cli_repo_get(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response({"id": 99, "name": "repo-test"}) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "repo", "get", "group/repo-test"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert res["id"] == 99 + assert res["name"] == "repo-test" + + +def test_cli_mr_diff(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response([{"diff": "@@ -1 +1 @@", "new_path": "a.py"}]) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "mr", "diff", "group/project", "5"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert len(res) == 1 + assert res[0]["new_path"] == "a.py" + + +def test_cli_mr_commits(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response([{"id": "c1", "message": "feat: init"}]) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "mr", "commits", "group/project", "5"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert len(res) == 1 + assert res[0]["id"] == "c1" + + +def test_cli_mr_pipelines(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response([{"id": 101, "status": "success"}]) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "mr", "pipelines", "group/project", "5"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert len(res) == 1 + assert res[0]["id"] == 101 + assert res[0]["status"] == "success" + + +def test_cli_pipeline_status(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response({"id": 55, "status": "running"}) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "pipeline", "status", "group/project", "55"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert res["id"] == 55 + assert res["status"] == "running" + + +def test_cli_issue_list_limit(mock_urlopen, mock_env, monkeypatch, capsys): + mock_urlopen.return_value = build_mock_response([{"id": 1}, {"id": 2}]) + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "issue", "list", "group/project", "--limit", "2"]) + + assert main() == 0 + + captured = capsys.readouterr() + res = json.loads(captured.out) + assert len(res) == 2 + + +def test_cli_issue_get_invalid_id_type_fails(monkeypatch): + """Passing a non-integer for an ID positional must fail argument parsing.""" + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "issue", "get", "group/project", "notanint"]) + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code != 0 + + +def test_cli_mr_diff_invalid_id_type_fails(monkeypatch): + """Path traversal attempt via ID string must fail argument parsing.""" + monkeypatch.setattr("sys.argv", ["magpie-gitlab", "mr", "diff", "group/project", "../../admin"]) + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code != 0 + + +def test_cli_state_invalid_choice_fails(monkeypatch): + """Unsupported state choice must fail argument parsing.""" + monkeypatch.setattr( + "sys.argv", + ["magpie-gitlab", "issue", "list", "group/project", "--state", "invalid_state"], + ) + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code != 0 + + +def test_cli_no_args_fails(monkeypatch): + monkeypatch.setattr("sys.argv", ["magpie-gitlab"]) + with pytest.raises(SystemExit) as exc_info: + main() + assert exc_info.value.code != 0 diff --git a/tools/gitlab/tests/test_client.py b/tools/gitlab/tests/test_client.py new file mode 100644 index 000000000..664b0a78d --- /dev/null +++ b/tools/gitlab/tests/test_client.py @@ -0,0 +1,421 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +import urllib.error +import urllib.request as _ur +from email.message import Message + +import pytest + +from magpie_gitlab.client import ( + GitLabConfig, + GitLabError, + _auth_headers, + _build_opener, + _SafeRedirectHandler, + get_json, + get_paged_json, + get_project, + load_config, + quote_path, +) + +from .conftest import build_mock_response + +# --------------------------------------------------------------------------- +# load_config +# --------------------------------------------------------------------------- + + +def test_load_config_default(monkeypatch): + monkeypatch.delenv("GITLAB_INSTANCE_URL", raising=False) + monkeypatch.delenv("CI_JOB_TOKEN", raising=False) + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_TOKEN", "token") + cfg = load_config() + assert cfg.instance_url == "https://gitlab.com" + assert cfg.token == "token" + assert cfg.token_type == "bearer" + assert cfg.auth_scheme == "" + + +def test_load_config_ci_job_token(monkeypatch): + """CI_JOB_TOKEN should be used when GITLAB_TOKEN is absent.""" + monkeypatch.delenv("GITLAB_INSTANCE_URL", raising=False) + monkeypatch.delenv("GITLAB_TOKEN", raising=False) + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("CI_JOB_TOKEN", "ci-job-tok-456") + cfg = load_config() + assert cfg.token == "ci-job-tok-456" + assert cfg.token_type == "job_token" + + +def test_load_config_gitlab_token_takes_precedence(monkeypatch): + """GITLAB_TOKEN wins when both are set.""" + monkeypatch.delenv("GITLAB_INSTANCE_URL", raising=False) + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_TOKEN", "pat-wins") + monkeypatch.setenv("CI_JOB_TOKEN", "ci-loses") + cfg = load_config() + assert cfg.token == "pat-wins" + assert cfg.token_type == "bearer" + + +def test_load_config_insecure_url(monkeypatch): + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_TOKEN", "token") + monkeypatch.setenv("GITLAB_INSTANCE_URL", "http://gitlab.insecure.com") + with pytest.raises( + GitLabError, + match="Insecure instance URL scheme 'http': HTTPS is required", + ): + load_config() + + +def test_load_config_localhost_http_allowed(monkeypatch): + """HTTP is allowed for localhost / 127.0.0.1 / ::1 (local dev / testing).""" + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_TOKEN", "token") + monkeypatch.setenv("GITLAB_INSTANCE_URL", "http://localhost:8080") + cfg = load_config() + assert cfg.instance_url == "http://localhost:8080" + + monkeypatch.setenv("GITLAB_INSTANCE_URL", "http://127.0.0.1:8080") + cfg2 = load_config() + assert cfg2.instance_url == "http://127.0.0.1:8080" + + +def test_load_config_localhost_non_http_rejected(monkeypatch): + """Non-HTTP schemes on localhost (ftp, file, etc.) must be rejected.""" + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_TOKEN", "token") + monkeypatch.setenv("GITLAB_INSTANCE_URL", "ftp://localhost:21") + with pytest.raises( + GitLabError, + match="Insecure instance URL scheme 'ftp': HTTPS is required", + ): + load_config() + + monkeypatch.setenv("GITLAB_INSTANCE_URL", "file://localhost/tmp") + with pytest.raises( + GitLabError, + match="Insecure instance URL scheme 'file': HTTPS is required", + ): + load_config() + + +def test_load_config_custom(mock_env): + cfg = load_config() + assert cfg.instance_url == "https://gitlab.example.com" + assert cfg.token == "glpat-test123" + + +# --------------------------------------------------------------------------- +# quote_path +# --------------------------------------------------------------------------- + + +def test_quote_path(): + assert quote_path("group/project") == "group%2Fproject" + + +# --------------------------------------------------------------------------- +# _build_opener (unmocked) +# --------------------------------------------------------------------------- + + +def test_build_opener_installs_safe_redirect_handler(): + opener = _build_opener() + handlers = getattr(opener, "handlers", []) + assert any(isinstance(h, _SafeRedirectHandler) for h in handlers) + + +# --------------------------------------------------------------------------- +# SafeRedirectHandler +# --------------------------------------------------------------------------- + + +def test_safe_redirect_blocks_https_to_http(): + """HTTPS->HTTP downgrade must be blocked.""" + handler = _SafeRedirectHandler() + req = _ur.Request("https://gitlab.example.com/api") + with pytest.raises(GitLabError, match="HTTPS-to-HTTP downgrade"): + handler.redirect_request(req, None, 302, "Found", {}, "http://gitlab.example.com/api") + + +def test_safe_redirect_blocks_cross_origin(): + """Cross-origin host redirect must be blocked.""" + handler = _SafeRedirectHandler() + req = _ur.Request("https://gitlab.example.com/api") + with pytest.raises(GitLabError, match="cross-origin redirect"): + handler.redirect_request(req, None, 302, "Found", {}, "https://evil.example.com/steal") + + +def test_safe_redirect_blocks_port_mismatch(): + """Cross-port redirect on same host must be blocked.""" + handler = _SafeRedirectHandler() + req = _ur.Request("https://gitlab.example.com/api") + with pytest.raises(GitLabError, match="cross-origin redirect"): + handler.redirect_request(req, None, 302, "Found", {}, "https://gitlab.example.com:8443/api") + + +def test_safe_redirect_allows_same_origin(): + """Same-origin same-scheme absolute redirect should be allowed.""" + handler = _SafeRedirectHandler() + req = _ur.Request("https://gitlab.example.com/api/old") + result = handler.redirect_request(req, None, 302, "Found", {}, "https://gitlab.example.com/api/new") + assert result is not None + assert result.full_url == "https://gitlab.example.com/api/new" + + +def test_safe_redirect_allows_relative_same_origin(): + """Relative redirect on same origin should be resolved and allowed.""" + handler = _SafeRedirectHandler() + req = _ur.Request("https://gitlab.example.com/api/v4/projects") + result = handler.redirect_request(req, None, 302, "Found", {}, "/api/v4/projects/1") + assert result is not None + assert result.full_url == "https://gitlab.example.com/api/v4/projects/1" + + +# --------------------------------------------------------------------------- +# _auth_headers +# --------------------------------------------------------------------------- + + +def test_auth_headers_unauthenticated(): + cfg = GitLabConfig(token=None, instance_url="https://gitlab.example.com") + headers = _auth_headers(cfg) + assert headers == {"Accept": "application/json"} + + +def test_auth_headers_glpat_default(): + cfg = GitLabConfig(token="glpat-secret", instance_url="https://gitlab.example.com") + headers = _auth_headers(cfg) + assert headers.get("PRIVATE-TOKEN") == "glpat-secret" + assert "Authorization" not in headers + + +def test_auth_headers_bearer_default(): + cfg = GitLabConfig(token="oauth-token", instance_url="https://gitlab.example.com") + headers = _auth_headers(cfg) + assert headers.get("Authorization") == "Bearer oauth-token" + assert "PRIVATE-TOKEN" not in headers + + +def test_auth_headers_job_token_default(): + cfg = GitLabConfig(token="job-tok", instance_url="https://gitlab.example.com", token_type="job_token") + headers = _auth_headers(cfg) + assert headers.get("JOB-TOKEN") == "job-tok" + + +def test_auth_headers_explicit_scheme_bearer(): + """Explicit GITLAB_AUTH_SCHEME=Bearer overrides glpat- default.""" + cfg = GitLabConfig( + token="glpat-token", + instance_url="https://gitlab.example.com", + auth_scheme="Bearer", + ) + headers = _auth_headers(cfg) + assert headers.get("Authorization") == "Bearer glpat-token" + assert "PRIVATE-TOKEN" not in headers + + +def test_auth_headers_explicit_scheme_private_token(): + cfg = GitLabConfig( + token="custom-pat", + instance_url="https://gitlab.example.com", + auth_scheme="Private-Token", + ) + headers = _auth_headers(cfg) + assert headers.get("PRIVATE-TOKEN") == "custom-pat" + + +def test_auth_headers_explicit_scheme_job_token(): + cfg = GitLabConfig( + token="custom-job-tok", + instance_url="https://gitlab.example.com", + auth_scheme="Job-Token", + ) + headers = _auth_headers(cfg) + assert headers.get("JOB-TOKEN") == "custom-job-tok" + + +def test_auth_headers_explicit_scheme_jobtoken_no_hyphen(): + cfg = GitLabConfig( + token="custom-job-tok", + instance_url="https://gitlab.example.com", + auth_scheme="JobToken", + ) + headers = _auth_headers(cfg) + assert headers.get("JOB-TOKEN") == "custom-job-tok" + + +def test_auth_headers_invalid_scheme_raises(): + cfg = GitLabConfig( + token="token", + instance_url="https://gitlab.example.com", + auth_scheme="Basic", + ) + with pytest.raises(GitLabError, match="Unsupported GITLAB_AUTH_SCHEME: 'Basic'"): + _auth_headers(cfg) + + +# --------------------------------------------------------------------------- +# get_json +# --------------------------------------------------------------------------- + + +def test_get_json_unauthenticated(mock_urlopen, monkeypatch): + """Unauthenticated public reads should succeed with no auth header.""" + monkeypatch.delenv("GITLAB_TOKEN", raising=False) + monkeypatch.delenv("CI_JOB_TOKEN", raising=False) + monkeypatch.delenv("GITLAB_AUTH_SCHEME", raising=False) + monkeypatch.setenv("GITLAB_INSTANCE_URL", "https://gitlab.example.com") + mock_urlopen.return_value = build_mock_response({"public": "repo"}) + cfg = load_config() + res = get_json("https://gitlab.example.com/api/v4/projects/public%2Frepo", cfg) + assert res == {"public": "repo"} + req = mock_urlopen.call_args[0][0] + assert "Authorization" not in req.headers + assert "Private-token" not in req.headers + assert "Job-token" not in req.headers + assert req.headers.get("Accept") == "application/json" + + +def test_get_json_http_error(mock_urlopen, mock_env): + mock_urlopen.side_effect = urllib.error.HTTPError("url", 404, "Not Found", Message(), None) + cfg = load_config() + with pytest.raises(GitLabError, match="HTTP 404: Not Found"): + get_json("https://gitlab.example.com/api", cfg) + + +# --------------------------------------------------------------------------- +# get_paged_json -- pagination & bounded limit +# --------------------------------------------------------------------------- + + +def test_get_paged_json_single_page(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1}], headers={"X-Next-Page": ""}) + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg) + assert items == [{"id": 1}] + assert mock_urlopen.call_count == 1 + + +def test_get_paged_json_multi_page(mock_urlopen, mock_env): + page1 = build_mock_response([{"id": 1}], headers={"X-Next-Page": "2"}) + page2 = build_mock_response([{"id": 2}], headers={"X-Next-Page": ""}) + mock_urlopen.side_effect = [page1, page2] + + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues?state=opened", cfg) + assert items == [{"id": 1}, {"id": 2}] + assert mock_urlopen.call_count == 2 + # Verify second request preserved query params + req2 = mock_urlopen.call_args_list[1][0][0] + assert "state=opened" in req2.full_url + assert "page=2" in req2.full_url + + +def test_get_paged_json_max_pages(mock_urlopen, mock_env, capsys): + """Pagination must halt when max_pages ceiling is reached.""" + page1 = build_mock_response([{"id": 1}], headers={"X-Next-Page": "2"}) + page2 = build_mock_response([{"id": 2}], headers={"X-Next-Page": "3"}) + page3 = build_mock_response([{"id": 3}], headers={"X-Next-Page": ""}) + mock_urlopen.side_effect = [page1, page2, page3] + + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg, max_pages=2) + assert items == [{"id": 1}, {"id": 2}] + assert mock_urlopen.call_count == 2 + err = capsys.readouterr().err + assert "Results capped at 2 items (2 pages)" in err + + +def test_get_paged_json_with_limit_fewer_than_page(mock_urlopen, mock_env, capsys): + """Limit fewer than page size fetches only 1 page, caps items, and emits notice if more exist.""" + data = [{"id": i} for i in range(100)] + mock_urlopen.return_value = build_mock_response(data, headers={"X-Next-Page": "2"}) + + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg, limit=5) + assert len(items) == 5 + assert mock_urlopen.call_count == 1 + err = capsys.readouterr().err + assert err == "" + + +def test_get_paged_json_with_limit_multi_page(mock_urlopen, mock_env): + """Limit spanning multiple pages only fetches the required number of pages.""" + page1 = build_mock_response([{"id": i} for i in range(100)], headers={"X-Next-Page": "2"}) + page2 = build_mock_response([{"id": i} for i in range(100, 200)], headers={"X-Next-Page": "3"}) + mock_urlopen.side_effect = [page1, page2] + + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg, limit=150) + assert len(items) == 150 + assert mock_urlopen.call_count == 2 + + +def test_get_paged_json_non_list_single(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response({"single": "object"}) + cfg = load_config() + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/resource", cfg) + assert items == [{"single": "object"}] + + +def test_get_paged_json_non_list_subsequent_raises(mock_urlopen, mock_env): + page1 = build_mock_response([{"id": 1}], headers={"X-Next-Page": "2"}) + page2 = build_mock_response({"error": "invalid"}, headers={"X-Next-Page": ""}) + mock_urlopen.side_effect = [page1, page2] + + cfg = load_config() + with pytest.raises(GitLabError, match="Unexpected non-list response"): + get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg) + + +def test_get_paged_json_limit_gt_1000(mock_urlopen, mock_env): + """Limit > 1000 should bypass the max_pages=10 default and fetch required pages.""" + pages = [] + # limit=1050 means we need 11 pages (100 items per page). + for i in range(1, 12): + next_page = str(i + 1) if i < 11 else "" + pages.append(build_mock_response([{"id": j} for j in range(100)], headers={"X-Next-Page": next_page})) + + mock_urlopen.side_effect = pages + + cfg = load_config() + # default max_pages is 10, but limit=1050 should override target_pages to 11 + items = get_paged_json("https://gitlab.example.com/api/v4/projects/test/issues", cfg, limit=1050) + assert len(items) == 1050 + assert mock_urlopen.call_count == 11 + + +# --------------------------------------------------------------------------- +# get_project +# --------------------------------------------------------------------------- + + +def test_get_project(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response({"id": 42, "name": "my-project"}) + cfg = load_config() + project = get_project("group/my-project", cfg) + assert project == {"id": 42, "name": "my-project"} + req = mock_urlopen.call_args[0][0] + assert req.full_url == "https://gitlab.example.com/api/v4/projects/group%2Fmy-project" diff --git a/tools/gitlab/tests/test_issues.py b/tools/gitlab/tests/test_issues.py new file mode 100644 index 000000000..4a13c4095 --- /dev/null +++ b/tools/gitlab/tests/test_issues.py @@ -0,0 +1,54 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +from magpie_gitlab.client import load_config +from magpie_gitlab.issues import get_issue, list_issues + +from .conftest import build_mock_response + + +def test_list_issues(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1, "title": "Issue 1"}]) + cfg = load_config() + res = list_issues("group/project", cfg) + assert len(res) == 1 + assert res[0]["title"] == "Issue 1" + + req = mock_urlopen.call_args[0][0] + assert ( + req.full_url + == "https://gitlab.example.com/api/v4/projects/group%2Fproject/issues?state=opened&per_page=100" + ) + + +def test_list_issues_limit(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1}, {"id": 2}]) + cfg = load_config() + res = list_issues("group/project", cfg, limit=2) + assert len(res) == 2 + + +def test_get_issue(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response({"id": 1, "title": "Issue 1"}) + cfg = load_config() + res = get_issue("group/project", 1, cfg) + assert res["id"] == 1 + + req = mock_urlopen.call_args[0][0] + assert req.full_url == "https://gitlab.example.com/api/v4/projects/group%2Fproject/issues/1" diff --git a/tools/gitlab/tests/test_merge_requests.py b/tools/gitlab/tests/test_merge_requests.py new file mode 100644 index 000000000..20e8f8b7f --- /dev/null +++ b/tools/gitlab/tests/test_merge_requests.py @@ -0,0 +1,84 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +from magpie_gitlab.client import load_config +from magpie_gitlab.merge_requests import get_mr, get_mr_commits, get_mr_diff, list_mrs + +from .conftest import build_mock_response + + +def test_list_mrs(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1, "title": "MR 1"}]) + cfg = load_config() + res = list_mrs("group/project", cfg) + assert len(res) == 1 + req = mock_urlopen.call_args[0][0] + assert ( + req.full_url + == "https://gitlab.example.com/api/v4/projects/group%2Fproject/merge_requests?state=opened&per_page=100" + ) + + +def test_list_mrs_limit(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1}, {"id": 2}]) + cfg = load_config() + res = list_mrs("group/project", cfg, limit=2) + assert len(res) == 2 + + +def test_get_mr(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response({"id": 1}) + cfg = load_config() + res = get_mr("group/project", 1, cfg) + assert res["id"] == 1 + req = mock_urlopen.call_args[0][0] + assert req.full_url == "https://gitlab.example.com/api/v4/projects/group%2Fproject/merge_requests/1" + + +def test_get_mr_diff(mock_urlopen, mock_env): + """MR diff must use the paginated /diffs endpoint.""" + mock_urlopen.return_value = build_mock_response([{"diff": "@@ -1 +1 @@\n-a\n+b", "new_path": "foo.py"}]) + cfg = load_config() + res = get_mr_diff("group/project", 1, cfg) + assert len(res) == 1 + assert res[0]["new_path"] == "foo.py" + req = mock_urlopen.call_args[0][0] + assert ( + req.full_url + == "https://gitlab.example.com/api/v4/projects/group%2Fproject/merge_requests/1/diffs?per_page=100" + ) + + +def test_get_mr_diff_limit(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"diff": "chunk 1"}, {"diff": "chunk 2"}]) + cfg = load_config() + res = get_mr_diff("group/project", 1, cfg, limit=2) + assert len(res) == 2 + + +def test_get_mr_commits(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": "abc"}]) + cfg = load_config() + res = get_mr_commits("group/project", 1, cfg) + assert len(res) == 1 + req = mock_urlopen.call_args[0][0] + assert ( + req.full_url + == "https://gitlab.example.com/api/v4/projects/group%2Fproject/merge_requests/1/commits?per_page=100" + ) diff --git a/tools/gitlab/tests/test_pipelines.py b/tools/gitlab/tests/test_pipelines.py new file mode 100644 index 000000000..b83c7db80 --- /dev/null +++ b/tools/gitlab/tests/test_pipelines.py @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from __future__ import annotations + +from magpie_gitlab.client import load_config +from magpie_gitlab.pipelines import get_pipeline_status, list_mr_pipelines + +from .conftest import build_mock_response + + +def test_get_pipeline_status(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response({"id": 1, "status": "success"}) + cfg = load_config() + res = get_pipeline_status("group/project", 1, cfg) + assert res["status"] == "success" + req = mock_urlopen.call_args[0][0] + assert req.full_url == "https://gitlab.example.com/api/v4/projects/group%2Fproject/pipelines/1" + + +def test_list_mr_pipelines(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1, "status": "success"}]) + cfg = load_config() + res = list_mr_pipelines("group/project", 1, cfg) + assert len(res) == 1 + req = mock_urlopen.call_args[0][0] + assert ( + req.full_url + == "https://gitlab.example.com/api/v4/projects/group%2Fproject/merge_requests/1/pipelines?per_page=100" + ) + + +def test_list_mr_pipelines_limit(mock_urlopen, mock_env): + mock_urlopen.return_value = build_mock_response([{"id": 1}, {"id": 2}]) + cfg = load_config() + res = list_mr_pipelines("group/project", 1, cfg, limit=2) + assert len(res) == 2 diff --git a/tools/gitlab/tool.md b/tools/gitlab/tool.md new file mode 100644 index 000000000..00f10865d --- /dev/null +++ b/tools/gitlab/tool.md @@ -0,0 +1,43 @@ + + + + +**Table of Contents** + +- [GitLab Tool Adapter](#gitlab-tool-adapter) + - [Operations catalogue](#operations-catalogue) + - [Options and flags](#options-and-flags) + - [Confidentiality](#confidentiality) + + + +# GitLab Tool Adapter + +Operations catalogue mapping for GitLab tracker, source control, and merge requests. + +## Operations catalogue + +| Operation | GitLab command | +| --- | --- | +| Read repository metadata | `magpie-gitlab repo get ` | +| Read issue body | `magpie-gitlab issue get ` | +| List issues | `magpie-gitlab issue list ` | +| List MRs | `magpie-gitlab mr list ` | +| Read MR | `magpie-gitlab mr get ` | +| MR Diff | `magpie-gitlab mr diff ` | +| MR Commits | `magpie-gitlab mr commits ` | +| List MR Pipelines | `magpie-gitlab mr pipelines ` | +| CI Pipeline Status | `magpie-gitlab pipeline status ` | + +## Options and flags + +- `--limit `: Cap the total number of items returned for paginated endpoints (`issue list`, `mr list`, `mr diff`, `mr commits`, `mr pipelines`). Automatically calculates and requests only the required number of pages (`ceil(limit / 100)`). +- `--state `: Filter items by state: + - For `issue list`: `opened` (default), `closed`, `all`. + - For `mr list`: `opened` (default), `closed`, `locked`, `merged`, `all`. + +## Confidentiality + +*Confidentiality Note*: Never log personal access tokens. All payload +bodies are handled purely in memory and output in JSON format. diff --git a/uv.lock b/uv.lock index d36baf3a3..83b8ac08c 100644 --- a/uv.lock +++ b/uv.lock @@ -29,6 +29,7 @@ members = [ "magpie-bitbucket", "magpie-dev", "magpie-fossil", + "magpie-gitlab", "magpie-maildir", "magpie-skills", "magpie-sourcehut", @@ -944,6 +945,21 @@ dev = [ [package.metadata.requires-dev] dev = [{ name = "magpie-dev", editable = "tools/dev" }] +[[package]] +name = "magpie-gitlab" +version = "0.1.0" +source = { editable = "tools/gitlab" } + +[package.dev-dependencies] +dev = [ + { name = "magpie-dev" }, +] + +[package.metadata] + +[package.metadata.requires-dev] +dev = [{ name = "magpie-dev", editable = "tools/dev" }] + [[package]] name = "magpie-maildir" version = "0.1.0"