Make Github actions/checkout CI tests merge into master HEAD, instead of into the master past

[alfsb]

DO NOT MERGE

This is a test only branch/PR, with the objective of studying ways to make Github CIs to not ignore posterior changes on main/master.

The doc-base history before branching:

commit c45b8a3d6085b8b18a0938e5ac571403da5541ed master HEAD
commit 982980a58b8b05b5cbdb832aa426a5b29df77a46
commit 9391318e46e6fdd09721a9f5e3553089e4382009 <- branch parent
commit 4555190558990db31583a997bfa010d1823d1cae
commit d9e120747d082ae07c1e924b4f85e8bede7f680a

The branching was intentionally done in the past of master to test a secondary question: future history changes anything in GH actions/checkout?

Changed README.md, added, committed and then PR opened. This is the git history from the perspective of the branch:

commit af0d2e63e0c1b5b1e8b6a047437dfedc09a324b1 <- first commit
commit 9391318e46e6fdd09721a9f5e3553089e4382009 <- branch parent
commit 4555190558990db31583a997bfa010d1823d1cae
commit d9e120747d082ae07c1e924b4f85e8bede7f680a

So far, so good. No tests are shown in the Open a pull request page, so we need to fully open the PR then examine the actions/checkout action in action (pun intended).

[alfsb]

After PR is fully opened, the Checkout step, defined as:

    steps:
      - name: "Checkout"
        uses: "actions/checkout@v7"
        with:
          path: "doc-base"

Generates:

Checking out the ref
  /usr/bin/git checkout --progress --force refs/remotes/pull/329/merge
...
  HEAD is now at c074472 Merge af0d2e63e0c1b5b1e8b6a047437dfedc09a324b1 into c45b8a3d6085b8b18a0938e5ac571403da5541ed

c45b is the current master HEAD, af0d is the current branch HEAD, and c074 is the temporary merge, created only for CI testing. So GI actions/checkout in fact observes the master HEAD at the creation time.

Again, so far so good. Now start the waiting game.

The observed problem occurs after the PR creation time. We now need to wait for a few new commits to arrive into master, to see if actions/checkout somehow "follows" the master HEAD, or if it gets stuck into the past of master, at c45b.

[alfsb]

Now there is some new commits on master HEAD, we can now analyse how the PR is affected.

The master HEAD moved, and commit history is now:

commit ba8225725f394cbff6d88490640965268e84a0c2 new master HEAD
commit c45b8a3d6085b8b18a0938e5ac571403da5541ed old master HEAD
commit 982980a58b8b05b5cbdb832aa426a5b29df77a46
commit 9391318e46e6fdd09721a9f5e3553089e4382009 <- branch parent
commit 4555190558990db31583a997bfa010d1823d1cae
commit d9e120747d082ae07c1e924b4f85e8bede7f680a

Before re-running all tests, the Checkout step shows, abbreviated:

HEAD is now at c074472 Merge af0d2e into c45b8a

After re-running all tests, the  Checkout step shows, abbreviated:

HEAD is now at c074472 Merge af0d2e into c45b8a

In other words, master HEAD moved, but the merge target is still stuck in the master's past. The results of GH CI ignored any posterior change. Whatever the change ba8225 does, it has no effect, at least from the perspective of CI.

But this PR is mergeable, that in turn begs the question. What happens with ba8225 changes? Are they lost? Are changes merged after ba8225? And if so, merging after ba8225 will recreate the temporary CI merge c074472, or will create a merge that has a different history than c074472, again, invalidating any CI results?

These are very good questions, and I will examine them soon. But first let's wait a few days, add a new commit on PR's branch, to examine what normal pushes causes on PRs automatic merges.

[alfsb]

A new day. Before changing anything, the merge step generated

HEAD is now at c074472 Merge af0d2e into c45b8a

both before and after re-running all jobs.

And after pushing a new commit, 75437e, the CI re-runs automatically, generating:

HEAD is now at 11da930 Merge 75437e into ba8225

What? The merge point changed from c45b8a to ba8225! From the master past to master HEAD!

In other words, when a PR is pushed, the merge point is recalculated.

This explains some wtf moments that I experienced over the years. Things like, on a already on PR:

• Adding a non breaking change breakes everything on CI;
• Adding a breaking change can breakes nothing on CI; and
• Re-running CI after a fix on same repo lands, fixes nothing; but
• Re-running CI after a fix on another repo lands, fixes everthing.

That is very confusing. I would even say. insane inducting.

[alfsb]

With these observations in place. I would like to proceed as following:

1. Keep this PR open;
2. Use it as in experiments;
3. With the objective of making CI test results always consider master HEAD;
4. At least, in job reruns.

But the first question would be: do we want that?

As noted above, this will be a very hard thing to test, as any push on PR "fixes" the semi-stuckness of actions/checkout. Testing this will be slow and noisy. And the only thing that we may gain is having better results of CI immediately after re-running CI jobs.

So I like to hear comments about proceeding with the plan above. Silent approvals also count.

Comments with ways to accomplish (3) and (4) are very welcome. I searched GH docs and found nothing that could change the behaviour of actions/checkout in this specific aspect.

I also found nothing in GH docs about how to invalidate or erase previous CI results. for when a push is done on master. Something like this would make the proposed funcionaly way more robust.