Skip to content

ci: run a reduced Python matrix on pull requests - #2003

Open
Bnjoroge1 wants to merge 1 commit into
httpie:masterfrom
Bnjoroge1:ci-pr-matrix
Open

Bnjoroge1 wants to merge 1 commit into
httpie:masterfrom
Bnjoroge1:ci-pr-matrix

Conversation

@Bnjoroge1

Copy link
Copy Markdown

A pull request currently expands to 36 jobs: os: [ubuntu-latest, macos-13, windows-latest] Γ— six Python versions (3.7–3.12) Γ— pyopenssl: [0, 1]. Most changes to httpie/**/*.py cannot be platform-specific, yet each one pays for macOS and Windows six times over.

Pull requests keep the two signals PR CI actually produces, at a ninth of the cost:

      matrix:
        os: ${{ github.event_name == 'pull_request' && fromJSON('["ubuntu-latest"]') || fromJSON('["ubuntu-latest","macos-13","windows-latest"]') }}
        python-version: ${{ github.event_name == 'pull_request' && fromJSON('["3.7","3.12"]') || fromJSON('["3.12","3.11","3.10","3.9","3.8","3.7"]') }}
        pyopenssl: [0, 1]
  • Pull requests: 4 cells β€” Linux, oldest and newest supported Python, both pyopenssl modes. That covers "does this still import on the oldest interpreter", "does CI run at all on the newest one" and "did the OpenSSL path break", which are the failure modes a Python change here can produce.
  • Pushes to master: unchanged, all 36 cells. A macOS or Windows regression still surfaces before a release; it surfaces on master rather than on every review iteration.
  • fail-fast: false is untouched, so one failing cell still does not hide the others.
  • The existing paths: filter already keeps docs-only PRs from starting the workflow.

If platform coverage on a specific PR matters, the same expression can key off a label β€” contains(github.event.pull_request.labels.*.name, 'full-ci') β€” which keeps the fast path fast and the full matrix available on demand.

Caveat on evidence

This change rests on the workflow shape (36 β†’ 4 cells), not on suite measurements: the suite could not be run locally on this machine (Python 3.15, where pytest-httpbin/Flask fails to import β€” unrelated to this repo's own code). The cell count and the covered/not-covered matrix are exact; the time saved per cell is not claimed.

A pull request currently expands to 36 jobs: 3 operating systems x 6 Python
versions x 2 pyopenssl modes. Most changes to httpie/**/*.py cannot be
platform-specific, yet every one of them pays for macOS and Windows six times.

Pull requests now run Linux with the oldest and newest supported Python (3.7
and 3.12) and both pyopenssl modes: 4 cells. They keep the signals PR CI
actually produces (does the oldest interpreter still import it, does the newest
one still work, did the OpenSSL path break) at a ninth of the cost.

Pushes to master are unchanged: all 36 cells, so a platform-specific regression
still surfaces before release. fail-fast: false is untouched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant