Running API Tests with Postman CLI & GitHub Actions

Recently I have been dabbing with running API tests written in Postman in CI/CD. As I've been using GitHub as a source control what better than GitHub Actions to integrate this in a CI/CD workflow?

I believe many people reading this article might already be using their Postman API tests/collection using the command line with an existing offering called Newman.

With this new offering, Postman can better control and Check API definition against configured API Governance and API Security Rules (if you are into that)

Newman was reliant on Node.js for running your tests in a CLI which was maintained by the community. Go through this article to find out what best suits your needs: https://learning.postman.com/docs/postman-cli/postman-cli-overview/#comparing-the-postman-cli-and-newman

Getting Started:

In order to start working with GitHub Actions you need to have an Actions workflow .yml file setup in your repository which can be manually placed under .github/workflows/{{some_name}}.yml or you can use the GitHub UI interface to do so.

Which looks like below:

name: run-on-ubuntu
run-name: ${{ github.actor }} started the action 🚀
on:
  workflow_dispatch:
    inputs:
      subdomain:
        required: true
        type: string
        description: 'The sub-domain for geo-location'
      domain:
        required: true
        type: string
        description: 'The url against which test to be run'
      
jobs:
  run-tests:
    runs-on: ubuntu-latest
    outputs:
      output1: ${{ steps.step1.outputs.stdout }}
    steps:
      - name: Checkout & Logging input details
        id: step1
        uses: actions/checkout@v3

      - name: Install Postman CLI
        id: step2
        run: |
          curl -o- "https://dl-cli.pstmn.io/install/linux64.sh" | sh
          echo "🤖 Verifying Postman CLI install.."

      - name: Verify Postman CLI install
        id: step3
        run: |
          postman -v
          echo "💯 Postman CLI installed."

      - name: Postman CLI Login
        id: step4
        run: |
          postman login --with-api-key ABCD-1234-1234-1234-1234-1234
        
      - name: Run Test Collections
        id: step5
        env:
          DOMAIN: ${{ inputs.domain }}
          SUBDOMAIN: ${{ inputs.subdomain }}s
        run: |
            echo $DOMAIN
            echo $SUBDOMAIN
            postman collection run postman_collection.json -g postman_globals.json -e env/QA/postman_environment.json

      - run: echo "✔️ All tests were run with status ${{ job.status }}."

Explanation:

This workflow file doesn't get run on each commit or PR merge rather it uses workflow_dispatch which is Jenkins equivalent for "Build with Parameters". You can have other Action trigger strategies

So basically under YOUR_REPO > Actions,  you will be able to trigger the workflow on demand like below

which should create a run like this:

You can also provide a bunch of Input variables that can be used during runtime passed as GitHub run-time environment variables:

Breaking down different Steps:

Step 1: Checkout & Logging input details

This step uses GitHub checkout@v3 to clone your repo to an Ubuntu machine, you can also extend this step to further log your input variables

Step 2: Install Postman CLI

This step is suggestive it uses Postman instructions for installing Postman CLI on an Ubuntu machine:

Step 3: Verify Postman CLI install

you can verify if the installation was successful before proceeding

Step 4: Postman CLI Login

You can/should authenticate Postman CLI with a valid Postman API-Key, the one used in the repo isn't valid anymore. (There is better handling of passing your security tokens provided by GitHub using Encrypted Secrets)

Step 5: Run Test Collections

This step is basically relying on Postman CLI options to trigger a run based on the items in your repo, for our case we are using something like the below run command:

$ postman collection run postman_collection.json -g postman_globals.json -e env/QA/postman_environment.json

You can have different variations of this command based on your requirements; additionally, if you want to have multiple iterations of your Tests based on varying data sets you can pass an additional param with the file

-d {{file_name)).json

or

-d {{file_name)).csv

That's all from the pre-requisite setup side.

Running the Workflow:

Just head to your GitHub YOUR_REPO > Actions  where you will see all available workflows for that repo and click "Run Workflow" drop-down

Provide your input variables and click the "Run Workflow" button after which you will see a notification that your workflow started

which upon completion should give you the following information

Run Tests Step result:

In this case, I am deliberately making my test fail so you see the overall workflow fails

SPECIAL NOTE:
When you export your Environments or Globals from Postman by default they will not have any values; so make sure before running it from CLI that the variables inside these files have proper values.

Happy Testing!!

GitHub Repo: