docs / GitHub human QA

Human video QA for pull requests

Add a real person to your CI/CD loop. A labeled pull request can create a one-time RentAHuman QA run against its preview deployment, then keep a PR comment and commit status updated with the tester's report and accepted video evidence.

What this adds

  • One tester, one run, and narrated screen-and-microphone video.
  • A durable PR comment with status, findings, reproduction steps, and accepted video links.
  • A RentAHuman / human QA commit status that stays pending while the human works and becomes success or failure at close.
  • Stable idempotency so retries for the same PR commit do not create or charge for a duplicate run.

Prerequisites

  1. Create an API key at Account → API Keys.
  2. Keep at least $3.54 in the wallet for one tester at $3.00 plus the 18% platform fee.
  3. Have a successful preview deployment addressable from the PR commit.
  4. Give the GitHub workflow access to the API key as an Actions secret, never as a repository variable or checked-in file.

Configure the repository

Add these repository settings under GitHub Settings → Secrets and variables → Actions:

Secret

RENTAHUMAN_API_KEY

Your raw RentAHuman API key.

Variable

RENTAHUMAN_HUMAN_QA_ENABLED=true

The kill switch for all label, manual, and scheduled runs.

Label

human-qa

Adding this label requests verification for the current commit.

Create the QA template

On the label event, discover the successful preview URL and create a one-time video template. The budget is integer USD cents. Reuse the same idempotency key when a request is retried after a timeout or workflow failure.

POST /api/v1/qa/templates
IDEMPOTENCY_KEY="github_qa_${GITHUB_REPOSITORY}_${PR_NUMBER}_${GITHUB_SHA}"

curl --fail-with-body -X POST https://rentahuman.ai/api/v1/qa/templates \
  -H "X-API-Key: ${RENTAHUMAN_API_KEY}" \
  -H "Idempotency-Key: ${IDEMPOTENCY_KEY}" \
  -H "Content-Type: application/json" \
  -d @- <<JSON
{
  "name": "GitHub QA: #${PR_NUMBER} @ ${GITHUB_SHA:0:12}",
  "targetUrl": "${PREVIEW_URL}",
  "instructions": "Test this deployed pull request as a real user. Record a narrated screen-and-microphone video of the complete flow. Explain every error or confusing interaction and include exact reproduction steps. If the flow succeeds, show the successful end state in the video.",
  "cadence": "once",
  "budgetPerRunCents": 354,
  "payPerTesterCents": 300,
  "testerCount": 1,
  "submissionMode": "video",
  "periodCapCents": 354,
  "idempotencyKey": "${IDEMPOTENCY_KEY}"
}
JSON

Poll and report

The create endpoint returns a template ID. Poll the runs endpoint on a schedule; do not keep a GitHub runner open while a human is testing. Once a run closes, fetch its detail and use the report plus accepted submission evidence to update the existing PR comment.

List and fetch QA runs
curl "https://rentahuman.ai/api/v1/qa/runs?templateId=${TEMPLATE_ID}" \
  -H "X-API-Key: ${RENTAHUMAN_API_KEY}"

# When a run appears, fetch its report:
curl "https://rentahuman.ai/api/v1/qa/runs/${RUN_ID}" \
  -H "X-API-Key: ${RENTAHUMAN_API_KEY}"

Pass criteria

Mark the commit successful only when the report is complete, has an accepted submission, is not degraded, has no findings, and includes at least one accepted video artifact. Otherwise mark it failed and leave the report available from the PR comment.

Secure GitHub Actions trigger

A label-triggered workflow uses pull_request_target so it can read the API secret. That event is safe only when the job checks out the trusted default branch and never executes code from the pull request head. Give the token pull-requests: write and issues: writeso it can create and update the durable comment.

.github/workflows/human-qa.yml
name: Human QA

on:
  pull_request_target:
    types: [labeled]
    branches: [main]
  workflow_dispatch:
    inputs:
      pr_number:
        description: Pull request number to verify
        required: true
        type: string
      target_url:
        description: Optional deployed URL
        required: false
        type: string
      instructions:
        description: Optional testing instructions
        required: false
        type: string
  schedule:
    - cron: '7,27,47 * * * *'

permissions:
  contents: read
  deployments: read
  issues: write
  pull-requests: write
  statuses: write

jobs:
  human-qa:
    if: vars.RENTAHUMAN_HUMAN_QA_ENABLED == 'true' &&
      (github.event_name != 'pull_request_target' || github.event.label.name == 'human-qa')
    runs-on: ubuntu-latest
    steps:
      # Always run trusted default-branch code for pull_request_target.
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.event.repository.default_branch }}
          persist-credentials: false

      # Call the create/poll implementation from the trusted checkout.
      - run: bunx tsx scripts/ci/human-qa.ts request
        if: github.event_name != 'schedule'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          RENTAHUMAN_API_KEY: ${{ secrets.RENTAHUMAN_API_KEY }}
          HUMAN_QA_PR_NUMBER: ${{ github.event.pull_request.number || inputs.pr_number }}
          HUMAN_QA_TARGET_URL: ${{ inputs.target_url }}
          HUMAN_QA_INSTRUCTIONS: ${{ inputs.instructions }}
      - run: bunx tsx scripts/ci/human-qa.ts poll
        if: github.event_name == 'schedule'
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          RENTAHUMAN_API_KEY: ${{ secrets.RENTAHUMAN_API_KEY }}

Add the human-qa label after the preview deployment is ready. You can also use workflow_dispatch with a PR number and an explicit target URL when the deployment is hosted outside GitHub. Scheduled polling should update existing tracked comments only; it must never create a new paid template.

Related API reference

See the complete QA runs REST API reference for template updates, run detail, escalation resolution, and wallet behavior.