Linting pull requests

A couple weeks ago on Twitter, I joked about adding a way to bypass Cozy’s Pull Request linter by including #YOLO in the pull request description. It spawned an interesting discussion and a few people asked for more details about how the linter works.

Some devs at @CozyCo said our new PR linter was too strict, so I made it less strict. #YOLO pic.twitter.com/pjaqdc2ct2

— Rob Galanakis (@techartistsorg) March 18, 2016

We have some pretty thorough commit and pull request guidelines at Cozy, based on recognized best practices like these from Chris Beam. In other words, “update this” as a commit message is won’t do. After a few too many cut off commit messages and PRs missing links to the issues they were fixing, I finally got fed up enough to enforce these guidelines with a linting bot.

The entire linting project was just a few hours work*. When the web works, the productivity is incredible. When it doesn’t, you spend 3 hours debugging a JavaScript binding error. This was definitely a positive and productive experience.

Github has some excellent tutorials for building these sorts of bots. Check out Building a CI server. Your server gets some webhooks, and uses the Statuses API to update the PR. And, your bot can use the API to get whatever data you need. It doesn’t need a Git client or anything.

The jury is still out whether this is a good idea or not. I much prefer to have agreed standards that are continuously discussed and improved while being rigorously enforced, rather than have things happen haphazardly. There’s no room for poor quality to sneak in. Standards are clear. This strategy warrants its own post, but this post is just about the experience of building the linter.

I’ll dive into some example code, but that’s basically it. Like I said, simple**. The code below is very much based on the “Building a CI server” tutorial, so follow that if you need context.

First, set up a route to receive the webhook. This is mostly copy and paste from various tutorials:

      post '/github' do
        request.body.rewind
        payload_body = request.body.read
        return halt 500, "Signatures didn't match!" 
          unless webhook_verified?(payload_body)

        event = request.env['HTTP_X_GITHUB_EVENT']
        return unless ['pull_request', 'issue_comment'].include?( event )

        payload = JSON.parse(payload_body)
        return if payload['action'] == 'closed'

        if event == 'pull_request'
          repo = payload['pull_request']['base']['repo']['full_name']
          number = payload['pull_request']['number']
        else
          repo = payload['repository']['full_name']
          number = payload['issue']['number']
        end
        pr = @client.pull_request(repo, number.to_i)
        initialize_from_resource( pr ) # Sets some variables on this instance
        process_pull_request
        return 'Pull request processed!'
      end

The process_pull_request sets the appropriate status based on problems with the PR. You are probably familiar with the statuses if you’ve used any Github PR bot.

        def process_pull_request
          @pending = create_status('pending', nil)
          if has_problems?
            create_status(
              'failure',
              'Issues with commit messages or missing context link. See details.')
          else
            create_status(
              'success',
              'PR looks good! (good commit messages, has issue/context link)')
          end
        rescue
          if @pending
            create_status(
              'error',
              'PullRequestLinter errored while looking at this PR.')
          end
          raise
        end


        def create_status( status, description )
          org, rep = @repo.split('/')
          @client.create_status(
            @repo,
            @head_sha,
            status,
            context: 'prlinter',
            description: description,
            target_url: "#{WebhooksApp.base_url(request: request)}/prlinter/#{org}/#{rep}/#{@number}")
        end

I won’t delve into the has_problems? method, since it will just be based on your guidelines.
The only interesting thing about this setup is the target_url for the status.
If you navigate to it, you’ll get a breakdown of any problems with your PR.
We set up another route for that, and render out some markdown/HTML:

      get '/prlinter/:org/:repo/:pr_number' do
        initialize_from_params( params )
        ctx = context_problems
        commits = commit_message_problems
        md = [
          '### Pull Request Linter Report',
          "[#{@repo}##{@number}](#{@pr.html_url}) **#{@pr.title}**",
          '',
          "#{ctx.first ? '✖' : '✓'} Context: #{ctx.last}",
          '',
          "#{commits.first ? '✖' : '✓'} Commit Messages: #{commits.last}",
          "\n"
        ]
        if has_problems?
          md << 'Fix these problems before merging (if needed, you can force-refresh a PR Lint by commenting and deleting the comment).'
          md << 'If you need any help, see [Git Workflow](link to document).'
        else
          md << "You're all ready to merge! See [Merging Your Branch](link to document) for what to do next."
        end
        return render_md(md.join("\n"))
      end

That gives you a report like this:

That’s about it. I hope this helps you build your own Pull Request bots. Feel free to get in touch if you have any questions.

*: “a few hours” does not include deployment. Since this was part of an existing webhook app, deployment was not a problem.

**: So simple that, like any good CI service, it is missing tests…