Experience Report: GitHub Flow and Pivotal Web Services

A few of us on the Tracker team started to explore different development processes with Tracker since we weren’t really satisfied with how our current development and release processes were aligned.

Our process used to look something like this:

• Step 1: Developer starts a story.
• Step 2: Developer makes a bunch of commits to master.
• Step 3: Developer delivers a story.
• Step 4: Product Manager or Tester rejects the story.

At around Step 3, the team wants to deploy a set of bug fixes, but they can’t deploy from master since previous code for the partially completed story has not been accepted. So we wait for the story to get accepted, then wait some more…when another commit sneaks into master and the problem repeats itself.

So, there are a few issues:

• Code for a future feature is not ready to release but is tangled up with features that do need to go out;
• There was no real lever for the Product team to say “ship it”;
• A limited set of test environments make it difficult to know if a story is fully or only partially deployed; and
• Feature B would break Feature A’s functionality, making accepting Feature A impossible.

The lever: GitHub Flow

We realized that the Product team really has no control over when to deliver value to the customer because all commits for past and future features were intertwined in master. We explored GitHub Flow and found that pull requests (PR) can be a great lever for the PM/Tester to decide if a feature is ready to go.

Pull requests gate the code before it can be merged into master with a big green Merge button. There are also visual indicators on a PR, identifying that the code passed CI build checks or that there has been a code review.

For us, the Merge button is the lever that says not only do all the engineering checks pass, but so do the requirements from Product. So for every story, we create a PR and let the PM or Tester merge in the code.

The PR is also a good indicator on whether or not you should do acceptance on a story. If a code review has not happened or a build check has not passed, the story might not be in a good state to do acceptance.

But in order for this approach to work for us, the old way of deploying to dedicated test environments like alpha, demo, and staging would have to change as well. Since each story has its own PR, the code is isolated and cannot be tested in one environment.

Isolated test environments: Enter Pivotal Web Services

For a while, we thought, “Wouldn’t it be nice to test each story in an isolated test environment?” With Pivotal Web Services (PWS), it’s extremely easy to push our app to its own environment with its own unique URL.

As a part of our Concourse CI pipeline, we automatically deploy our code to PWS. PWS then spins up the app (and related services like P-MySQL, Redis, etc.). For each story, we now have a stable test environment and a bit of test data to do acceptance and exploratory testing.

With isolated environments, there is no longer any confusion about where a commit is—we now know exactly where a story lives.

So now our process looks like this:

• Step 1: Developer starts a story.
• Step 2: Developer makes a branch, which looks like <projectname>-<storyid>-<short-title>.
• Step 3: Developer makes a bunch of commits to the branch.
• Step 4: Each commit is deployed to its own environment by CI https://<projectname>-<storyid>-<short-title>.cfapps.io.
• Step 5: When the feature is complete, the Developer creates a pull request and delivers the Tracker story with the PR link embedded in the story.
• Step 6: PM and Testers try out the feature in the isolated test environment; if the story is complete, either the PM or Tester will merge the PR on GitHub.

After trying out this experiment to our process, we’ve noticed a few benefits:

• Other stories’ code blocking a release is no longer an issue.
• Our feedback loops have gotten much tighter since we no longer have to ask, “Is the code on environment X yet?” or “Did all the development tests pass?” before testing a story.
• The code is isolated so we don’t have the problem of Feature A breaking Feature B.
• PWS really scales as the development team is cranking; we’ve easily had 50 test environments spun up without any issues.
• PR make sure that all engineering checks are completed before a story is accepted.

We’re beginning to build out features around pull requests and commit statuses in Pivotal Tracker. If you’d like to help shape these features, please reach out to us via email.