Using CodeCommit and GitHub Credential Helpers

I have git repos on both GitHub and AWS CodeCommit, but I found CodeCommit's HTTPS credential management to be a bit problematic. CodeCommit's credential helper does not follow the typical name/password pattern, and the default git credential helpers installed for both Windows and OSX do not naturally play nice side-by-side with CodeCommit.

1. I followed Amazon's documentation for setting up CodeCommit's credential helper and overwrote my GitHub-compliant credential helper configuration.
2. I tried to configure them side-by-side with CodeCommit's helper namespaced to the CodeCommit HTTPS domain. I learned about git credential helper configuration settings figuring out the earlier problem of getting CodeCommit to work with EC2 Role Credentials. So I felt really smart, and I was proud of myself for figuring out domain scoping of credential helpers for a few minutes -- until this stopped working because credential helpers are executed in a cascading chain.
3. Finally, I configured both helpers by HTTPS URL scope, so they play nicely side-by-side.

Yay.

Before CodeCommit

I started with the default Git configurations, but no CodeCommit. My ~/.gitconfig files contained the following credential helper configurations:

Mac OSX:

[credential]
    helper = osxkeychain

Windows:

[credential]
    helper = wincred

AWS Recommended CodeCommit Install

The AWS documentation for installing the CodeCommit credential helper sets CodeCommit as your user default credential helper ("global" in Git config terminology). For example, on MacOS, Amazon recommends the following commands to set the configuration:

git config --global credential.helper '!aws --profile CodeCommitProfile codecommit credential-helper $@'
git config --global credential.UseHttpPath true

This overwrites the configuration previously set up for GitHub, resulting in the following setting in the .gitconfig file:

[credential]
    helper = !aws --profile CodeCommitProfile codecommit credential-helper $@
    UseHttpPath = true

Great for CodeCommit, not good for GitHub or Bitbucket.

Credential Helpers by HTTPS Domain

Git allows credential helpers to be configured for individual domains, and this will allows us to install different helpers side-by-side. What we want is a .gitconfig that looks something like this (MacOS):

[credential]
    helper = osxkeychain

[credential "https://git-codecommit.us-east-1.amazonaws.com"]
    helper = !aws --profile CodeCommitProfile codecommit credential-helper $@
    UseHttpPath = true

In this configuration, the default credential helper continues to be the osxkeychain, which works for GitHub. But if the remote URL matches AWS CodeCommit, it uses the CodeCommit credential helper. This is closer to what I want.

One minor issue might be the region-specific URLs. Since CodeCommit is only available in the us-east-1 region at the moment, there is only one URL to configure. You would eventually need a different setting for each CodeCommit endpoint you use.

Similar settings for Windows look like this:

[credential]
    helper = wincred

[credential "https://git-codecommit.us-east-1.amazonaws.com"]
    helper = !'C:\\Users\\somedude\\AppData\\Roaming\\GitCredStore\\git-credential-AWSSV4.exe' --profile=CodeCommitProfile
    UseHttpPath = true

Wait! We're Not Done Yet...

Scoping the CodeCommit credential helper doesn't really fix things. It just looks like it works for long enough to trick you into declaring victory, then it will be broken when you come back later. The root problem is that CodeCommit's credential helper generates temporary passwords with a 15-minute lifetime, and these can get cached by the default git credential helpers on both Windows and Mac OSX. So you fetch from or push to CodeCommit successfully, but 15 minutes later it stops working and starts giving you errors:

fatal: unable to access 'https://git-codecommit.us-east-1.amazonaws.com/v1/repos/test-repo/': The requested URL returned error: 403

This happened to me because I had set up the AWS CodeCommit credential helper only for the https://git-codecommit.us-east-1.amazonaws.com domain, but kept the global wincred/osxkeychain helpers. In this configuration, the helpers are actually executed in sequence, so the temporary password was permanently stored and retrieved.

The git credential documentation kind of describes this:

If there are multiple instances of the credential.helper configuration variable, each helper will be tried in turn, and may provide a username, password, or nothing. Once Git has acquired both a username and a password, no more helpers will be tried.

Although that doesn't seem to quite explain why the credentials returned by the CodeCommit helper both work AND are cached. My observation suggests that they are run in sequence regardless of what they return, although I do not understand exactly very well.

So there are two separate configuration issues for side-by-side credential helpers:

1. How do we use the AWS CodeCommit helper for CodeCommit repos, but the default credential helper for GitHub and other hosts?
2. How do we NOT cache the temporary creds generated by the AWS CodeCommit helper?

The CodeCommit troublehooting FAQ describes this problem and one potential solution for Mac OSX via Keychain settings. But I also experienced problems on Windows, and was looking for more general solutions.

Scoping All Credential Helpers

Since I mostly just use GitHub and CodeCommit, the easy solution was to add HTTPS domain scoping to both credential helpers. I used the following commands (Windows):

git config --global --remove-section credential
git config --global credential.https://github.com.helper wincred
git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.helper !"'C:\Users\james\AppData\Roaming\GitCredStore\git-credential-AWSSV4.exe' --profile=CodeCommitProfile"
git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.UseHttpPath true

or for the Mac:

git config --global --remove-section credential
git config --global credential.https://github.com.helper osxkeychain
git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.helper '!aws --profile CodeCommitProfile codecommit credential-helper $@'
git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.UseHttpPath true

This results in a .gitconfig file that looks like this (Windows):

[credential "https://github.com"]
helper = wincred

[credential "https://git-codecommit.us-east-1.amazonaws.com"]
helper = !'C:\\Users\\james\\AppData\\Roaming\\GitCredStore\\git-credential-AWSSV4.exe' --profile=CodeCommitProfile
UseHttpPath = true

With this configuration, I can switch back and forth between repos without issue. However, if I add completely new hosts, I would have to repeat the wincred configuration for that host.

Troubleshooting Cached Credentials

If you are unsure how your helpers are behaving, you can look in your operating system's credential management tool to check the dates and scope of cached credentials. I was surprised to find my CodeCommit credentials there at all, of course, I though the credential helper was covering it. But you can tell when they were cached and for which repo URL.

• Windows: Go to Control Panel -> Credential Manager -> Windows Credentials. Cached git credentials are listed under Generic Credentials.
• OSX: Go to Launchpad -> Other -> Keychain Access

You can safely remove these entries, the passwords expire after 15 minutes anyways.

Making the Configuration Changes

You're supposed to make git configuration changes via the git config utility, and you might do this as follows (MacOS):

git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.helper '!aws --profile CodeCommitProfile codecommit credential-helper $@'
git config --global credential.https://git-codecommit.us-east-1.amazonaws.com.UseHttpPath true

But after figuring out what these commands do to the .gitconfig file, I found it easier to just make the edits to ~/.gitconfig myself.

In fact, just go ahead and back up your .gitconfig file now, even if you don't make any changes.

Repo Config?

One possible solution would be to define these settings at the repo-level, and not touch the global settings at all. I have not figured out how to do that yet, it seems that the generic global helper runs regardless of the scoped configuration I put in the repo config. If anyone has done this, please let me know how you did it.