Handle authentication for Cloud Function client using Google Service Account in Golang

Note: I’ve published an open-source library go-cloudfunction-auth for this purpose. If you run into the same issue, you can just go ahead with the library (see the sample here). This post will focus on how I investigate and resolve the problem.

A few weeks ago, I needed to integrate my backend with another service deployed on Google Cloud Function. My backend and the new service are both backed by Google Cloud Platform. Originally, I assumed that authenticating to Cloud Function should be straight forward using my google service accounts.

This assumption was, at least, correct with Javascript. google-auth-libary works pretty well to support this case. The JS code below works perfectly:

const {GoogleAuth} = require('google-auth-library');

const targetAudience = "cloud-function-url"

async function run() {
	const auth = new GoogleAuth();

	const client = await auth.getIdTokenClient(targetAudience);
	const res = await client.request({ url });
	console.info(res.data);
}

It was not the same on Golang!

Google also has OAuth2 library in Golang, which contains sub package google to support authentication for google’s services. After reading the docs for a while, I came up with the below code. But it didn’t work…

import	"golang.org/x/oauth2/google"
func getToken() (err error) {
    scope := "https://www.googleapis.com/auth/cloud-platform"
    client, err := google.DefaultClient(context.Background(), scope)
    if err != nil {
    	return
    }
    res, err := client.Get("cloud-function-url")
    if err != nil {
	    return
    }
    fmt.Println(res)
    return
}

Compared to the Nodejs code, the code above was missing targetAudience param while it could be a mandatory parameter. Diving deeper into the library, I ended with this code.

baseUrl := "your-cloudfunction-baseurl"
ctx := context.Background()
targetAudience := baseUrl
credentials, err := google.FindDefaultCredentials(ctx)
if err != nil {
	fmt.Printf("cannot get credentials: %v", err)
	os.Exit(1)
}

tokenSrc, err := google.JWTAccessTokenSourceFromJSON(credentials.JSON, targetAudience)
if err != nil {
	fmt.Printf("cannot create jwt source: %v", err)
	os.Exit(1)
}

client := oauth2.NewClient(context.Background(), tokenSrc)
if err != nil {
	return
}
res, err := client.Get(baseUrl + "sub-url")
if err != nil {
	return
}

It still didn’t work!

After double checking to make sure the service account was set up correctly in my local environment, I knew it’s time to dive deeper into the OAuth2 code!

Investigation

Since the Nodejs code worked, we started to compare the JWT tokens generated by both sides. Here was the structure of the JWT token from GCP which allow us to authenticate the Cloud Function call:

{
 alg: "RS256",
 kid: "...",
 typ: "JWT"
}.
{
 aud: "<my cloud function base url>",
 azp: "<my service account email>",
 email: "<my service account email>",
 email_verified: true,
 exp: <time>,
 iat: <time>,
 iss: "https://accounts.google.com",
 sub: "<a string of digits>"
}.
[signature]

And here was the JWT I received from my Golang code:

	{
 alg: "RS256",
 kid: "...",
 typ: "JWT"
}.
{
 iss: "<my service account email>",
 aud: "<my cloud function base url>",
 exp: <time>,
 iat: <time>,
 sub: "<my service account email>"
}.
[signature]

The structures differed a lot. Could the invalid login be due to the invalid structure of the JWT. We started to implement a modified version of google’s library to change the JWT structure following my Nodejs output.

Unfortunately, it still didn’t work after we made the output structures matched. The issue was more complicated than we thought. We decided to dive deeper into Google’s OAuth2 protocol.

The auth flow has two main steps as the diagram below:

There are two different JWTs generated in this flow. The first one is created and signed by the client code, whereas the second one is produced by Google’s servers. Google’s returned JWT should be the final one and attached to HTTP clients’ request headers. The Nodejs library does similarly.

When we added a breakpoint to travel along with its execution flow, we could see it generated a signed JWT, called to Google, and captured the responded JWT for later requests.

The Golang library does not. It produces a JWT and uses this value directly. That flow may work for other Google’s products, where Google API accepts the token generated from our service accounts. Our cloud function, on the other hand, is a custom API and requires a signed JWT from Google.

At this point, we figured out two problems with the Golang library:

• It does not follow Google’s OAuth2 flow. While this implementation may work for other services, it does not work with Cloud Function.
• Structure of output JWTs of Golang library does not match with the first JWT generated by Nodejs library. Custom code is required for this purpose.

Solution

Now that we identified the cause, our custom code worked. Below is a summary of what we did:

• Generate and sign a JWT locally. The token should include the audience field, which set to our cloud function’s base URL.
• Send this JWT to Google’s follow their spec, detach returned token from the response.
• Attach this final token to all HTTP clients’ headers

Here are some main parts from our code:

const GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token"

func JWTAccessTokenSourceFromJSON(jsonKey []byte, audience string) (oauth2.TokenSource, error) {
	cfg, err := google.JWTConfigFromJSON(jsonKey)
	if err != nil {
		return nil, fmt.Errorf("google: could not parse JSON key: %v", err)
	}
	pk, err := internal.ParseKey(cfg.PrivateKey)
	if err != nil {
		return nil, fmt.Errorf("google: could not parse key: %v", err)
	}
	ts := &jwtAccessTokenSource{
		email:    cfg.Email,
		audience: audience,
		pk:       pk,
		pkID:     cfg.PrivateKeyID,
	}
	tok, err := ts.Token()
	if err != nil {
		return nil, err
	}
	return oauth2.ReuseTokenSource(tok, ts), nil
}

type TokenResponse struct {
	IdToken string `json:"id_token"`
}

func Authenticate(tokenSource oauth2.TokenSource) (token oauth2.Token, err error) {
	jwt, err := tokenSource.Token()
	if err != nil {
		return
	}

	client := &http.Client{Timeout: time.Second * 10}
	payload := strings.NewReader("grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=" + jwt.AccessToken)
	req, _ := http.NewRequest("POST", GOOGLE_TOKEN_URL, payload)
	req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
	res, err := client.Do(req)
	if err != nil {
		return
	}
	defer res.Body.Close()
	body, err := ioutil.ReadAll(res.Body)
	if err != nil {
		return
	}
	tokenRes := &TokenResponse{}
	err = json.Unmarshal(body, tokenRes)
	if err != nil {
		fmt.Println(err.Error())
	}
	token = oauth2.Token{
		AccessToken: tokenRes.IdToken,
	}
	return
}

func NewClient(jwtSource oauth2.TokenSource) *http.Client {
	token, err := Authenticate(jwtSource)
	if err != nil {
		fmt.Printf("cannot authenticate with google: %v", err)
		os.Exit(1)
	}

	return &http.Client{
		Transport: &oauth2.Transport{
			Base: http.DefaultClient.Transport,
			Source: &googleTokenSource{
				GoogleToken: &token,
			},
		},
	}
}

Conclusion

This part took us much longer than we planned. Since our other integration with Nodejs run pretty smoothly, we thought it also would be straightforward with Golang. We also didn’t find any good documents or conversations on the internet.

On the positive side, diving deeper into this issue gain us more understanding about JWT generation and OAuth2 flow. We published our code as an open-source library go-cloudfunction-auth. Hope this article and the library will help other people if they run into a similar problem.