Sign Google Cloud Storage URLs with Google Compute Engine default service account












0















I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:



import google.auth
import google.auth.iam
from google.auth.transport.requests import Request as gRequest

creds, _ = google.auth.default(request=gRequest(), scopes=[
'https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.read_write',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/firebase',
'https://www.googleapis.com/auth/compute.readonly',
'https://www.googleapis.com/auth/userinfo.email',
])
## creds is a google.auth.compute_engine.credentials.Credentials
## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
signer = google.auth.iam.Signer(
gRequest(), credentials=creds,
service_account_email=creds.service_account_email)

signer.sign('stuff')


Error:



Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<TRUNCATED - my code>
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
response = self._make_signing_request(message)
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
response.data))
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'


Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?










share|improve this question





























    0















    I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:



    import google.auth
    import google.auth.iam
    from google.auth.transport.requests import Request as gRequest

    creds, _ = google.auth.default(request=gRequest(), scopes=[
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/devstorage.read_write',
    'https://www.googleapis.com/auth/logging.write',
    'https://www.googleapis.com/auth/firebase',
    'https://www.googleapis.com/auth/compute.readonly',
    'https://www.googleapis.com/auth/userinfo.email',
    ])
    ## creds is a google.auth.compute_engine.credentials.Credentials
    ## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
    signer = google.auth.iam.Signer(
    gRequest(), credentials=creds,
    service_account_email=creds.service_account_email)

    signer.sign('stuff')


    Error:



    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    <TRUNCATED - my code>
    File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
    response = self._make_signing_request(message)
    File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
    response.data))
    google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
    n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'


    Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?










    share|improve this question



























      0












      0








      0








      I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:



      import google.auth
      import google.auth.iam
      from google.auth.transport.requests import Request as gRequest

      creds, _ = google.auth.default(request=gRequest(), scopes=[
      'https://www.googleapis.com/auth/cloud-platform',
      'https://www.googleapis.com/auth/devstorage.read_write',
      'https://www.googleapis.com/auth/logging.write',
      'https://www.googleapis.com/auth/firebase',
      'https://www.googleapis.com/auth/compute.readonly',
      'https://www.googleapis.com/auth/userinfo.email',
      ])
      ## creds is a google.auth.compute_engine.credentials.Credentials
      ## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
      signer = google.auth.iam.Signer(
      gRequest(), credentials=creds,
      service_account_email=creds.service_account_email)

      signer.sign('stuff')


      Error:



      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      <TRUNCATED - my code>
      File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
      response = self._make_signing_request(message)
      File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
      response.data))
      google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
      n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'


      Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?










      share|improve this question
















      I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:



      import google.auth
      import google.auth.iam
      from google.auth.transport.requests import Request as gRequest

      creds, _ = google.auth.default(request=gRequest(), scopes=[
      'https://www.googleapis.com/auth/cloud-platform',
      'https://www.googleapis.com/auth/devstorage.read_write',
      'https://www.googleapis.com/auth/logging.write',
      'https://www.googleapis.com/auth/firebase',
      'https://www.googleapis.com/auth/compute.readonly',
      'https://www.googleapis.com/auth/userinfo.email',
      ])
      ## creds is a google.auth.compute_engine.credentials.Credentials
      ## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
      signer = google.auth.iam.Signer(
      gRequest(), credentials=creds,
      service_account_email=creds.service_account_email)

      signer.sign('stuff')


      Error:



      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      <TRUNCATED - my code>
      File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
      response = self._make_signing_request(message)
      File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
      response.data))
      google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
      n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'


      Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?







      python google-cloud-storage google-compute-engine google-iam






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 19 at 21:47









      Doug Stevenson

      75.3k988108




      75.3k988108










      asked Jan 19 at 21:08









      Scott CrunkletonScott Crunkleton

      538517




      538517
























          1 Answer
          1






          active

          oldest

          votes


















          1














          You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.



          [UPDATE 01/19/2019]



          When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:



          auth_url = "https://www.googleapis.com/oauth2/v4/token"
          headers = {}
          request = google.auth.transport.requests.Request()
          creds.before_request(request, "POST", auth_url, headers)


          [END UPDATE]



          Grant your service account the role roles/iam.serviceAccounts.signBlob



          gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob


          Now this service account can sign data using the private key.



          Now grant this service account the role roles/iam.serviceAccountTokenCreator



          gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator


          Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.



          In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).






          share|improve this answer


























          • I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

            – Scott Crunkleton
            Jan 20 at 1:24











          • @ScottCrunkleton - I am not sure what you are trying to tell me.

            – John Hanley
            Jan 20 at 1:27











          • I updated my answer with the solution to your service account email address problem.

            – John Hanley
            Jan 20 at 1:47











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54271402%2fsign-google-cloud-storage-urls-with-google-compute-engine-default-service-accoun%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.



          [UPDATE 01/19/2019]



          When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:



          auth_url = "https://www.googleapis.com/oauth2/v4/token"
          headers = {}
          request = google.auth.transport.requests.Request()
          creds.before_request(request, "POST", auth_url, headers)


          [END UPDATE]



          Grant your service account the role roles/iam.serviceAccounts.signBlob



          gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob


          Now this service account can sign data using the private key.



          Now grant this service account the role roles/iam.serviceAccountTokenCreator



          gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator


          Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.



          In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).






          share|improve this answer


























          • I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

            – Scott Crunkleton
            Jan 20 at 1:24











          • @ScottCrunkleton - I am not sure what you are trying to tell me.

            – John Hanley
            Jan 20 at 1:27











          • I updated my answer with the solution to your service account email address problem.

            – John Hanley
            Jan 20 at 1:47
















          1














          You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.



          [UPDATE 01/19/2019]



          When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:



          auth_url = "https://www.googleapis.com/oauth2/v4/token"
          headers = {}
          request = google.auth.transport.requests.Request()
          creds.before_request(request, "POST", auth_url, headers)


          [END UPDATE]



          Grant your service account the role roles/iam.serviceAccounts.signBlob



          gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob


          Now this service account can sign data using the private key.



          Now grant this service account the role roles/iam.serviceAccountTokenCreator



          gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator


          Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.



          In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).






          share|improve this answer


























          • I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

            – Scott Crunkleton
            Jan 20 at 1:24











          • @ScottCrunkleton - I am not sure what you are trying to tell me.

            – John Hanley
            Jan 20 at 1:27











          • I updated my answer with the solution to your service account email address problem.

            – John Hanley
            Jan 20 at 1:47














          1












          1








          1







          You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.



          [UPDATE 01/19/2019]



          When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:



          auth_url = "https://www.googleapis.com/oauth2/v4/token"
          headers = {}
          request = google.auth.transport.requests.Request()
          creds.before_request(request, "POST", auth_url, headers)


          [END UPDATE]



          Grant your service account the role roles/iam.serviceAccounts.signBlob



          gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob


          Now this service account can sign data using the private key.



          Now grant this service account the role roles/iam.serviceAccountTokenCreator



          gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator


          Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.



          In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).






          share|improve this answer















          You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.



          [UPDATE 01/19/2019]



          When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:



          auth_url = "https://www.googleapis.com/oauth2/v4/token"
          headers = {}
          request = google.auth.transport.requests.Request()
          creds.before_request(request, "POST", auth_url, headers)


          [END UPDATE]



          Grant your service account the role roles/iam.serviceAccounts.signBlob



          gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob


          Now this service account can sign data using the private key.



          Now grant this service account the role roles/iam.serviceAccountTokenCreator



          gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator


          Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.



          In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jan 20 at 1:47

























          answered Jan 19 at 23:30









          John HanleyJohn Hanley

          15.6k2629




          15.6k2629













          • I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

            – Scott Crunkleton
            Jan 20 at 1:24











          • @ScottCrunkleton - I am not sure what you are trying to tell me.

            – John Hanley
            Jan 20 at 1:27











          • I updated my answer with the solution to your service account email address problem.

            – John Hanley
            Jan 20 at 1:47



















          • I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

            – Scott Crunkleton
            Jan 20 at 1:24











          • @ScottCrunkleton - I am not sure what you are trying to tell me.

            – John Hanley
            Jan 20 at 1:27











          • I updated my answer with the solution to your service account email address problem.

            – John Hanley
            Jan 20 at 1:47

















          I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

          – Scott Crunkleton
          Jan 20 at 1:24





          I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that creds.service_account_email returns 'default' until it fetches that info.

          – Scott Crunkleton
          Jan 20 at 1:24













          @ScottCrunkleton - I am not sure what you are trying to tell me.

          – John Hanley
          Jan 20 at 1:27





          @ScottCrunkleton - I am not sure what you are trying to tell me.

          – John Hanley
          Jan 20 at 1:27













          I updated my answer with the solution to your service account email address problem.

          – John Hanley
          Jan 20 at 1:47





          I updated my answer with the solution to your service account email address problem.

          – John Hanley
          Jan 20 at 1:47


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54271402%2fsign-google-cloud-storage-urls-with-google-compute-engine-default-service-accoun%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Liquibase includeAll doesn't find base path

          How to use setInterval in EJS file?

          Petrus Granier-Deferre