Sign Google Cloud Storage URLs with Google Compute Engine default service account
I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:
import google.auth
import google.auth.iam
from google.auth.transport.requests import Request as gRequest
creds, _ = google.auth.default(request=gRequest(), scopes=[
'https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.read_write',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/firebase',
'https://www.googleapis.com/auth/compute.readonly',
'https://www.googleapis.com/auth/userinfo.email',
])
## creds is a google.auth.compute_engine.credentials.Credentials
## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
signer = google.auth.iam.Signer(
gRequest(), credentials=creds,
service_account_email=creds.service_account_email)
signer.sign('stuff')
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<TRUNCATED - my code>
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
response = self._make_signing_request(message)
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
response.data))
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'
Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?
python google-cloud-storage google-compute-engine google-iam
add a comment |
I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:
import google.auth
import google.auth.iam
from google.auth.transport.requests import Request as gRequest
creds, _ = google.auth.default(request=gRequest(), scopes=[
'https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.read_write',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/firebase',
'https://www.googleapis.com/auth/compute.readonly',
'https://www.googleapis.com/auth/userinfo.email',
])
## creds is a google.auth.compute_engine.credentials.Credentials
## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
signer = google.auth.iam.Signer(
gRequest(), credentials=creds,
service_account_email=creds.service_account_email)
signer.sign('stuff')
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<TRUNCATED - my code>
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
response = self._make_signing_request(message)
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
response.data))
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'
Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?
python google-cloud-storage google-compute-engine google-iam
add a comment |
I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:
import google.auth
import google.auth.iam
from google.auth.transport.requests import Request as gRequest
creds, _ = google.auth.default(request=gRequest(), scopes=[
'https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.read_write',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/firebase',
'https://www.googleapis.com/auth/compute.readonly',
'https://www.googleapis.com/auth/userinfo.email',
])
## creds is a google.auth.compute_engine.credentials.Credentials
## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
signer = google.auth.iam.Signer(
gRequest(), credentials=creds,
service_account_email=creds.service_account_email)
signer.sign('stuff')
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<TRUNCATED - my code>
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
response = self._make_signing_request(message)
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
response.data))
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'
Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?
python google-cloud-storage google-compute-engine google-iam
I'm trying to sign GCS URLs with the GCE default service account. I gave the compute default service account the necessary "Service Account Token Creator" role. When I try to sign a url in the following Python code, I get an error:
import google.auth
import google.auth.iam
from google.auth.transport.requests import Request as gRequest
creds, _ = google.auth.default(request=gRequest(), scopes=[
'https://www.googleapis.com/auth/cloud-platform',
'https://www.googleapis.com/auth/devstorage.read_write',
'https://www.googleapis.com/auth/logging.write',
'https://www.googleapis.com/auth/firebase',
'https://www.googleapis.com/auth/compute.readonly',
'https://www.googleapis.com/auth/userinfo.email',
])
## creds is a google.auth.compute_engine.credentials.Credentials
## creds.service_account_email is '<project-id>-compute@developer.gserviceaccount.com'
signer = google.auth.iam.Signer(
gRequest(), credentials=creds,
service_account_email=creds.service_account_email)
signer.sign('stuff')
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
<TRUNCATED - my code>
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 101, in sign
response = self._make_signing_request(message)
File "/usr/local/lib/python3.6/dist-packages/google/auth/iam.py", line 85, in _make_signing_request
response.data))
google.auth.exceptions.TransportError: Error calling the IAM signBytes API: b'{n "error": {n "code": 400,
n "message": "Invalid service account email (default).",n "status": "INVALID_ARGUMENT"n }n}n'
Is using the GCE default SA not allowed? Are there other default SAs that aren't allowed (specifically the Google App Engine Flexible SA)?
python google-cloud-storage google-compute-engine google-iam
python google-cloud-storage google-compute-engine google-iam
edited Jan 19 at 21:47
Doug Stevenson
75.3k988108
75.3k988108
asked Jan 19 at 21:08
Scott CrunkletonScott Crunkleton
538517
538517
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.
[UPDATE 01/19/2019]
When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:
auth_url = "https://www.googleapis.com/oauth2/v4/token"
headers = {}
request = google.auth.transport.requests.Request()
creds.before_request(request, "POST", auth_url, headers)
[END UPDATE]
Grant your service account the role roles/iam.serviceAccounts.signBlob
gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob
Now this service account can sign data using the private key.
Now grant this service account the role roles/iam.serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator
Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.
In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out thatcreds.service_account_email
returns'default'
until it fetches that info.
– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54271402%2fsign-google-cloud-storage-urls-with-google-compute-engine-default-service-accoun%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.
[UPDATE 01/19/2019]
When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:
auth_url = "https://www.googleapis.com/oauth2/v4/token"
headers = {}
request = google.auth.transport.requests.Request()
creds.before_request(request, "POST", auth_url, headers)
[END UPDATE]
Grant your service account the role roles/iam.serviceAccounts.signBlob
gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob
Now this service account can sign data using the private key.
Now grant this service account the role roles/iam.serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator
Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.
In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out thatcreds.service_account_email
returns'default'
until it fetches that info.
– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
add a comment |
You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.
[UPDATE 01/19/2019]
When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:
auth_url = "https://www.googleapis.com/oauth2/v4/token"
headers = {}
request = google.auth.transport.requests.Request()
creds.before_request(request, "POST", auth_url, headers)
[END UPDATE]
Grant your service account the role roles/iam.serviceAccounts.signBlob
gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob
Now this service account can sign data using the private key.
Now grant this service account the role roles/iam.serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator
Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.
In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out thatcreds.service_account_email
returns'default'
until it fetches that info.
– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
add a comment |
You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.
[UPDATE 01/19/2019]
When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:
auth_url = "https://www.googleapis.com/oauth2/v4/token"
headers = {}
request = google.auth.transport.requests.Request()
creds.before_request(request, "POST", auth_url, headers)
[END UPDATE]
Grant your service account the role roles/iam.serviceAccounts.signBlob
gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob
Now this service account can sign data using the private key.
Now grant this service account the role roles/iam.serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator
Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.
In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).
You need two roles. One of which you cannot grant in the Google Cloud Console to your default GCE service account. Make note of your service account email address that you want to use.
[UPDATE 01/19/2019]
When you create credentials, they are not initialized until required (e.g. no access token is requested). To preinitialize the credentials:
auth_url = "https://www.googleapis.com/oauth2/v4/token"
headers = {}
request = google.auth.transport.requests.Request()
creds.before_request(request, "POST", auth_url, headers)
[END UPDATE]
Grant your service account the role roles/iam.serviceAccounts.signBlob
gcloud projects add-iam-policy-binding <project-id> --member=serviceAccount:<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccounts.signBlob
Now this service account can sign data using the private key.
Now grant this service account the role roles/iam.serviceAccountTokenCreator
gcloud iam service-accounts add-iam-policy-binding <project-id>-compute@developer.gserviceaccount.com --member=<project-id>-compute@developer.gserviceaccount.com --role=roles/iam.serviceAccountTokenCreator
Now this service account can use this service account to create tokens. In the command you are giving the first service_account_email the priviledge to use the second service_account_email. Think of this as delegation. Notice that one role is at the project level and the other role is assigned to the service account itself.
In my code, I actually create a new service account and use that service account's email address for signing. I grant permissions to the new service account (with the first command) and I grant permission to use the new service account with my credentials (the second command).
edited Jan 20 at 1:47
answered Jan 19 at 23:30
John HanleyJohn Hanley
15.6k2629
15.6k2629
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out thatcreds.service_account_email
returns'default'
until it fetches that info.
– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
add a comment |
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out thatcreds.service_account_email
returns'default'
until it fetches that info.
– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that
creds.service_account_email
returns 'default'
until it fetches that info.– Scott Crunkleton
Jan 20 at 1:24
I gave the compute default service account the necessary "Service Account Token Creator" role. I think it might actually be due to trying to use the credentials object before it has a chance to fetch the service account info from the metadata server. Turns out that
creds.service_account_email
returns 'default'
until it fetches that info.– Scott Crunkleton
Jan 20 at 1:24
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
@ScottCrunkleton - I am not sure what you are trying to tell me.
– John Hanley
Jan 20 at 1:27
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
I updated my answer with the solution to your service account email address problem.
– John Hanley
Jan 20 at 1:47
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54271402%2fsign-google-cloud-storage-urls-with-google-compute-engine-default-service-accoun%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown