How to fix deferred tasks being sent to not initialized instance?
My Google AppEngine website is using deferred tasks to do a lot of extra work. Sometimes the amount of deferred tasks cause a new instance to be created. When this happens, all the tasks that are delegated to this new instance fail because they get sent there before the instance has had a chance to initialize. If enough instances are already started up before the tasks come in, no issues occur.
Here is an example of one of the failed tasks:
Permanent failure attempting to execute task (/.../google/appengine/ext/deferred/deferred.py:327)
Traceback (most recent call last):
File "/.../google/appengine/ext/deferred/deferred.py", line 318, in post
self.run_from_request()
File "/.../google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
run(self.request.body)
File "/.../google/appengine/ext/deferred/deferred.py", line 153, in run
raise PermanentTaskFailure(e)
PermanentTaskFailure: No module named django
Is there a way to tell the deferred system to wait until an instance is fully initialized before sending tasks to it? I'm losing hair over this problem.
python django google-app-engine deferred djangae
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
My Google AppEngine website is using deferred tasks to do a lot of extra work. Sometimes the amount of deferred tasks cause a new instance to be created. When this happens, all the tasks that are delegated to this new instance fail because they get sent there before the instance has had a chance to initialize. If enough instances are already started up before the tasks come in, no issues occur.
Here is an example of one of the failed tasks:
Permanent failure attempting to execute task (/.../google/appengine/ext/deferred/deferred.py:327)
Traceback (most recent call last):
File "/.../google/appengine/ext/deferred/deferred.py", line 318, in post
self.run_from_request()
File "/.../google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
run(self.request.body)
File "/.../google/appengine/ext/deferred/deferred.py", line 153, in run
raise PermanentTaskFailure(e)
PermanentTaskFailure: No module named django
Is there a way to tell the deferred system to wait until an instance is fully initialized before sending tasks to it? I'm losing hair over this problem.
python django google-app-engine deferred djangae
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04
add a comment |
My Google AppEngine website is using deferred tasks to do a lot of extra work. Sometimes the amount of deferred tasks cause a new instance to be created. When this happens, all the tasks that are delegated to this new instance fail because they get sent there before the instance has had a chance to initialize. If enough instances are already started up before the tasks come in, no issues occur.
Here is an example of one of the failed tasks:
Permanent failure attempting to execute task (/.../google/appengine/ext/deferred/deferred.py:327)
Traceback (most recent call last):
File "/.../google/appengine/ext/deferred/deferred.py", line 318, in post
self.run_from_request()
File "/.../google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
run(self.request.body)
File "/.../google/appengine/ext/deferred/deferred.py", line 153, in run
raise PermanentTaskFailure(e)
PermanentTaskFailure: No module named django
Is there a way to tell the deferred system to wait until an instance is fully initialized before sending tasks to it? I'm losing hair over this problem.
python django google-app-engine deferred djangae
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
My Google AppEngine website is using deferred tasks to do a lot of extra work. Sometimes the amount of deferred tasks cause a new instance to be created. When this happens, all the tasks that are delegated to this new instance fail because they get sent there before the instance has had a chance to initialize. If enough instances are already started up before the tasks come in, no issues occur.
Here is an example of one of the failed tasks:
Permanent failure attempting to execute task (/.../google/appengine/ext/deferred/deferred.py:327)
Traceback (most recent call last):
File "/.../google/appengine/ext/deferred/deferred.py", line 318, in post
self.run_from_request()
File "/.../google/appengine/ext/deferred/deferred.py", line 313, in run_from_request
run(self.request.body)
File "/.../google/appengine/ext/deferred/deferred.py", line 153, in run
raise PermanentTaskFailure(e)
PermanentTaskFailure: No module named django
Is there a way to tell the deferred system to wait until an instance is fully initialized before sending tasks to it? I'm losing hair over this problem.
python django google-app-engine deferred djangae
python django google-app-engine deferred djangae
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited Jan 18 at 19:03
Perry Goy
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
asked Jan 18 at 15:57
Perry GoyPerry Goy
32
32
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Perry Goy is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04
add a comment |
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04
add a comment |
1 Answer
1
active
oldest
votes
GAE warmup requests are designed to do exactly that:
Loading your app's code to a new instance can result in loading
requests. Loading requests can result in increased request latency
for your users, but you can avoid this latency using warmup
requests. Warmup requests load your app's code into a new instance
before any live requests reach that instance.
You just need to:
- enable the warmup requests
- register a handler for them
- add the handler code
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Perry Goy is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54257423%2fhow-to-fix-deferred-tasks-being-sent-to-not-initialized-instance%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
GAE warmup requests are designed to do exactly that:
Loading your app's code to a new instance can result in loading
requests. Loading requests can result in increased request latency
for your users, but you can avoid this latency using warmup
requests. Warmup requests load your app's code into a new instance
before any live requests reach that instance.
You just need to:
- enable the warmup requests
- register a handler for them
- add the handler code
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
add a comment |
GAE warmup requests are designed to do exactly that:
Loading your app's code to a new instance can result in loading
requests. Loading requests can result in increased request latency
for your users, but you can avoid this latency using warmup
requests. Warmup requests load your app's code into a new instance
before any live requests reach that instance.
You just need to:
- enable the warmup requests
- register a handler for them
- add the handler code
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
add a comment |
GAE warmup requests are designed to do exactly that:
Loading your app's code to a new instance can result in loading
requests. Loading requests can result in increased request latency
for your users, but you can avoid this latency using warmup
requests. Warmup requests load your app's code into a new instance
before any live requests reach that instance.
You just need to:
- enable the warmup requests
- register a handler for them
- add the handler code
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
GAE warmup requests are designed to do exactly that:
Loading your app's code to a new instance can result in loading
requests. Loading requests can result in increased request latency
for your users, but you can avoid this latency using warmup
requests. Warmup requests load your app's code into a new instance
before any live requests reach that instance.
You just need to:
- enable the warmup requests
- register a handler for them
- add the handler code
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
answered Jan 18 at 18:08
Dan CornilescuDan Cornilescu
28.3k113164
28.3k113164
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
add a comment |
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
Wow, thanks! This does exactly what we need it to.
– Perry Goy
Jan 18 at 19:10
add a comment |
Perry Goy is a new contributor. Be nice, and check out our Code of Conduct.
Perry Goy is a new contributor. Be nice, and check out our Code of Conduct.
Perry Goy is a new contributor. Be nice, and check out our Code of Conduct.
Perry Goy is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54257423%2fhow-to-fix-deferred-tasks-being-sent-to-not-initialized-instance%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
When you say that the instance is not fully initialized, do you mean that the instance itself is receiving tasks before it's up and running, or your code on the instance hasn't initialized yet?
– Dustin Ingram
Jan 18 at 16:44
I've edited the question to include an example of the errors i'm getting. It seems like the code tries to run before my Django app has actually booted up, since the first thing the app does is fix the python path.
– Perry Goy
Jan 18 at 19:04