Comments on: Emulating a long-running process (and a scheduler) in Google App Engine

By: William Vambenepe — PaaS as a satisfying and success-ready hobbyist plaform

Thu, 01 Oct 2009 05:47:12 +0000

[...] at the cost of too many longer-term limitations (now that the environment has gown a bit from the original limitations and provides scheduled and background jobs). Unlike Yahoo Pipes, for which the first impression is [...]

By: William Vambenepe’s blog » Blog Archive » Long-running processes on Google App Engine: it finally works

Fri, 13 Feb 2009 17:14:51 +0000

[...] Requests” that quickly result in your application being disabled. It’s all explained here and [...]

By: William Vambenepe’s blog » Blog Archive » Google App Engine is teasing me

Thu, 12 Feb 2009 07:21:46 +0000

[...] finally going to be able to use timeouts on outgoing HTTP requests. I care about this because my earlier attempt to emulate a long-running process in GAE was stymied by the GAE quota system, something I think I can work around if I can timeout a request [...]

By: William Vambenepe’s blog » Blog Archive » Now I know why GAE has been killing me

Wed, 17 Dec 2008 07:37:08 +0000

[...] on GAE today because the Google team just published a blog entry that explains why my attempts to create a long-running process in GAE resulted in my application being disabled for having too many “high CPU requests”, a [...]

By: Google App Engine limitations, and how to get around them | Digitalistic - Mashup or die trying

Tue, 16 Sep 2008 13:40:48 +0000

[...] about this in William Vambenepe’s very informative post Emulating a long-running process (and a scheduler) in Google App Engine. He has some tips on how to get around this limitation, even if it is not recommended since you [...]

By: Mashup.se » Google App Engine, del 4: Begränsningar och hur man tar sig runt dem

Fri, 05 Sep 2008 08:41:37 +0000

[...] om detta kan ni hitta på William Vambenepe’s mycket informativa blogginlägg Emulating a long-running process (and a scheduler) in Google App Engine. Han har några tips på hur man kan försöka ta sig förbi denna begränsning, även om det inte [...]

By: William Vambenepe

William Vambenepe — Thu, 28 Aug 2008 20:10:21 +0000

Yes. That’s the result of my attempt: there are some things you can use for a short period but nothing that’s functional in any practical way.

By: Andrew Bilyk

Andrew Bilyk — Thu, 28 Aug 2008 16:02:30 +0000

Thank you for answer. Just to be shure, I can’t implement described functionality with standard GAE Developers Tools. Am I right?

By: William Vambenepe

William Vambenepe — Thu, 28 Aug 2008 15:42:57 +0000

Sorry Andrew, I don’t know of a GAE-provided cron service. There are such services out there that you can use to invoke your GAE app once a while though.

By: Andrew Bilyk

Andrew Bilyk — Thu, 28 Aug 2008 14:54:50 +0000

Hello. Please help me with a next problem. I need to implement functionality of testing some urls (currently get response code only) and the problem is that I need to run this test each 10-15 seconds. Can I provide this with Google App Engine somehow? May be GAE has some services? I have read somewhere about Cron and Commet GAE services but have’t found any examples.
Thank you.

By: William Vambenepe

William Vambenepe — Wed, 25 Jun 2008 07:05:22 +0000

Yiqiang, Not sure I understand the question. If this is what you are asking, yes it is possible for an app to call itself. The problem is that once this is done the calling instance of the app has to hang and wait for the response to come back. It can't just call itself and let the old instance die. The old instance dies eventually (killed by GAE environment after approx 9 seconds) but it is marked as having used to much CPU (even though in fact it is a clock time metric, not a CPU time metric). And eventually your app gets killed for that. That's what I described in the earlier blog entry (http://stage.vambenepe.com/archives/207)

By: Yiqiang

Yiqiang — Wed, 25 Jun 2008 06:59:10 +0000

I am also trying to a long time running app. Is it possible to fetch the app by itself to make it running in a long time?

By: William Vambenepe’s blog » Blog Archive » Some breathing room for Google App Engine requests

Fri, 13 Jun 2008 08:16:04 +0000

[...] promised to Felix here is the code that shows how to give extra breathing room to Google App Engine (GAE) requests [...]

By: William Vambenepe

William Vambenepe — Thu, 12 Jun 2008 16:47:17 +0000

Yes and in fact I wrote another prototype that does just that. This time I was going after another use case. Rather than trying to simulate a long-running process, I was simply trying to solve the problem of having too many user requests fail because they take too long to execute. By using a redirect in the same manner you suggest, I allowed the request to spread its work over several invocations thus getting several 9 seconds slots (since this seems to be how much time GAE gives you per request right now). In theory it could go on for ever, but in practice, as I’ve reported in the blog post above, you get killed after a few minutes. But if you have an app that usually returns within one invocation but runs out of time say 10% of the time, this approach can make it a lot more robust and is potentially sustainable (you don’t encounter the wrath of the GAE resource cop if most of you requests complete fast enough). It’s a bit like giving kids w/ ADHD extra time for school exams, except this time you give requests with special needs extra CPU time to complete.

Since you inquired about this, I’ll post the code for that prototype when I get home tonight.

[UPDATED 2008/6/13: Done.]

By: felix

felix — Thu, 12 Jun 2008 14:25:54 +0000

I’ve been trying to tackle this problem myself! Your solution is quite interesting, I’m wondering if a modification to it might work – instead of using urlfetch to grab another one and letting the originals be killed, what if you used a simple location redirect, so you’d have to have your browser open the whole time, but you could watch your progress in the url bar and can easily stop the process by closing the tab. Hmm….

Also, you can delete a whole application? I can’t figure out how to do this, when I try to hit the delete button it pops a message that says it can’t delete the default version. I would very much love an easy way to delete an app, or at least delete it’s datastore.