TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Colorado. Besides tech, I enjoy landscape photography and share my work at ColoradoPhoto.gallery.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 241 other followers

  • Sleepless Nights…

    March 2010
    S M T W T F S
    « Feb   Apr »
     123456
    78910111213
    14151617181920
    21222324252627
    28293031  
  • Blog Stats

    • 857,849 hits

AppEngine Cold Starts Considered

Posted by David Chandler on March 26, 2010

I’ve been able to reduce my cold start time on AppEngine from an average of 8.1s to 2.5s, a 69% reduction. If you’re already familiar with the cold start issue, you can skip the next paragraph.

Cold starts seem to be the #1 complaint about AppEngine for Java. The root issue is that many developers coming to AppEngine are expecting it to work like the enterprise Java stacks they work on during their day jobs. AppEngine can indeed run an amazing variety of enterprise frameworks like Spring and JSF (see Will it play in AppEngine), but the reason Google can give it away for free to get started is because they’re not running dedicated servers just for your app. Instead, Google has done what few, if any, Web hosting companies have ever been able to do, which is to provide secure shared hosting for Java. If your app is not getting used at the moment, Google will swap out your JVM and fire it up again when a request comes in, thereby freeing up server memory for active apps. Unfortunately, for lightly used apps, this means AppEngine may have to spin up your JVM for every new user session. This includes initializing the servlet context and all frameworks your app may be using. Consequently, cold start times between 8-12s are not uncommon, and some larger stacks can’t even start within in the 30s request deadline.

Frankly, 10s isn’t a bad time to start up a Java stack. Most enterprise apps I’ve worked on take a minute or more. AppEngine doesn’t take that long because lots of AppEngine services are always running: Datastore, Memcache, etc. The problem is that due to the shared nature of the AppEngine platform, you have to pay this startup penalty very often. Lots of folks have asked Google to create a pay-for-warm-JVM option, which has recently been added to the AppEngine for Java roadmap. A nice way to do this might be set a minimum fee for billing-enabled apps that would guarantee you a JVM, but which you could credit toward actual resource usage.

For now, however, you can reduce your cold start time by rethinking (and refactoring) your app to work with AppEngine rather than trying to force your enterprise stack on the platform.

Let’s start with dependency injection. I love DI frameworks like Spring and Guice. But they’re not designed for shared hosting. They deliberately pre-load as much as possible at startup, which is the right thing to do on a dedicated server when startups are infrequent. But on AppEngine, it will cost you. Guice on AppEngine is configured via a servlet context listener. Unfortunately, this means that every servlet, including cron jobs and task invocations, trigger Guice initialization during a cold start, even though those servlets don’t need any Guice-provided objects. Worse, Guice eagerly loads all singletons in production mode, so all my gwt-dispatch ActionHandlers were getting loaded with every hourly cron job (which doesn’t even need the ActionHandlers). The solution was to replace Guice with gwt-dispatch’s basic LazyActionHandlerRegistry as described on the wiki. That saved me 5+ seconds.

Next, consider a fast-startup persistence framework like Objectify-appengine. JPA and JDO incur significant overhead to create a new PersistenceManager (by some reports, 2-3s). “So,” you say, “I’ll use a DI framework to create only one instance of the PM at startup,” and now you’re back to the previous paragraph. A better alternative for AppEngine is to use a lightweight persistence framework designed specifically for AppEngine. ObjectifyService.begin() takes only miliseconds, and IMHO is easier to use than JDO or JPA, although I appreciate that Google makes those APIs available for portability.

Bottom line: by eliminating DI frameworks and using Objectify for persistence, I’m seeing cold starts in 2.5s average. I am more than happy to pay this small penalty for the privilege of running my code for free on the world’s most scalable Java infrastructure. This approach also conserves community resources vs. running a useless cron job just to keep your app warm, which makes cold starts happen all that more often for the rest of us.

Of course, once my app traffic takes off or Google come out with a pay-for-JVM option, all this goes away and you can use dependency injection, cron jobs, etc. with abandon. But for now, the name of the game is, how small can you make your app? Personally, I love the challenge. I much prefer lightweight, plain old Java to layer upon layer of frameworks, anyway.

Advertisements

10 Responses to “AppEngine Cold Starts Considered”

  1. Hey David, I was able to knock a few seconds off Guice start up time by using the non-AOP version which does not create enhanced classes. To use the non-AOP jar I had to modify a few classes in GuiceServlet which were suddenly not visible. But that was just a 5 minute job – adding “public” to what ever method threw an error.

    I also deploy it in development mode which stops those singletons from being eagerly instantiated. All in all it starts up pretty quick now. Have you looked into “shrinking” your jars at all to just include the code actually used by your app?

  2. Dan Billings said

    Hey David-

    Some rainy day, could you post your GuiceLazyDispatch solution?

    I haven’t seen any conclusive examples of it, and you said you had great success, so…

    Thanks and keep up the good work!

  3. ГОСТ said

    MyFaces+Objectify +PrimeFaces + some libs like guava, slf4j, jdom and so on = 8s cold start

  4. krkr said

    Hi,

    I’ve two interesting links about appengine cold starts.

    http://code.google.com/p/slim3/ instead of objectify.

    and : http://thoughts.inphina.com/2010/10/09/managing-cold-start-on-google-app-engine/

    • raymond said

      Hey K,

      why do you say that slim3 instead of objectify helps you with the cold starts? Do you have any hints, real life experience about it?
      I’m about to develop a new application, and I’m trying to do the best decisions about what framework to use, and how to design the app.
      All the documentation points to use objectify. But I’m curious about slim3.

      Thanks

      • Krkr said

        Hi Raymond,

        I have not real returns from real life. I can not find time to finish a personal project with my work. Maybe soon …

  5. krkr said

    Sorry but I am disappointed in the second url.

    http://arcbees.wordpress.com/2010/09/29/how-i-made-my-gwtappengine-application-appear-to-load-quicker/

    Oops.

  6. Ken Xu said

    Hi,

    I used Guice one year ago, but not on GAE. Guice itself doesn’t use servlet context listener unless put together with Guice-servlet. Instead of injecting business delegates to servlet, we can call Guice get(class) in servlet. So only the business delegate being gotten is initiated. This means it is better to have multiple small business delegates than a big one. If there is only one business delegates to handle all the work, when it is initiated, all the underneath objects will be initiated.

    • Ken Xu said

      Sorry I have to make a correction. By default in production, singletons are eagerly initiated when calls Guice.createInjector(). We can switch to lazy initialization by change stage to “DEVELOPMENT” though.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: