AppEngine Cold Starts Considered

I’ve been able to reduce my cold start time on AppEngine from an average of 8.1s to 2.5s, a 69% reduction. If you’re already familiar with the cold start issue, you can skip the next paragraph.

Cold starts seem to be the #1 complaint about AppEngine for Java. The root issue is that many developers coming to AppEngine are expecting it to work like the enterprise Java stacks they work on during their day jobs. AppEngine can indeed run an amazing variety of enterprise frameworks like Spring and JSF (see Will it play in AppEngine), but the reason Google can give it away for free to get started is because they’re not running dedicated servers just for your app. Instead, Google has done what few, if any, Web hosting companies have ever been able to do, which is to provide secure shared hosting for Java. If your app is not getting used at the moment, Google will swap out your JVM and fire it up again when a request comes in, thereby freeing up server memory for active apps. Unfortunately, for lightly used apps, this means AppEngine may have to spin up your JVM for every new user session. This includes initializing the servlet context and all frameworks your app may be using. Consequently, cold start times between 8-12s are not uncommon, and some larger stacks can’t even start within in the 30s request deadline.

Frankly, 10s isn’t a bad time to start up a Java stack. Most enterprise apps I’ve worked on take a minute or more. AppEngine doesn’t take that long because lots of AppEngine services are always running: Datastore, Memcache, etc. The problem is that due to the shared nature of the AppEngine platform, you have to pay this startup penalty very often. Lots of folks have asked Google to create a pay-for-warm-JVM option, which has recently been added to the AppEngine for Java roadmap. A nice way to do this might be set a minimum fee for billing-enabled apps that would guarantee you a JVM, but which you could credit toward actual resource usage.

For now, however, you can reduce your cold start time by rethinking (and refactoring) your app to work with AppEngine rather than trying to force your enterprise stack on the platform.

Let’s start with dependency injection. I love DI frameworks like Spring and Guice. But they’re not designed for shared hosting. They deliberately pre-load as much as possible at startup, which is the right thing to do on a dedicated server when startups are infrequent. But on AppEngine, it will cost you. Guice on AppEngine is configured via a servlet context listener. Unfortunately, this means that every servlet, including cron jobs and task invocations, trigger Guice initialization during a cold start, even though those servlets don’t need any Guice-provided objects. Worse, Guice eagerly loads all singletons in production mode, so all my gwt-dispatch ActionHandlers were getting loaded with every hourly cron job (which doesn’t even need the ActionHandlers). The solution was to replace Guice with gwt-dispatch’s basic LazyActionHandlerRegistry as described on the wiki. That saved me 5+ seconds.

Next, consider a fast-startup persistence framework like Objectify-appengine. JPA and JDO incur significant overhead to create a new PersistenceManager (by some reports, 2-3s). “So,” you say, “I’ll use a DI framework to create only one instance of the PM at startup,” and now you’re back to the previous paragraph. A better alternative for AppEngine is to use a lightweight persistence framework designed specifically for AppEngine. ObjectifyService.begin() takes only miliseconds, and IMHO is easier to use than JDO or JPA, although I appreciate that Google makes those APIs available for portability.

Bottom line: by eliminating DI frameworks and using Objectify for persistence, I’m seeing cold starts in 2.5s average. I am more than happy to pay this small penalty for the privilege of running my code for free on the world’s most scalable Java infrastructure. This approach also conserves community resources vs. running a useless cron job just to keep your app warm, which makes cold starts happen all that more often for the rest of us.

Of course, once my app traffic takes off or Google come out with a pay-for-JVM option, all this goes away and you can use dependency injection, cron jobs, etc. with abandon. But for now, the name of the game is, how small can you make your app? Personally, I love the challenge. I much prefer lightweight, plain old Java to layer upon layer of frameworks, anyway.

Advertisement

Server-side browser detection in a servlet filter

Second things first: if you need a polite, pretty, and usable way to help your users upgrade their browser, point them to WhatBrowser.org, created just for that purpose.

But first you have to find out what browser they’re currently running. I’ve implemented this as a servlet filter on my signup and login pages so as to inform users as early as possible if their browser is inadequate. The basic idea is to find some part of the User-Agent header that uniquely identifies each browser. “WebKit” and “Mozilla” are out because these appear in so many browsers. Fortunately, most browsers have their common name in the User-Agent string somewhere. By default, the filter allows Chrome, Firefox, Safari, Opera, and IE versions 6-8. You can configure the filter with your own list if desired.

package com.turbomanage.gwt.server.servlet;

import java.io.IOException;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.inject.Singleton;

@Singleton
public class BrowserFilter implements Filter
{
	// Be default, support all GWT-capable browsers
	// Assume any version recent enough except IE
	private static final String[] DEFAULT_BROWSERS =
	{ "Chrome", "Firefox", "Safari", "Opera", "MSIE 8", "MSIE 7", "MSIE 6" };

	// Filter param keys
	public static final String KEY_BROWSER_IDS = "browserIds";
	public static final String KEY_BAD_BROWSER_URL = "badBrowserUrl";

	// Configured params
	private String[] browserIds;
	private String badBrowserUrl;

	@Override
	public void init(FilterConfig cfg) throws ServletException
	{
		String ids = cfg.getInitParameter(KEY_BROWSER_IDS);
		this.browserIds = (ids != null)?ids.split(","):DEFAULT_BROWSERS;

		badBrowserUrl = cfg.getInitParameter(KEY_BAD_BROWSER_URL);
		if (badBrowserUrl == null)
		{
			throw new IllegalArgumentException("BrowserFilter requires param badBrowserUrl");
		}
	}

	@Override
	public void doFilter(ServletRequest req, ServletResponse resp, FilterChain chain)
		throws IOException, ServletException
	{
		String userAgent = ((HttpServletRequest) req).getHeader("User-Agent");
		for (String browser_id : browserIds)
		{
			if (userAgent.contains(browser_id))
			{
				chain.doFilter(req, resp);
				return;
			}
		}
		// Unsupported browser
		((HttpServletResponse) resp).sendRedirect(this.badBrowserUrl);
	}

	@Override
	public void destroy()
	{
		this.browserIds = null;
	}
}

I’m using Guice, so this is what my configuration looks like. Of course, you can also use standard web.xml configuration instead.

package com.roa.server.guice;

import java.util.HashMap;

import com.google.inject.Singleton;
import com.google.inject.servlet.ServletModule;
import com.turbomanage.gwt.server.servlet.BrowserFilter;

public class DispatchServletModule extends ServletModule
{
	@Override
	public void configureServlets()
	{
		HashMap<String, String> filterCfg = new HashMap<String,String>();
		filterCfg.put(BrowserFilter.KEY_BROWSER_IDS, "Chrome,Firefox,Safari,Opera,MSIE 8");
		filterCfg.put(BrowserFilter.KEY_BAD_BROWSER_URL, "/s/bb.html");
		filter("/roa/signup/signup.jsp", "/roa/app/login", "/roa/app/index.html").through(
			BrowserFilter.class, filterCfg);
		...
	}
}

You initialize the filter with one or two parameters. The badBrowserUrl param is required. The filter will redirect any unsupported browser to this location. You can optionally set your own comma-separated list of browser identification strings in the browserIds param.

For further research on browser server-side identification, check out http://www.zytrax.com/tech/web/browser_ids.htm.

How to distinguish between AppEngine dev and prod environments

How can you tell whether your server-side AppEngine code is running in dev or production? (Hopefully, you don’t need to know too often, but for purposes of substituting test email addresses, etc., it might come in handy). I’ve discovered this AppEngine system property tucked away in Vince Bonfanti’s Task Queue helper class Deferred.java:

    import com.google.appengine.api.utils.SystemProperty;
    ...
    private static boolean isDevelopment() {
        return ( SystemProperty.environment.value() ==
                    SystemProperty.Environment.Value.Development );
    }

If you need to distinguish between AppEngine environments in GWT code, you would have to make an RPC call, which would probably leave you with an if statement in your application code somewhere, although someone very clever might come up with a way to do something GWT-side similar to what Andy Stevko wrote up on StackOverflow a few weeks ago.

If you’ve done something like this in GWT or GIN, I’m all ears.

Presenting at DevNexus

For those readers in the Atlanta area, I’ll be presenting case studies on AppEngine and GWT at DevNexus Mar 8-9. I’ll be highlighting the frameworks I’ve written about here: gwt-presenter, gwt-dispatch, and Objectify, along with Guice and GIN.

If you plan to be there and want to see specific topics covered, you can post your request as a comment here.

Yet another way to get the Guice injector

I previously wrote about a way to get ahold of the Guice injector in a JSP by putting it in the ServletContext; however, I have another case where I need it in a class that doesn’t have access to the ServletContext, either. The class happens to be a background task that implements the Deferrable interface, which for sake of interface simplicity does not provide access to the ServletContext even though the task does in fact run in a servlet.

The solution is simply to create the Guice injector in a static factory:

package com.roa.server.guice;

import com.google.inject.Guice;
import com.google.inject.Injector;

public class GuiceFactory
{
	private static final Injector inj = Guice.createInjector(...);
	
	public static Injector getInjector()
	{
		return inj;
	}
}

Now any class that needs access to a Guice-managed object can get it easily, including the ultra-simplified Guice context listener that’s registered in web.xml:

package com.roa.server.guice;

import com.google.inject.Injector;
import com.google.inject.servlet.GuiceServletContextListener;

public class GuiceConfig extends GuiceServletContextListener
{

	@Override
	protected Injector getInjector()
	{
		return GuiceFactory.getInjector(); 
	}
	
}

WARNING: this works fine in AppEngine because there’s only one WAR per JVM in that environment. If you’re not running in AppEngine and have multiple WARs in a single JVM, as is commonly the case, you may or may not want a single Guice injector for the whole JVM. I have not tested it, but I would think you wouldn’t be able to use Guice ServletModules in that case.

Unit testing with MockHttpSession and Guice

The more I learn about Guice, the more I like it. First, I discovered I could get access to the HttpSession inside my Guice-injected service classes (which are in turn called from my Guice-injected gwt-dispatch ActionHandlers) like this:

public class ROAUserServiceImpl implements ROAUserService
{
	@Inject
	Provider<HttpSession> httpSession;

	@Override
	public LoginInfo login(String requestUri)
	{
		httpSession.get().invalidate();
		...
	}
	...
}

Then came the unhappy moment that I ran my unit tests, which call ActionHandlers, all of which call ROAUserService to verify that the user is logged in. In my unit test environment, Guice cannot inject an HttpSession because my tests are not currently running in a servlet container; thus, every test fails.

I could bring in HttpUnit / ServletRunner, but that is really overkill for what I’m trying to do at this point, which is just to simulate one session attribute being set to represent the logged in user.

I could also create a mock ROAUserService and inject that in my ActionHandlers instead of the real one. But I’d prefer to use the real service for testing, and sooner or later, I’ll need to stuff something else in session, anyway.

Enter Guice Providers.

In the example above, the HttpSession is being injected by way of a Guice Provider, which in the real app, comes for free when your Guice injector includes a ServletModule. Fortunately, we can easily tell Guice to provide a mock HttpSession by using the @Provides annotation in a plain old AbstractModule (no ServletModule required):

package com.turbomanage.gwt.server.guice;

import javax.servlet.http.HttpSession;

import com.google.inject.AbstractModule;
import com.google.inject.Provides;
import com.roa.test.mock.MockHttpSession;

public class DispatchTestModule extends AbstractModule
{
	@Override
	protected void configure()
	{
		// application bindings here
	}

	@Provides
	HttpSession getMockSession()
	{
		return new MockHttpSession();
	}
}

I could probably snag a mock session from some other framework, but all I need to do is get and set attributes, hence:

package com.roa.test.mock;

import java.util.Enumeration;
import java.util.HashMap;
import java.util.Map;

import javax.servlet.ServletContext;
import javax.servlet.http.HttpSession;
import javax.servlet.http.HttpSessionContext;

public class MockHttpSession implements HttpSession
{
	private Map<String, Object> attributes = new HashMap<String, Object>();

	@Override
	public Object getAttribute(String name)
	{
		return attributes.get(name);
	}

	@Override
	public void setAttribute(String name, Object value)
	{
		attributes.put(name, value);
	}

	// All other methods auto-generated
	...
}

Now my TestCase setUp() method can simulate user login, logout, etc. by calling the real ROAUserService, which Guice provides with our mock HttpSession.

		...
		User u = addTestUser();
		Injector inj = Guice.createInjector(..., new DispatchTestModule());
		ROAUserService userService = inj.getInstance(ROAUserService.class);
		userService.login(...);

Guice Providers can also accept user-defined annotations to allow you to select among multiple implementations of a provided class. That might be another handy testing tool…

How to inject Guice objects in a JSP

My current project is written mostly in GWT; however, the GWT module is hosted in a JSP page that needs to obtain the user’s first name from my user service, which is managed by Guice. Since I have only one elementary JSP page, it is not worth introducing Struts or another MVC framework with Guice integration already built in, so I need a way to way to get the Guice context inside the JSP.

It now occurs to me that my login servlet could simply put the needed information into the session, where it could easily be accessed via JSP sessionScope; however, lucky for you, I had to go the long way around before I thought of this. It’s pretty simple, really. In a Web application, Guice gets initialized at application startup in your subclass of GuiceServletContextListener. Normally, you only need to override the getInjector() method of this class, like this:

public class DispatchServletGuiceConfig extends GuiceServletContextListener
{
	@Override
	protected Injector getInjector()
	{
		return Guice.createInjector(new ServerModule(),
			new DispatchServletModule());
	}
}

However, this simple version does not permit direct access to the injector (and therefore instances managed by Guice) except in other classes created by Guice. Since the JSP servlet doesn’t know about Guice, we need to make the Guice injector available ourselves. Fortunately, the GuiceServletContextListener and JSP pages both share access to the ServletContext, so we can override the other two methods in our Guice context listener to put the injector in the ServletContext. The GuiceServletContextListener no doubt does this already, but not where we can predictably find it when a new version comes out, etc. Here’s the expanded implementation:

package com.roa.server.guice;

import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;

import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.servlet.GuiceServletContextListener;
import com.turbomanage.gwt.server.guice.DispatchServletModule;

/**
 * This Guice context listener creates the Guice injector. It's a little
 * more complicated than the usual single-method version in order to
 * store the injector in ServletContext for access by JSPs.
 *
 * @author David Chandler
 */
public class DispatchServletGuiceConfig extends GuiceServletContextListener
{
	@Override
	protected Injector getInjector()
	{
		return Guice.createInjector(new ServerModule(),
			new DispatchServletModule());
	}

	@Override
	public void contextInitialized(ServletContextEvent servletContextEvent)
	{
		// No call to super as it also calls getInjector()
		ServletContext sc = servletContextEvent.getServletContext();
		sc.setAttribute(Injector.class.getName(), getInjector());
	}

	@Override
	public void contextDestroyed(ServletContextEvent servletContextEvent)
	{
		ServletContext sc = servletContextEvent.getServletContext();
		sc.removeAttribute(Injector.class.getName());
		super.contextDestroyed(servletContextEvent);
	}

}

Now in our JSP, we can access the injector via the ServletContext:

<%@ page import="com.google.inject.Injector"%>
<%@ page import="com.google.inject.Guice"%>
<%@ page import="com.roa.server.service.common.ROAUserService"%>
<%@ page import="com.roa.common.domain.User"%>
...
<%
	Injector inj = (Injector) pageContext.getServletContext().getAttribute(Injector.class.getName());
	ROAUserService roaUserService = inj.getInstance(ROAUserService.class);
	User user = roaUserService.getLoggedInUser();
%>

The merits of this approach in this case are admittedly debatable. However, it never hurts to have another tool in your toolbox…

Writing common test data services for gwt-dispatch with Guice

The way it stands now after my last several posts on unit testing is that each test method in a JUnit TestCase runs its own context, including its own PersistenceManager injected with Guice. Because I’m initializing the AppEngine test environment with LocalDatastoreService.NO_STORAGE_PROPERTY = TRUE, any test data I create in one test is not available to the next test method, even in the same TestCase class. This is typical behavior for JUnit test cases, and generally a good thing as each test should be independent, but it does mean we need a convenient way to create test data. My current strategy is twofold:

1. Create test data used by all tests in the TestCase setUp() method that runs before each test
2. Create common test data (used by multiple TestCases) in test data generator services

The first problem to solve is how to get access to a dispatch service and PersistenceManager inside the test data generators. Both are now being injected by Guice as covered in previous posts, so I’ve written a BaseTestDataService that does the Guice magic:

package com.roa.test.service;

import net.customware.gwt.dispatch.client.standard.StandardDispatchService;

import com.google.inject.Guice;
import com.google.inject.Injector;
import com.roa.server.guice.ServerModule;
import com.turbomanage.gwt.server.PMF;
import com.turbomanage.gwt.server.guice.DispatchTestModule;

public class BaseTestDataService
{
	protected static Injector inj = Guice.createInjector(new ServerModule(),
		new DispatchTestModule());;

	protected static StandardDispatchService getService()
	{
		return inj.getInstance(StandardDispatchService.class);
	}

	protected static PMF getPMF()
	{
		return inj.getInstance(PMF.class);
	}
}

I’m intentionally using a static getPMF() method to make it as simple as possible to use the test data services in a unit test via static methods. Alternatively, I could have used instance methods and constructor injection, but then you’d have to create a new instance of a test data service in each test, which is just that much more code to write… Also, constructor injection is not possible in this case because the TestCases themselves are not instantiated by Guice, so instantiating a new test data service from a test case would not involve Guice, either.

It does not matter that the BaseTestDataService and BaseTest (below) are both calling Guice.createInjector() and thereby creating multiple Guice contexts, each having its own PMF instance. The important thing for these database tests is that they’re all going against one Datastore, not one PersistenceManager.

Here’s a simple test data generator that extends BaseTestDataService and provides a method to add a test user:

package com.roa.test.service;

import com.roa.client.domain.User;
import com.roa.shared.rpc.AddUserAction;
import com.roa.shared.rpc.AddUserResult;

public class UserTestDataService extends BaseTestDataService
{
	public static User addTestUser() throws Exception
	{
		// Create new user
		User u = new User();
		u.setEmailAddress("test@example.com");
		u.setFirstName("Test");
		u.setLastName("User");
		u.setGoogleAccountId("testAccountId");
		AddUserResult userResult = (AddUserResult) getService().execute(
			new AddUserAction(u));
		u = userResult.getUser();
		return u;
	}
}

Note the call to getService() on line 17. Thanks to Guice, test data generator services can invoke gwt-dispatch ActionHandlers the same way as the tests themselves.
Finally, here’s a test case that calls the above service to create a test user in the Datastore:

package com.roa.test;

import java.util.List;

import javax.jdo.Query;

import com.appenginefan.toolkit.unittests.BaseTest;
import com.roa.client.domain.User;
import com.roa.test.service.UserTestDataService;

public class UserTestCase extends BaseTest
{

	@Override
	protected void setUp() throws Exception
	{
		super.setUp();
		UserTestDataService.addTestUser();
	}

	public void testUserAdded() throws Exception
	{
		// Run tests here
		Query q = pm.newQuery(User.class, "emailAddress == e");
		q.declareParameters("java.lang.String e");
		List<User> users = (List<com.roa.client.domain.User>) q.execute("test@example.com");
		assertEquals(1, users.size());
		User user = users.get(0);
		assertNotNull(user.getId());
	}
}

I should probably show my BaseTest method, too. I’m using AppEngineFan’s BaseTest as discussed in previous posts and modified the setUp() method for Guice injection as follows:

	/**
	 * Sets up the App Engine environment.
	 */
	@Override
	protected void setUp() throws Exception
	{
		if (initializer != null)
		{
			throw new UnsupportedOperationException(
				"setup may only be called once!");
		}
		super.setUp();
		initializer = new TestInitializer(getEnvironmentOrNull());
		initializer.setUp();
		Injector inj = Guice.createInjector(new ServerModule(),
			new DispatchTestModule());
		this.testSvc = inj.getInstance(StandardDispatchService.class);
		this.pm = inj.getInstance(PMF.class).getPersistenceManager();
	}

Thanks to Guice, we can now get access to the PersistenceManager and call ActionHandlers via a dispatch test service even from static methods in test data generators. This greatly streamlines unit testing.

More on unit testing with an injected JDO PersistenceManager

Regarding my previous post, it turns out that I needed a TestPMF implementation sooner than I thought. The reason is the way I’m injecting a DispatchTestService in my unit tests (as described in this post). I’m calling createInjector() in the setUp() method, which gets run before each test method:

	@Override
	protected void setUp() throws Exception
	{
		super.setUp();
		Injector inj = Guice.createInjector(new ServerModule(),
			new DispatchTestModule());
		testSvc = inj.getInstance(StandardDispatchService.class);
		// This pm is needed only for code in this class that calls a PM directly
		// ActionHandlers are injected with the PMF singleton from Guice
		pm = inj.getInstance(PMF.class).getPersistenceManager();
	}

Since setUp() gets called before each test method, Guice is initialized for each test method, and therefore Guice calls the constructor for my DefaultPMF class for each test method. Repeated from the previous post, the DefaultPMF class looks like:

package com.turbomanage.gwt.server;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public final class DefaultPMF implements com.turbomanage.gwt.server.PMF
{
	private final PersistenceManagerFactory pmfInstance = JDOHelper
			.getPersistenceManagerFactory("transactions-optional");

	public DefaultPMF()
	{
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmfInstance.getPersistenceManager();
	}
}

Because DefaultPMF creates a named PersistenceManagerFactory, JDO complains that the named PersistenceManagerFactory has already been created. My solution for now is to replace the DefaultPMF with a TestPMF that uses a Properties map to initialize the PersistenceManager, just as the AppEngineFan TestInitializer does. Here’s a working TestPMF:

package com.turbomanage.gwt.server;

import java.util.Properties;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public class TestPMF implements PMF
{
	private final PersistenceManagerFactory pmf;

	public TestPMF()
	{
		Properties newProperties = new Properties();
		newProperties
			.put("javax.jdo.PersistenceManagerFactoryClass",
				"org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManagerFactory");
		newProperties.put("javax.jdo.option.ConnectionURL", "appengine");
		newProperties.put("javax.jdo.option.NontransactionalRead", "true");
		newProperties.put("javax.jdo.option.NontransactionalWrite", "true");
		newProperties.put("javax.jdo.option.RetainValues", "true");
		newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns",
			"true");
		newProperties.put("datanucleus.appengine.autoCreateDatastoreTxns",
			"true");
		pmf = JDOHelper.getPersistenceManagerFactory(newProperties);
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmf.getPersistenceManager();
	}

}

Now simply bind PMF to its TestPMF implementation in your Guice module for tests (DispatchTestModule in the setUp() method above), and each unit test method will run with a freshly created PersistenceManager.

Unit testing with JDO PersistenceManager injected via Guice

This is a follow-up to last week’s post on unit testing ActionHandlers with Guice. David Peterson pointed out on the gwt-dispatch mailing list that I could inject a PersistenceManager into my ActionHandlers in order to provide an alternate PersistenceManager for unit testing. I don’t actually need an alternate PM yet as it is handled transparently by my AppEngine test environment, but I thought it would be easier to do it sooner rather than later, so here goes.

I’ve created an interface called PMF (in order to avoid confusion with JDO’s PersistenceManagerFactory).

package com.turbomanage.gwt.server;

import javax.jdo.PersistenceManager;

public interface PMF
{
	PersistenceManager getPersistenceManager();
}

My default PMF implementation works exactly the same as before:

package com.turbomanage.gwt.server;

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManager;
import javax.jdo.PersistenceManagerFactory;

public final class DefaultPMF implements com.turbomanage.gwt.server.PMF
{
	private final PersistenceManagerFactory pmfInstance = JDOHelper
			.getPersistenceManagerFactory("transactions-optional");

	public DefaultPMF()
	{
	}

	@Override
	public PersistenceManager getPersistenceManager()
	{
		return pmfInstance.getPersistenceManager();
	}
}

The DefaultPMF gets bound in my Guice ServerModule:

	@Override
	protected void configureHandlers()
	{
		...
		bind(PMF.class).to(DefaultPMF.class).in(Singleton.class);
		...
	}

And finally, the PMF is injected into an ActionHandler:

public class AddUserHandler implements
	ActionHandler<AddUserAction, AddUserResult>
{
	@Inject
	private PMF pmf;
	private PersistenceManager pm;

	@Override
	public AddUserResult execute(AddUserAction action, ExecutionContext context)
		throws ActionException
	{
		this.pm = pmf.getPersistenceManager();
		...
	}
	...
}

Now, when the need arises for a test implementation of PMF, I can easily bind a different implementation in my Guice TestModule as shown in the earlier post.

Update: I did not test this thoroughly before I posted, and the need for a TestPMF arose just hours later. See my next post for the solution.