TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Colorado. Besides tech, I enjoy landscape photography and share my work at ColoradoPhoto.gallery.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 241 other followers

  • Sleepless Nights…

    January 2010
    S M T W T F S
    « Dec   Feb »
     12
    3456789
    10111213141516
    17181920212223
    24252627282930
    31  
  • Blog Stats

    • 880,310 hits

Simplify with Objectify: a lightweight persistence framework for AppEngine

Posted by David Chandler on January 28, 2010

It is wonderful that Google provides JDO and JPA interfaces to the AppEngine Datastore. However, the learning curve is somewhat steep due to AppEngine’s non-relational nature and unique concepts such as entity groups, owned and un-owned relationships, etc. In addition, the PersistenceManager lifecycle (and particularly the necessity of detaching objects) often gets in the way. I’ve written a lot of code on this blog just to handle injecting the PM where it’s needed in services, unit tests, etc.

In stark contrast to all this stands the DataStore low-level API.  You can call the DatastoreService from anywhere in your code via the DatastoreServiceFactory, and there are only four methods to understand: get, put, delete, and prepare (query). It makes no pretense of being relational, but unfortunately, no pretense of being an object store, either, as the low-level API stores only objects of type com.google.appengine.api.datastore.Entity.

Enter Objectify, a recently-announced open source persistence framework for AppEngine. Objectify preserves the simplicity and transparency of the low-level API and does all the work of converting your domain objects to and from Datastore Entities. It’s a tiny jar (36k) with no external dependencies and zero startup time, which helps mitigate the AppEngine cold start problem.

In the process of Objectify-ing my old JDO code, I’m building a base DAO class that I can simply extend for each domain class and get all the standard CRUD operations for free. Because most Objectify methods use parameterized types, my generic DAO is largely redundant, but nevertheless provides a consistent layer in which to add type-specific methods like findUserByEmailOrPhone. Let’s look at some code.

First, a simple domain class. Note there are fewer and simpler annotations required than for JPA / JDO.

package com.roa.common.domain;

import java.io.Serializable;

import javax.persistence.Entity;
import javax.persistence.Id;

@Entity
public class User implements Serializable
{
	private static final long serialVersionUID = -1126191336687818754L;

	@Id
	// Objectify auto-generates Long IDs just like JDO / JPA
	private Long id;
	private String firstName;
	private String lastName;
	private String emailAddress;
	private String zipCode;
	private String googleAccountId;

	public User()
	{
		// Empty constructor needed for GWT serialization and Objectify
	}

	public Long getId()
	{
		return id;
	}

	public void setId(Long id)
	{
		this.id = id;
	}

	...
}

We can do persistence operations by calling the UserDao:

		// Create new user
		User u = new User();
		u.setEmailAddress("test@example.com");
		u.setFirstName("Test");
		u.setLastName("User");
		u.setGoogleAccountId("testAccountId");
		UserDao userDao = new UserDao();
		OKey<User> key = userDao.add(u);

		// Retrieve user by ID
		dao.get(u.getId());

		// Find users matching an email address
		users = dao.listByProperty("emailAddress", "test@example.com");

All the DAO methods above are provided in my base DAO, so the UserDao itself is quite simple:

package com.roa.server.dao;

import java.util.logging.Logger;

import com.googlecode.objectify.ObjectifyService;
import com.roa.common.domain.User;

public class UserDao extends ObjectifyDao<User>
{
	private static final Logger LOG = Logger.getLogger(UserDao.class.getName());

	static
	{
		ObjectifyService.register(User.class);
	}

	public UserDao()
	{
		super(User.class);
	}

}

And finally, my generic base DAO (a work in progress, caveat emptor). Note that I’ve created a couple methods to allow query by example, which I generally prefer to enumerating methods for every use case (getByEmail, getByLastName, getByFirstAndLastName, etc.). The getByProperty and listByProperty methods take a single property name and value on which to filter, and getByExample / listByExample allow you to specify multiple property names on which to filter.

package com.roa.server.dao;

import java.beans.BeanInfo;
import java.beans.IntrospectionException;
import java.beans.Introspector;
import java.beans.PropertyDescriptor;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;

import com.google.appengine.api.datastore.EntityNotFoundException;
import com.googlecode.objectify.OKey;
import com.googlecode.objectify.OQuery;
import com.googlecode.objectify.ObjectifyService;
import com.googlecode.objectify.helper.DAOBase;

public class ObjectifyDao<T> extends DAOBase
{
	private Class<T> clazz;

	/**
	 * We've got to get the associated domain class somehow
	 *
	 * @param clazz
	 */
	protected ObjectifyDao(Class<T> clazz)
	{
		this.clazz = clazz;
	}

	public OKey<T> add(T entity)

	{
		OKey<T> key = ofy().put(entity);
		return key;
	}

	public void delete(T entity)
	{
		ofy().delete(entity);
	}

	public void delete(OKey<T> entityKey)
	{
		ofy().delete(entityKey);
	}

	public T get(Long id) throws EntityNotFoundException
	{
		T obj = ofy().get(this.clazz, id);
		return obj;
	}

	/**
	 * Convenience method to get an object matching a single property
	 *
	 * @param propName
	 * @param propValue
	 * @return T matching Object
	 */
	public T getByProperty(String propName, Object propValue)
	{
		OQuery<T> q = ObjectifyService.createQuery(clazz);
		q.filter(propName, propValue);
		T obj = ofy().prepare(q).asSingle();
		return obj;
	}

	public List<T> listByProperty(String propName, Object propValue)
	{
		OQuery<T> q = ObjectifyService.createQuery(clazz);
		q.filter(propName, propValue);
		List<T> list = ofy().prepare(q).asList();
		return list;
	}

	public T getByExample(T u, String... matchProperties)
	{
		OQuery<T> q = ObjectifyService.createQuery(clazz);
		// Find non-null properties and add to query
		for (String propName : matchProperties)
		{
			Object propValue = getPropertyValue(u, propName);
			q.filter(propName, propValue);
		}
		T obj = ofy().prepare(q).asSingle();
		return obj;
	}

	public List<T> listByExample(T u, String... matchProperties)
	{
		OQuery<T> q = ObjectifyService.createQuery(clazz);
		// Find non-null properties and add to query
		for (String propName : matchProperties)
		{
			Object propValue = getPropertyValue(u, propName);
			q.filter(propName, propValue);
		}
		List<T> list = ofy().prepare(q).asList();
		return list;
	}

	private Object getPropertyValue(Object obj, String propertyName)
	{
		BeanInfo beanInfo;
		try
		{
			beanInfo = Introspector.getBeanInfo(obj.getClass());
		}
		catch (IntrospectionException e)
		{
			throw new RuntimeException(e);
		}
		PropertyDescriptor[] propertyDescriptors = beanInfo.getPropertyDescriptors();
		for (PropertyDescriptor propertyDescriptor : propertyDescriptors)
		{
			String propName = propertyDescriptor.getName();
			if (propName.equals(propertyName))
			{
				Method readMethod = propertyDescriptor.getReadMethod();
				try
				{
					Object value = readMethod.invoke(obj, new Object[] {});
					return value;
				}
				catch (IllegalArgumentException e)
				{
					throw new RuntimeException(e);
				}
				catch (IllegalAccessException e)
				{
					throw new RuntimeException(e);
				}
				catch (InvocationTargetException e)
				{
					throw new RuntimeException(e);
				}
			}
		}
		return null;
	}

}

Have fun!

Advertisements

23 Responses to “Simplify with Objectify: a lightweight persistence framework for AppEngine”

  1. Harald said

    Very interesting post! I’d like to use Objectify for my GAE/GWT app. In my entities I have references to com.google.appengine.api.users.User. Do you know if Objectify supports persistence of com.google.appengine.api.users.User?

    Greetings
    Harald

  2. Scott said

    You should be able to do the QBE without having to pass in the property list. If you just walk over all the fields and check the values for not null that should give you a list of the fields to filter by, and their values. Then it will look like this: get/listByExample(T example) {…}

    You may also want to throw an exception in getByExample(T example) if the return list has more than one item. Hiding the fact that you had multiple matches could be a dangerous thing.

    • Thank you, Scott. I had considered writing the byExample methods exactly as you suggest, but was concerned that walking over the values to find non-nulls involves more calls to Method.invoke(). Probably not a big deal, though, and would be cleaner.

      Your second point raises an interesting question: because my getByExample calls .asSingle(), I assumed that Objectify would throw an exception on multiple matches. Upon further examination, I see that OPreparedQueryImpl.asSingle calls AppEngine’s PreparedQuery.asSingleEntity(), which according to http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/PreparedQuery.html#asSingleEntity() throws TooManyResultsException (unchecked). So I think an exception would bubble up in this case, albeit unchecked.

      • Scott said

        You gotta love un/checked exceptions. I never felt good about java exceptions for reasons like this. It is too complicated and confusing.

        Good catch.

        As for the method.Invoke() calls, I hear you. I am kinda mad at people who don’t realise the ramifications of this (and there are lots of them). But I was suggesting using the fields since that is how Objectify works; No getters, just fields.

      • Oh, I see. I hadn’t noticed that Objectify was using fields directly. Given that I would be repeating a lot of the logic found in EntityMetadata.visit() to likewise iterate over fields, could Objectify possibly expose some of the metadata through the BaseDao class? Would be nice to get at readables and a getProperty/setProperty method. Or maybe just provide a queryByExample method in Objectify to prevent leaky abstractions…

      • Scott said

        I don’t know if you were watching the checkins (2 weeks ago) but we made most of the EntityMetadata intenrals protected for just this reason. I will have to look at the current 2.0 (trunk) stuff and see if it is still accessible. You could then create your own EntityMetadata derived class to do what you want.

  3. Haroon Ahmad said

    David,
    Thanks for your post. The one issue I’m struggling with is how to get the ‘join’ functionality offered by Relational Databases. If I have a ‘Position’ entity and an ‘Applicant’ entity, how can I query a list of all ‘positions’ an applicant has applied to, and get the ‘Applicant’ name back in the result of that one query, rather than just the applicant ID?

    This is the problem that ‘join’ solves in and RDBMS. Can you please suggest how I can do this using Objectify or JDO in the google datastore?

    Thanks!

    Haroon

    • If you haven’t already, I recommend you watch the video from Google I/O referenced in my previous post. While AppEngine Datastore does not allow true relational joins, you can simulate a simple many-to-many relationship like the one you’ve described by using Datastore list properties as described in the video. Hope that helps…

      • Haroon Ahmad said

        Thanks, David. Will take a look at that. Meanwhile, looks like BigTable expects us to do some de-normalization in our queries. Which would mean, that I should keep the applicant’s name in the ‘Application’ table, as well as the ‘Position’ title, so that when I display the positions, that the applicant has applied to, I don’t have to fetch either the applicant name or the position title, from the ‘Position’ or ‘Applicant’ table.

        I could do that, but then if the position’s title changes, would I be expected to update all occurrences of position title, in the ‘Applications’ table? Is that the way BigTable data management is supposed to be done?

        Thanks again.

  4. kc said

    how come com.googlecode.objectify.OQuery; is not being found? thanx

  5. thomas said

    Great Post, thanks!

    Just my 2 cents, as the API has changed the DAO Utility class should now read:

    import java.beans.BeanInfo;
    import java.beans.IntrospectionException;
    import java.beans.Introspector;
    import java.beans.PropertyDescriptor;
    import java.lang.reflect.InvocationTargetException;
    import java.lang.reflect.Method;
    import java.util.List;

    import com.google.appengine.api.datastore.EntityNotFoundException;
    import com.googlecode.objectify.Key;
    import com.googlecode.objectify.Objectify;
    import com.googlecode.objectify.ObjectifyService;
    import com.googlecode.objectify.Query;
    import com.googlecode.objectify.helper.DAOBase;

    public class ObjectifyDAO extends DAOBase
    {
    private Class clazz;

    /**
    * We’ve got to get the associated domain class somehow
    *
    * @param clazz
    */
    protected ObjectifyDAO(Class clazz)
    {
    this.clazz = clazz;
    }

    public Key add(T entity)

    {
    Key key = ofy().put(entity);
    return key;
    }

    public void delete(T entity)
    {
    ofy().delete(entity);
    }

    public void delete(Key entityKey)
    {
    ofy().delete(entityKey);
    }

    public T get(Long id) throws EntityNotFoundException
    {
    T obj = ofy().get(this.clazz, id);
    return obj;
    }

    public T get(String id) throws EntityNotFoundException
    {
    T obj = ofy().get(this.clazz, id);
    return obj;
    }

    /**
    * Convenience method to get an object matching a single property
    *
    * @param propName
    * @param propValue
    * @return T matching Object
    */
    public T getByProperty(String propName, Object propValue)
    {
    T result = ofy().query(this.clazz).filter(propName, propValue).get();

    return result;
    }

    public List listByProperty(String propName, Object propValue)
    {
    List list = ofy().query(this.clazz).filter(propName, propValue).list();
    return list;
    }

    public T getByExample(T u, String… matchProperties)
    {
    Query q = ofy().query(this.clazz);
    // Find non-null properties and add to query
    for (String propName : matchProperties)
    {
    Object propValue = getPropertyValue(u, propName);
    q.filter(propName, propValue);
    }
    T obj = q.get();

    return obj;
    }

    public List listByExample(T u, String… matchProperties)
    {
    Query q = ofy().query(this.clazz);
    // Find non-null properties and add to query
    for (String propName : matchProperties)
    {
    Object propValue = getPropertyValue(u, propName);
    q.filter(propName, propValue);
    }
    List list = q.list();

    return list;
    }

    private Object getPropertyValue(Object obj, String propertyName)
    {
    BeanInfo beanInfo;
    try
    {
    beanInfo = Introspector.getBeanInfo(obj.getClass());
    }
    catch (IntrospectionException e)
    {
    throw new RuntimeException(e);
    }
    PropertyDescriptor[] propertyDescriptors = beanInfo.getPropertyDescriptors();
    for (PropertyDescriptor propertyDescriptor : propertyDescriptors)
    {
    String propName = propertyDescriptor.getName();
    if (propName.equals(propertyName))
    {
    Method readMethod = propertyDescriptor.getReadMethod();
    try
    {
    Object value = readMethod.invoke(obj, new Object[] {});
    return value;
    }
    catch (IllegalArgumentException e)
    {
    throw new RuntimeException(e);
    }
    catch (IllegalAccessException e)
    {
    throw new RuntimeException(e);
    }
    catch (InvocationTargetException e)
    {
    throw new RuntimeException(e);
    }
    }
    }
    return null;
    }

    }

  6. yaa said

    Hi, I’m using objectify-3.0.jar but getting an error from eclipse:
    com.googlecode.objectify.ObjectifyService is not supported by Google App Engine’s Java runtime environment
    any idea why?

  7. yaa said

    Hi, Some more details:
    using GAE SDK 1.5.0
    GWT SDK 2.3.0

    and verified that objectifyService exists in the objectify jar

  8. RV said

    Hi Chandler,

    I think it is a great post, I also tried to use this in my GWT project. I tried to save my data objects through DAO from RemoteServiceServlet on the server side. I am not sure what is the problem but it is throwing error while I am creating an instance of one of my DAO class which is extending your ObjectifyDAO class

    java.lang.NoClassDefFoundError: com/googlecode/objectify/util/DAOBase

    Can you provide the reason behind it?

  9. Vicente Tommasi said

    Hi David. I’m working in a tool that generates Java web applications (www.jrapid.com), and I wanted to generate also GAE/J ready applications. What do you recommend to use for persistence? Should I use Objectify or plain JPA?

    Regards,

    Vicente

    • Hi Vicente, it really depends on your intended audience. My hunch is that most brand new apps (startups, social apps) would choose Objectify or one of the other 3rd party Datastore libs (Twig, Slim3), whereas enterprise developers would be more inclined toward JPA. But even there, it’s probably easier to rewrite a persistence layer in Objectify from scratch than try to fit JPA from relational world into Datastore world. And it would almost certainly be easier for your tool to generate Objectify code.

      • Vicente Tommasi said

        Thanks David! The intended audience are enterprise developers. But I agree with you, I think it would be easier to generate Objectify code.

  10. Hi David, this post is simply awesome, but…. imagine my domain: 25 POJO-domain classes (Invoice, Customer, Bank, and so on…).

    Do I have to create 25 “public class InvoiceDao extends ObjectifyDao” for each one of my domain classes (here I took Invoice as example) ?
    Isn’t it a bit redundant?

    Or I rather prefer to directly use instances of “ObjectifyDao ” directly, without creating lot of pretty identical subclasses?

    Hope you still read this post’s comment
    Thank you, sorry for my english and keep up the good work!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: