TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Peru. I am currently offering public and private developer training courses in the US and Latin America as well as working on Android, GWT, and App Engine projects.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 227 other followers

  • Sleepless Nights…

    December 2012
    S M T W T F S
    « Sep   Feb »
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    3031  
  • Blog Stats

    • 661,079 hits

stORM: a lightweight DAO generator for Android SQLite

Posted by David Chandler on December 11, 2012

In my previous blog post on writing a Java annotation processor, I mentioned that I’d been working on a lightweight ORM for Android SQLite. I’m finally ready to offer a preview of project stORM with this caveat: at this stage, it’s more of a DAO (data access object) code generator than a full-fledged ORM. The generated DAO classes offer a simple interface to save and retrieve objects to SQLite without having to write any SQL code; however, stORM doesn’t attempt to offer features of full-fledged ORMs such as lazy loading of an object graph.

Goals

stORM is intended as an easy way to create a basic object store with SQLite. It lets you insert, update, delete, and query objects without having to write any SQL. It can persist fields of any primitive or wrapper type, including enums. in addition, for more complex types, you can provide your own custom TypeConverter with @Converter. stORM does not yet support modeling of relations. Support for foreign keys and indexes is planned.

stORM is implemented as a Java annotation processor so it can be used with or without an IDE. To use it, you simply annotate POJO (plain old Java object) classes that you want to persist in a SQLite database and stORM automatically generates a SQLiteOpenHelper class and DAO classes for you.

Example

Here’s an example of stORM usage. The Person class is a POJO with a few String and primitive types (firstName, lastName, weight, and height). Note that it includes an id field of type long. An ID field of type long is required along with corresponding accessor methods. To name the ID field differently, place the @Id annotation on any field of type long.

package com.turbomanage.demo.entity;

import com.turbomanage.storm.api.Entity;

@Entity
public class Person {

	private long id;
	private String firstName, lastName;
	private int weight;
	private double height;

	// accessor methods omitted
}

The Person class is annotated with @Entity, which enables the annotation processor to generate a corresponding DAO class. To insert a new Person in the database, we simply use the generated DAO class. The DAO package name is derived by appending “.dao” to your entity package, and the DAO classname is the entity classname + “Dao”.

import com.turbomanage.demo.entity.dao.PersonDao;
...
PersonDao dao = new PersonDao(getContext());
Person newPerson = new Person();
newPerson.setFirstName("David");
newPerson.setLastName("Chandler");
// insert
long newPersonId = dao.insert(newPerson);
// query by example
Person examplePerson = new Person();
examplePerson.setFirstName("David");
List allDavids = dao.listByExample(examplePerson);

The insert() method returns the id of the newly inserted row. In addition, the id field of the Person object is populated with this value.

The DAO’s listByExample() method is the simplest way to do a query. It lets you retrieve all rows matching the fields of an example object with non-default values. There is also a filter() method which lets you specify conditions directly.

Setup

To get started with stORM:

  1. Download the storm-api and storm-apt jars from storm-gen.googlecode.com.
  2. Add storm-api.jar to your project’s libs/ folder. This contains the stORM annotations and runtime.
  3. Add storm-apt.jar to your project’s annotation factory class path. In Eclipse, you can find this under project properties | Java Compiler | Annotation Processing | Factory Path. The apt jar contains the annotation processor which inspects your project’s source code and generates the required database classes.
Eclipse properties dialog

Add storm-apt.jar to your project’s annotation factory classpath in Eclipse.

Usage

Once you’ve added the jars to your project, it’s easy to use stORM. You just need a DatabaseHelper class annotated with @Database and one or more POJO classes annotated with @Entity.

  1. Create a new class that extends DatabaseHelper.
  2. Add @Database and supply a database name and version.
  3. Override getUpgradeStrategy() to choose one of the available strategies (DROP_CREATE for new projects).
  4. Create a POJO class you want to persist to the database.
  5. Make sure it has a field of type long named “id” or annotated with @Id.
  6. Add the @Entity annotation to the class.

Following these steps, you’ll see 3 generated classes under .apt_generated (you might need to unfilter hidden resources in the Eclipse Project Explorer to see it):

  • a DbFactory class. This provides a singleton instance of your DatabaseHelper class (which in turn extends SQLiteOpenHelper). Having only one instance of SQLiteOpenHelper ensures that SQLite’s own threadsafe mechanisms work as intended.
  • a DAO class for each entity. This extends SQLiteDao and mainly supplies the necessary type parameter to the base class.
  • a Table class for each entity. This contains the generated code to convert an entity instance to and from a SQL row.

You can use the DAO simply by creating a new instance. You must pass it the application context as this is required by the underlying SQLiteOpenHelper.

PersonDao dao = new PersonDao(getContext());

The DAO constructor obtains the singleton instance of your DatabaseHelper class from the generated factory class. Because of this, it’s safe to create new DAO instances anywhere you need them. You can now invoke any of the public methods on the DAO, such as insert(), update(), and listAll(). There are also listByExample() and getByExample() methods that build a query from an example object.

You can also use the FilterBuilder API (still under contruction) to run a query like this:

List<Person> allDavids = dao.filter().eq(Columns.FIRSTNAME, "David").list();

Version upgrades

One of the more challenging aspects of using SQLite can be upgrading your app’s database schema with new versions. stORM makes this easier by providing several upgrade strategies which you can choose in your DatabaseHelper class:

  • DROP_CREATE will drop all tables and create the database again using the new schema implied by your new entity definitions. This will destroy all data in the database, so should only be used in testing, not after your app is in production.
  • BACKUP_RESTORE will first backup the database to CSV files, then drop and recreate the database, and finally restore all the data from the backup CSV files. Any new fields (columns) in your entities will receive default values, and dropped columns will simply disappear. It is not yet possible to rename a field or change the field type.
  • UPGRADE lets you implement your own strategy (probably an ALTER TABLE statement) by overriding the onUpgrade() method in the DatabaseHelper class.

Dump to CSV

Finally, stORM can automatically backup your database to CSV files and restore it from the same. This is used by the BACKUP_RESTORE UpgradeStrategy. You can also manually dump to CSV or restore from CSV by calling methods directly on your DatabaseHelper class. Example:

DemoDbFactory.getDatabaseHelper(getContext()).backupAllTablesToCsv();
DemoDbFactory.getDatabaseHelper(getContext()).restoreAllTablesFromCsv();

You can browse the CSV files using adb shell under /data/data/your_app_package/files. Be aware that any byte[] fields are Base64 encoded, and fields of type double are encoded using their exact hex representation. See the code in BlobConverter.fromString() and DoubleConverter.fromString() if you want to read and parse these values independently of stORM.

Coming soon: ContentProvider

Another pain point for many developers is creating a ContentProvider. Stay tuned for a way to generate ContentProvider contract stubs and implementations also.

Source code

storm-gen.googlecode.com

The issue tracker is open…

Enjoy!

37 Responses to “stORM: a lightweight DAO generator for Android SQLite”

  1. Tin Tvrtković said

    Looks interesting. The name might be a little unfortunate; there’s already a Python ORM called Storm :)

  2. I cant seem to be able to get this working.My project builds fine but does not generate the .apt_generated folder and the classes(I have unfiltered resources in Eclipse). No errors reported.But the classes are not generated.

    • I got it working. I had forgotten to enable annotation processing.Works now.It works wonderfully well.

      • Hello David,

        I am very keen on using stORM, but just like Abhijith Srivatsav, the DAO and dbFactory classes aren’t generated at all. No errors, nothing. I have annotation processiong enabled; I’ve upgraded the Eclipse (Juno) ADT bundle to the plugin 21.1 (on Windows 7 32bit) and still it doesn’t generate the files. I definitely followed the steps you mentioned above.

        I’ll appreciate it loads if you can guide me to resolve this soones. Thanks.

      • Hi Ugo, I fixed a bug in ProcessorLogger that will hopefully resolve this. Try with version 0.94 which I just posted and let me know.

        On Thu, Feb 28, 2013 at 10:36 AM, TurboManage

      • I got it working alright. Inspired by steps for activating AndroidAnnotations (here: https://github.com/excilys/androidannotations/wiki/Eclipse-Project-Configuration) I clicked on the “Java Compiler” top-level entry, inside of the project Properties, and then changed the “Compiler compliance level” to a higher value of “1.6”. Then clicked “Apply” and confirmed “Rebuild” request, and the “.apt_generated” folder was immediately created, with the DAO and dbFactory class files inside it.

        Hope this helps someone too.

      • Thanks, Ugo. I’ll add that to the docs.

      • Hello David. Thanks for responding to my earlier message.

        So, what’s really new with the 0.94 release? There isn’t any kind of changelog/doc explaining what’s new (aside what you indicated in your comment above). BTW, have you considered moving this project to Github, or does being a Googler prohibit you from doing so? :-)

        Anyways, keep up the good work.

      • Lastpop said

        I’ve got the problem about the “.apt_generated” folder It’s create but it empty

  3. [...] stORM: a lightweight DAO generator for Android SQLite [...]

  4. Admiring the time and energy you put into your site and in depth information you present.
    It’s awesome to come across a blog every once in a while that isn’t the same outdated rehashed information.

    Excellent read! I’ve saved your site and I’m including your RSS feeds to my Google account.

  5. Is it possible do not persist some members of POJO, for example static final constants?

  6. [...] David Chandler: stORM: a lightweight DAO generator for Android SQLite [...]

  7. jaxx said

    Hi, David!

    Is there any examples how to implement many-to-many and one-to-many relationships?

    • stORM does not support relationship modeling of any kind.

      • jakko100 said

        But do you plan to support relationships?

      • Yes, to a limited degree. I’m thinking of an @Key annotation similar to Objectify-appengine. The presence of this annotation would generate the SQL FOREIGN KEY constraint. I also plan to enhance the query API with overloaded methods that let you indicate whether to retrieve key-related objects at the time of the query. And I’m thinking of an ObjectCursor wrapper which could do this lazily.

  8. Harsha Mv said

    i am using the ADT and a demo project which uses stORM. When i create the project all works fine. When i close eclipse and restart it i am getting the following error for the project that uses stORM.

    Errors occurred during the build.
    Errors running builder ‘Android Pre Compiler’ on project ‘stORM’.
    java.lang.NullPointerException

    I have also posted a question on SO: http://stackoverflow.com/questions/14251841/android-storm-causes-building-workspace-error

    I think there is some problem with ADT vs stORM. If i remove the stORM project all works normal. Any idea how to solve it ?

  9. Johan Pelgrim said

    Hi David,

    Great article on your stORM framework. It looks promising and I’m looking forward to future developments. E.g. you’re saying Another pain point for many developers is creating a ContentProvider. Stay tuned for a way to generate ContentProvider contract stubs and implementations also. Is there any outlook when you will include these generators in the framework as well, or when you will describe how one can do this? I’m currently firing up a new Android project and am in the library / framework selection phase ;-)

    Thanks in advance for any comment on this!

    Kind regards,

    Johan.

    • Hi Johan, I’m still grappling with the best way to do this. The challenge is that the ORM interface and ContentProvider interfaces are very different (object vs. SQL-oriented). One of my colleagues has prototyped some ContentProvider templates using stORM’s annotation processor engine, but we’re still discussing whether / how to tie the projects together. If anything, stORM should wrap the ContentProvider since the object API is higher level, but unless you need a ContentProvider, there’s no reason to add this layer just to use the object API. Are you looking to use ContentProviders in place of the object API or in addition to it?

      • Johan Pelgrim said

        Hi David,

        Thanks for your reply! I’m more interested in the ContentProvider template work than the DAO / ORM solution (sorry ;-). I need to work with SyncAdapters (or at least, that’s what I’m aiming for and want a proof-of-concept of) and also want to make use of fragments and LoaderManagers and the whole kit and kaboodle. ContentProviders fit well in that architecture. Could you attach a link to your colleagues work? I’d like to see what he’s doing. My current approach is simply working with the Framework and coding everything by hand, which can become cumbersome and tedious at times.

        Kind regards,

        Johan.

  10. Hey David, thanks for the introduction to your ORM solution, it is nice&straightforward, but I think there is one important step missing (assume it was omitted to keep text short) :

    There is no import statement for the generated PersonDao object, nor its package is mentioned anywhere in the text, it is not clear if the generated class is in the same package as the original Person class, or is put somewhere else (where?).

    Perhaps you could add this detail, it will help readers to comprehend how the stORM works. Thanks and keep the updates to your framework going. Seems its next best thing to have real object database instead of SQL Lite available for Android.

  11. orijit said

    com.turbomanage.storm.api.Persistable cannot be resolved. It does not exist in storm-api-0.94.jar. Am I missing out on something? Also, is it compulsory to have the long id field?

    • Persistable has been removed as it’s no longer necessary for entities to implement it. It is compulsory to have an id field of type long, but it doesn’t have to be named “id”. You can annotate any long field with @Id.

  12. Wondering how to do an “order by COLUMN.NAME desc” with storm

  13. This project is one of the most useful for Android! Thanks a lot!

  14. Prnay said

    Sorry sir . But i am not able to implement Persistable. There is no such interface in stORM Api 0.96
    Any solution ?

  15. Per Kylemark said

    Thanks David for your great work. I tried it yesterday and it is a great tool. I have a problem though. The objects that we are using are rather complex and includes subclasses and also lists of objects. Is there any way that your project could support such classes (or does it today?)? One way could be to implement a relational database structure?

    Do you know of any other way to do this or other packages that might help me?

    kind regards,

    Per Kylemark / Android developer.

    • Thanks, Per. There’s an issue in the tracker to support relations. I’m hoping to get to it this summer. But you will probably still have to traverse the object graph manually. You could do this today by including a parentId in each child class, then querying the child table by parentId.

      I think that GreenDAO has support for relationship modeling, so you could also try that.

      • Per Kylemark said

        Thanks David,

        I actually started that implementation today: I use a row in each other table where I include the top objects ID. The autogeneration of tables is a lifesaver as the object contains a multitude of parameters which are dynamic at this stage in the development. I am looking forward to future releases (contentproviders for complex objects?) and will follow this project closely.

        Per

  16. […] http://turbomanage.wordpress.com/2012/12/11/storm-preview/ […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 227 other followers

%d bloggers like this: