TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Peru. I am currently offering public and private developer training courses in the US and Latin America as well as working on Android, GWT, and App Engine projects.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 227 other followers

  • Sleepless Nights…

    October 2014
    S M T W T F S
    « Jul    
     1234
    567891011
    12131415161718
    19202122232425
    262728293031  
  • Blog Stats

    • 672,862 hits

Painless REST+JSON API with Jersey and RestyGWT

Posted by David Chandler on July 23, 2014

It’s been a while since I’ve been active in GWT programming, but I recently picked up an old GWT + App Engine project and dusted it off. This time around, I need a service layer that will work with native mobile apps as well as the GWT desktop app: in other words, a REST+JSON API.

The server side

I decided to use Jersey on the server, which makes it incredibly easy to create REST services, especially in combination with the fabulous objectify-appengine. Here’s one of my services:

@Path("api/user")
@Produces(MediaType.APPLICATION_JSON)
public class UserDao extends ObjectifyDao<User>
{
    @GET
    @Path("me")
    public User getMe() {
        return AuthFilter.getUser(); // long live ThreadLocal!
    }
}

All the other standard CRUD methods (list, get, delete, etc.) are inherited from a generic DAO. A servlet filter verifies user authentication, XSRF tokens, etc. At only one class per entity, I find this much more agreeable than the four classes per entity I used to write with gwt-dispatch (Action, Result, Handler, DAO).

The GWT REST+JSON client

On the client side, GWT developers have long needed something which combines:

  1. The convenience of GWT-RPC (automatic serialization of pretty much everything)
  2. JSON data format
  3. Simple verb-based service APIs
  4. Benefits of the command pattern such as request caching and batching
  5. Minimal boilerplate

RestyGWT delivers on all counts.

The interface is actually simpler than GWT-RPC because you don’t have to define a synchronous and asynchronous method, only async (although you can reuse the server interface and call it using DirectRestService if it helps you sleep better). Here’s a sample interface using RestyGWT. This is a generic CRUD API which I simply extend for each entity class.

public interface RestApi<T> extends RestService {

    @GET
    @Path("own")
    public void getForOwner(MethodCallback<T> callback);

    @GET
    @Path("get")
    public void get(@QueryParam("id")Long id, MethodCallback<T> callback);

    @GET
    @Path("all")
    public void listAll(MethodCallback<ListResponse<T>> callback);

    @POST
    @Path("save")
    public void save(T obj, MethodCallback<T> callback);

    ...
}

Here’s a sample service API and the GWT code that calls it:

public class UserPrefsService
{
    private static final UserPrefsRestService service = GWT.create(UserPrefsRestService.class);

    @Path("/api/userPrefs")
    public interface UserPrefsRestService extends RestApi<UserPrefs> {
        // just in case you missed it, all the CRUD methods are inherited
    }

    private UserPrefs prefs;

    public void loadUserPrefs()
    {
        service.getForOwner(new AppCallback<UserPrefs>() {
            @Override
            public void handleSuccess(UserPrefs result) {
                prefs = result;
                App.getEventBus().fireEvent(new UserPrefsLoadedEvent(prefs));
            }
        });
    }
}

I really like the benefits of the Command pattern from the old gwt-dispatch framework such as the ability to do caching and queuing centrally. The downside of the Command pattern is the proliferation of classes associated with turning every API method into two or more classes (in the case of gwt-dispatch, an Action and Result class for every service method). GWT’s RequestFactory (the GWT team’s “final answer” to RPC after GWT-RPC and deRPC) used generators to remove some of the boilerplate, but I wasn’t in love with having to call someSingleton.getRequestFactory().someRequest().create(BeanProxy.class) each and every time I wanted a new Bean. And it uses a proprietary data format, not REST+JSON.

This is where RestyGWT really shines. It lets you define simple API interfaces using methods, but uses a Command pattern dispatcher under the covers with an ingenious filter mechanism for adding capabilities like XSRF protection, caching, and retries. The magic is possible thanks to RestyGWT’s generator classes which transform your annotated service methods into MethodRequest classes that get sent by a dispatcher. To add caching capability to all your APIs, for example, just put this in your GWT onModuleLoad():

DispatcherFactory factory = new DispatcherFactory();
org.fusesource.restygwt.client.Defaults.setDispatcher(factory.cachingDispatcher());

For a look at some of the other dispatcher options, have a look at the RestyGWT User Guide and the example DispatchFactory class.

Tips for success with Jersey + RestyGWT

Just a couple final notes to ease your migration to Jersey + RestyGWT.

If you’re using RestyGWT with Jersey + Jackson on the server, you’ll want to set the default date format to avoid Date serialization errors. Put this in your GWT’s onModuleLoad():

Defaults.setDateFormat(null);

Similarly, recent versions of Jersey complain about “self-referencing cycles” or some such with Objectify’s Ref properties, so you’ll probably want to annotate those in your server-side entities with @JsonIgnore. If you need the id of a related object, use a differently named getter instead. Jersey will create a corresponding JSON property. Example of an Objectified entity on the server:

package com.my.server.domain;

@Entity
public class UserPrefs {
    @Id
    private long id;
    @JsonIgnore
    private Ref<User> ownerKey;
    ...
    public long getOwnerId() {
        return ownerKey.get().getId();
    }
}

Then on the client you can reference the ownerId property:

package com.my.client.domain;

public class UserPrefs
{
    public Long id;
    public long ownerId;
    ...
}

I would prefer to use the same object representation on client and server, but RestyGWT makes using DTOs about as painless as possible. The client-side object is a class not an interface, so you can create a new instance anywhere you need it vs. having to GWT.create(BeanProxy.class). I could probably hack Jersey and maybe objectify-appengine to automatically replace all Refs with a long id, but honestly, it’s easy enough just to copy the server entities and replace Refs with long ids on the client.

Also, here’s a pair of classes you may find useful for sending any type of list from server to client. For security purposes, a JSON response should always return a root object, not an array directly. The ListWrapper (server) and ListResponse (client) serve as this root object, having a single field containing the list.

package com.my.server.domain;

/**
 * Wraps a List<T> in a JSON root object.
 */
public class ListWrapper<T> {
    private List<T> list;

    public ListWrapper(List<T> list) {
        this.list = list;
    }
    public List<T> getList() {
        return list;
    }
}

Use it like this in one of your Jersey-annotated methods:

    @Path("all")
    @GET
    public ListWrapper<Subscription> findAll() {
        User user = AuthFilter.getUser();
        List<Subscription> userAll = this.listByOwner(user);
        return new ListWrapper<Subscription>(userAll);
    }

Using ListWrapper this way has the additional advantage of causing Objectify to fetch any lazily loaded properties such as Refs used in the getOwnerId() method above while producing the JSON. If you were to return the List directly, this would not occur!

Here’s the corresponding representation on the client:

package com.my.client.domain;

public class ListResponse<T> {

    public List<T> list;

}

See the RestApi example earlier for how it’s used.

Summary

Jersey and RestyGWT in combination make a powerful and easy way to create a REST API for your GWT client and mobile apps, combining all the benefits of a request dispatcher with straightforward service interfaces and minimal boilerplate.

Happy coding!

Posted in AppEngine, Google Web Toolkit | 9 Comments »

Debug an Android annotation processor with gradle and IntelliJ (or Eclipse)

Posted by David Chandler on June 9, 2014

Alex Gherschon and I have recently added maven support to storm-gen (note: I plan to push to Maven Central this week). You can now build and install storm-gen to your local maven repository with

git clone https://github.com/turbomanage/storm-gen.git
cd storm-gen/storm-apt
mvn clean install

Since gradle can pull in maven artifacts, you can then include storm-gen in your gradle build file like this:

apply plugin: 'android'
apply plugin: 'android-apt'


android {
    compileSdkVersion 19
    buildToolsVersion "19.0.3"

    defaultConfig {
        minSdkVersion 10
        targetSdkVersion 19
        versionCode 1
        versionName "1.0"
    }
    buildTypes {
        release {
            runProguard false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}

dependencies {
    compile 'com.android.support:appcompat-v7:+'
    compile 'log4j:log4j:1.2.17'
    compile 'javax.persistence:persistence-api:1.0'
    compile 'com.turbomanage.storm:storm-api:0.98'
    apt 'com.turbomanage.storm:storm-impl:0.98'
}

Now here’s the fun part. Previously, in order to debug and set breakpoints in the annotation processor itself, I used Eclipse RCP and ran it as an Eclipse plugin. Fortunately, there’s now an easier way using Java remote debugging. There are several variations on this technique (configuring an annotation processor directly in IntelliJ or debugging a gradle script). Unfortunately, Android Studio rejected my adding the Xdebug JVM args directly to the gradle script. However, I found this workaround. It is not fully integrated with AS, but does allow me to debug the annotation processor on demand by running the gradle script.

Add these lines to your ~/.gradle/gradle.properties:

org.gradle.daemon=true
org.gradle.jvmargs=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005

Then do something that causes the gradle daemon to start.

gradle --daemon

Now you can go into IntelliJ or Android Studio and attach a remote debugger. I use IntelliJ because AS won’t import the storm-gen maven projects and the whole point of this exercise is to set breakpoints in the storm-gen annotation processor. The screenshot shows configuring a remote debugger in IntelliJ. Just accept the defaults and make sure the port number matches your gradle.properties. It should also work in Eclipse.

remote debug config

Configure a remote debugger in IntelliJ

The remote debugger should attach to the running gradle daemon. You can then tickle the annotation processor by running a build. For example, this runs the storm-gen tests on an already-running emulator or connected device:

cd storm-test
gradle clean connectedCheck

When the annotation processor runs, your breakpoint in IntelliJ (or Eclipse) should fire.

One caution with this technique is that Android Studio will also use the gradle daemon because it’s now configured in your gradle.properties. But if you launched the daemon from the command line, AS will try to launch it again and you’ll see an error message about the port already in use. In that case, simply kill the running gradle process and AS should be happy.

Posted in Android, IntelliJ | 1 Comment »

GWT, App Engine, maven, and… IntelliJ!

Posted by David Chandler on June 5, 2014

I’ve been putting off migrating to Android Studio for a while now because I’m frankly loathe to learn a new IDE. I’ve used Eclipse for a decade and grew to become very productive in it. But a completely different team at Google may have just well forced me into it.

When I joined the GWT Developer Relations team in 2010, I worked closely with the Google Plugin for Eclipse team to get maven support into GPE. With their excellent work, we eventually achieved the holy grail: you could import a POM containing maven-gae-plugin and gwt-maven-plugin into Eclipse and all the GWT + GAE stuff from GPE like launching dev mode would just work (well, if you had the right supporting plugins like m2e-wtp). So the other day, I picked up a POM that worked in those days and tried it out on Kepler + GPE + m2e-wtp. Amazingly enough, the GWT stuff still works. Google now supports its own appengine-maven-plugin, so I swapped out the old GAE plugin for the new. It’s supposed to work with GPE and WTP, but so far no dice. It will be great when they get the kinks worked out. Funny thing is, I remember seeing exactly the same problem with GAE-maven integration way back in my GWT days. Only then, I could just walk over to Rajeev’s desk and he would fix it :-)

In the mean time, imagine my surprise to discover that I could just import the POM into IntelliJ (full edition) and everything works. The maven project imported, I can launch GWT dev mode, set breakpoints and debug, etc. It’s funny to me that JetBrains can keep up to date with Google App Engine better than the GPE team, but that’s how big companies move sometimes….

So I’m off to learning new keyboard shortcuts (I could use the Eclipse keymap, but I had customized Eclipse, too. It’s easier than I thought to learn new tricks). And I have to say, the performance of IntelliJ is impressive. A couple of my teammates way back at Intuit will no doubt be glad to hear that I finally came around to maven + IntelliJ, not to mention some of my Android DevRel mates. We’ll see how this goes, but so far, I’m impressed.

 

Posted in AppEngine, Eclipse, Google Web Toolkit | 1 Comment »

Android Camp in the Rockies

Posted by David Chandler on April 29, 2014

Photo of Android green robot in RMNP

Join me in the Rockies Aug 25-29

Calling all mountain lovers who’ve been thinking about learning Android development…

Ever feel like you’re just a bit too ADD to spend all week in a nondescript classroom and pay big bucks to stay in a downtown hotel where there’s not much to do? Does your brain hurt after working exercise after tedious exercise using courseware that adds little value over the API docs? Do you gain weight eating on the expense account without much exercise?

Me too. Besides these problems, in my past training experience I’ve found that many students find it difficult to maintain concentrated technical attention 8 hours at a time (yeah, it happens in the workplace, too). For myself, I find that I frequently want take a walk, preferably somewhere more refreshing than the parking lot or around the downtown block.

So I started looking around for venues in more refreshing places. I thought of Denver, Boulder, and Colorado Springs (see the pattern?). But then I realized that as long as my mind would still be in the mountains, I might as well put my body there, too.

So this is a bit of an experiment. I’m offering a 5-day Android bootcamp with lodging and meals included in one of the most refreshing spots on earth, YMCA of the Rockies Estes Park Center. We have about 3 solid days of material, so we can cover it in 3, 4, or 5 days, and work around the weather for hiking. In the evenings, you can head into the bustling town of Estes Park or hang out in the meeting room to work on an app project with others in the group. We’ll have some special activities, too, like a BBQ or ice cream social. Because most schools in the US now start in early Aug, the last two weeks see a thinning out of the crowds. Which makes it a great time to visit RMNP, especially if you want to access the high trails free of snow.

Over the five days, we will learn the basics of Android development with an emphasis on enterprise apps. If you finish with a lab early or need a short break, you can play mini-golf or go elk-spotting in the woods at the center. During longer breaks, you can grab a buddy and go for a hike in Rocky Mountain National Park. I know the trails quite well and look forward to sharing them with you if you’re so inclined. If you’re really nice, I’ll talk you to the spot where I took the photo above. Bring your tripod for HDR :-)

For further description, syllabus, cost, etc., see the full event listing on EventBrite below.

See you in Colorado!

Eventbrite - Android Bootcamp in the Rockies Aug 18-22

Posted in Uncategorized | Leave a Comment »

Ruh-roh, I think my brain is re-wiring

Posted by David Chandler on April 26, 2014

Spanish is starting to sound natural to me. This is very, very good news. But English… I can hardly spit this out: I’ve been making spelling mistakes. With increasing frequency, Gmail or Google Docs has put a red squiggly under a word after I’ve moved on the next one. At first I questioned why the spellchecker didn’t have certain words in the dictionary. But as it’s kept happening, I’m afraid there might be a better explanation. Spelling mistakes are very much a new thing for me (go ahead and laugh–please remember that I was very disappointed when I came runner up in the 3rd or 4th grade spelling bee). The other night, it took me until the next morning to remember the instinctive spelling of “legitimate.” For a long time, I sat staring at the red squiggly. I tried all kinds of things that didn’t make the squiggly go away and only the next day while I was walking downtown or maybe sitting in the kitchen, the word suddenly sort of appeared in my brain and it was obvious again.

Spelling isn’t all that’s wrong, however. I routinely find, upon re-reading what I’ve written, that left a word out or put in one the wrong order. This is very troubling! I have rarely needed to double-check my writing for grammar and spelling mistakes. No the purpose of revision is to smooth any rough edges [ok, this is getting ridiculous--the errors in the second sentence were for fun, but I really did forget the comma after "No" just now]. There are two possible explanations for this phenomenon. One is that I’m getting old. Of course, I know better than that. The second is that as I learn Spanish, my brain is somehow re-wiring. If that truly is the case, then I am left in complete awe of the Creator of the human mind. The capacity for learning our first language is a miracle, especially when you consider that two year olds do it. But the capacity to learn a second language at age 43, well… it blows my mind.

Posted in Uncategorized | 4 Comments »

storm-gen has moved

Posted by David Chandler on March 20, 2014

In order to more easily support pull requests, I’ve moved storm-gen, the annotation-based open source ORM for Android SQLite, to Github. You can now find it here:

https://github.com/turbomanage/storm-gen

I’ve migrated all open issues to the Github tracker and, following six months of language school in Peru, have recently resumed work on the framework. Thanks to Alex G, we’re about to add support for maven, gradle, and Android Studio. I’m also adding more operators and the long-awaited .order() method to FilterBuilder.

Also, there is now a G+ community for feature discussion and a StackOverflow tag.

Enjoy!

Posted in Uncategorized | Leave a Comment »

Open for business in Peru

Posted by David Chandler on February 18, 2014

This is my last week in language school (at least until I need a refresher course) and with it come some big changes. My family has settled in to life in Arequipa, at least to the extent that can be expected after five months in a foreign country speaking a new language. I have received my work visa and have started a Peruvian company to focus on training / consulting for software developers.

My first client / partner is Lima-based Tekton Labs, which has hired me to provide technical training and architectural guidance to the cloud and mobile development teams. I am looking forward to working with the team for several reasons:

  1. Tekton has a talented team and strongly believes in personal development. This is very important because, while there are a few good computer science programs in the country, there is no substitute for continuing education in our field.
  2. The company has a mature, agile software development process (Scrum), which in my view is a prerequisite for taking on large projects.
  3. Tekton has a sizable portfolio of work for large enterprises, but still offers significantly lower prices than US-based companies.

In the US, some of the most coveted jobs in programming are with Internet and software companies. In Peru, on the other hand, they are in finance and telecommunications because they typically pay the best ($500 – $2,000 / mo. + benefits). Unfortunately, this means there are few cloud and mobile jobs here; however, there ARE talented developers with the necessary training as well as self-taught hackers just like in the US. My aim is to help provide further training for these developers and to connect them with US-based companies like startups who may have great ideas, but can’t afford US-based labor. Over the next year or two, I’ll keep you posted on how it’s going. In the mean time, if you have an idea which might be a fit for a Peruvian development team, ping me on G+ and let’s talk.

Posted in Business of Software | 3 Comments »

A long pause

Posted by David Chandler on October 21, 2013

IMG_5806_DxOHello faithful readers! I apologize for my long absence on these pages. I have not fallen off the edge of cyberspace, just below the equator. In the preceding months, my family has sold most of our earthly possessions (save the small electronic ones :-)) and moved to Peru. We are currently attending language school in fulfillment of a lifetime goal to become fluent in Spanish. I am currently on unpaid leave with status to be determined after language school. Until then, I have precious little time for coding, but hope to squeeze in some work on storm-gen shortly and to continue writing about Android, App Engine, GWT, and other topics as time permits.

At some point, I’d also like to write more about how technology has facilitated our move, but suffice it to say that a scanner, Google Drive, Google Play Books, the Kindle app, and a host of phones and tablets make possible a near weightless existence, which is extremely helpful when conducting an international move. Thankfully, Internet connectivity (both mobile and fixed) is quite good in the major cities of Peru, so our “everything in the cloud” strategy has worked well thus far, although my wife doesn’t think the kitchen made the move very well in the cloud.

Posted in Uncategorized | 6 Comments »

Android + Cloud with Mobile Backend Starter

Posted by David Chandler on May 21, 2013

The Google Cloud Solutions team has made it easier than ever to create a cloud backend for your Android application. Brad Abrams and I presented it last week at Google I/O, and Brad has published a gigantic step-by-step blog post with copious screenshots on how we built our sample app, Geek Serendipity. Here’s all the related content in one handy list:

Posted in Android, AppEngine | 8 Comments »

Android demo tips: behind the scenes at Google I/O

Posted by David Chandler on May 20, 2013

Last week at Google I/O, Brad Abrams and I decided to go for the gold and give a talk on building a cloud backend for your Android app, complete with live coding on stage. For a more interesting example, we chose to build a sample app which relies on location services and Google Cloud Messaging. It all seemed like a good idea at the time. For the benefit of other Android presenters, I wanted to give a behind-the-scenes view of how we overcame some significant challenges to pull off the demos. Also, I will explain the anomalies we saw.

Device display. The easiest way to show an app is using an emulator, but Maps V2 API is part of Google Play Services, which doesn’t currently run on the Android emulator, so we needed physical devices. Droid@Screen works, but it updates slowly and requires the USB cable, which we needed for hardwired Ethernet. Some devices support HDMI out (Galaxy Nexus, N10) which can be used with a BlackMagic or similar HDMI capture box to show the device display on your Mac screen. Note that this requires a Mac with two Thunderbolt ports (hello Retina! BlackMagic in, Mac display adapter out). We could not use this because the phone’s HDMI connector is shared with the USB connector, which we needed for hardwired Ethernet. Fortunately, we had a Wolfvision projector camera available and a 4-port video switcher to show our Macs or the camera. For best results with a projector, set the phone display brightness to max.

The network. Gather 5000+ geeks in a massive concrete structure, give them all a new wifi device, and then try to get a working Internet connection… Despite Google’s stellar efforts, getting wireless Internet at Moscone Center remains very challenging. 3G / 4G / LTE is inhibited by the concrete and steel, and even 5GHz wifi suffered from RF interference or sheer demand. Thus, we knew in advance that we would need hardwired Internet. This was no problem for our laptops, but we also needed 2 phones for the demos to show continuous queries in action. Fortunately, Galaxy Nexus phones support wired connections using a USB OTG adapter paired with a USB to Ethernet adapter, and it just worked. So far so good.

Deferred launch. Because we were coding live in Eclipse, we deployed the code to the phones in real time via the USB cable. When you run an Android app in Eclipse, the default action is to install and run the app. But this  created a problem for the simple version of our demo app, which needed an Internet connection on launch to send a location update immediately. We had to disconnect the USB cable, walk the phone over to the camera, plug in the hardwired Ethernet, and only then launch the app via the home screen. Fortunately, Eclipse has a way to support this. In the project’s run configuration on the Android tab, simply select “Do nothing” for the launch action. This installs the app, but doesn’t launch it.

Live coding. Speaking of live coding, we have all seen those demos with lots of tedious cut and paste operations from a text file to the IDE. As a presenter, I want something visible in the IDE to prompt me for the fully manually steps as well as an easy way to paste in blocks of code. Eclipse snippets are just the thing. You can show them off to the side of your main editor window, then double-click on a snippet to paste it in. I used snippet titles without code to prompt me for the manual steps. Another challenge of live coding is that you might make a mistake at an early stage of the demo which ruins a later stage. In order to facilitate recovery, we created snapshots of working demo code at six different stages during our presentation. It would have been nice to preinstall these app versions on the devices; however, they all use the same package name and the Android installer doesn’t allow more than one at the same time. Hence the need for adb install and the deferred launch technique. Also we had to uninstall the prior version of the app on the device before each coding stage. Furthermore, Eclipse doesn’t let you create multiple projects with the same name. There is a workaround, but it kind of spoils the magic of our talk and I’ve already spilled enough secrets… let’s just say Eclipse workspaces and “Copy project” are a beautiful thing.

Location services. This is the one that snagged us in the first demo. It turns out that location services are not in fact magic, but require GPS (in Moscone?!) or wifi, which we turned off just before speaking to eliminate the possibility of conflict with the hardwired Ethernet. Fortunately, we were able to get a good enough wifi signal in a later demo to show location. On desktop devices, geolocation works with a hard-wired connection, so I’m not yet sure whether this is a bug or a feature on mobile… it is admittedly an uncommon use case.

The unanticipated. After recovering from the first demo glitch by turning on wifi, I demonstrated that location services worked, but it never sent the location to the server. It turns out the code was working as intended–I had already enabled authentication for my next segment, but the app from the previous segment which I revisited did not have auth turned on. Finally, the last demo showed two markers when it should have showed only one. The second marker turned out be a fellow Googler in the audience who had helped dogfood an early version of the demo app. So once again, the code worked as intended, just not as I intended at the moment. But… this is the stuff live demos are made of. I still prefer live demos with a few glitches to an hour full of slides :-)

So now you know what Brad and I were busily doing while the other was talking and why talks of this complexity are pretty much impossible without two people and a lot of presentation gear. It was a teeny bit intimidating knowing that our session was part of the Google I/O live stream with thousands of viewers, but in the end, it came off (almost) without glitches and our gracious audience was wowed by the Mobile Backend Starter project. Mission accomplished!

See my next post for a complete list of resources on Mobile Backend Starter.

Posted in Android, Eclipse, Headsmack | 10 Comments »

 
Follow

Get every new post delivered to your Inbox.

Join 227 other followers

%d bloggers like this: