TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Peru. I am currently offering public and private developer training courses in the US and Latin America as well as working on Android, GWT, and App Engine projects.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 227 other followers

  • Sleepless Nights…

    October 2014
    S M T W T F S
    « Jul    
     1234
    567891011
    12131415161718
    19202122232425
    262728293031  
  • Blog Stats

    • 664,190 hits

Archive for the ‘AppEngine’ Category

Painless REST+JSON API with Jersey and RestyGWT

Posted by David Chandler on July 23, 2014

It’s been a while since I’ve been active in GWT programming, but I recently picked up an old GWT + App Engine project and dusted it off. This time around, I need a service layer that will work with native mobile apps as well as the GWT desktop app: in other words, a REST+JSON API.

The server side

I decided to use Jersey on the server, which makes it incredibly easy to create REST services, especially in combination with the fabulous objectify-appengine. Here’s one of my services:

@Path("api/user")
@Produces(MediaType.APPLICATION_JSON)
public class UserDao extends ObjectifyDao<User>
{
    @GET
    @Path("me")
    public User getMe() {
        return AuthFilter.getUser(); // long live ThreadLocal!
    }
}

All the other standard CRUD methods (list, get, delete, etc.) are inherited from a generic DAO. A servlet filter verifies user authentication, XSRF tokens, etc. At only one class per entity, I find this much more agreeable than the four classes per entity I used to write with gwt-dispatch (Action, Result, Handler, DAO).

The GWT REST+JSON client

On the client side, GWT developers have long needed something which combines:

  1. The convenience of GWT-RPC (automatic serialization of pretty much everything)
  2. JSON data format
  3. Simple verb-based service APIs
  4. Benefits of the command pattern such as request caching and batching
  5. Minimal boilerplate

RestyGWT delivers on all counts.

The interface is actually simpler than GWT-RPC because you don’t have to define a synchronous and asynchronous method, only async (although you can reuse the server interface and call it using DirectRestService if it helps you sleep better). Here’s a sample interface using RestyGWT. This is a generic CRUD API which I simply extend for each entity class.

public interface RestApi<T> extends RestService {

    @GET
    @Path("own")
    public void getForOwner(MethodCallback<T> callback);

    @GET
    @Path("get")
    public void get(@QueryParam("id")Long id, MethodCallback<T> callback);

    @GET
    @Path("all")
    public void listAll(MethodCallback<ListResponse<T>> callback);

    @POST
    @Path("save")
    public void save(T obj, MethodCallback<T> callback);

    ...
}

Here’s a sample service API and the GWT code that calls it:

public class UserPrefsService
{
    private static final UserPrefsRestService service = GWT.create(UserPrefsRestService.class);

    @Path("/api/userPrefs")
    public interface UserPrefsRestService extends RestApi<UserPrefs> {
        // just in case you missed it, all the CRUD methods are inherited
    }

    private UserPrefs prefs;

    public void loadUserPrefs()
    {
        service.getForOwner(new AppCallback<UserPrefs>() {
            @Override
            public void handleSuccess(UserPrefs result) {
                prefs = result;
                App.getEventBus().fireEvent(new UserPrefsLoadedEvent(prefs));
            }
        });
    }
}

I really like the benefits of the Command pattern from the old gwt-dispatch framework such as the ability to do caching and queuing centrally. The downside of the Command pattern is the proliferation of classes associated with turning every API method into two or more classes (in the case of gwt-dispatch, an Action and Result class for every service method). GWT’s RequestFactory (the GWT team’s “final answer” to RPC after GWT-RPC and deRPC) used generators to remove some of the boilerplate, but I wasn’t in love with having to call someSingleton.getRequestFactory().someRequest().create(BeanProxy.class) each and every time I wanted a new Bean. And it uses a proprietary data format, not REST+JSON.

This is where RestyGWT really shines. It lets you define simple API interfaces using methods, but uses a Command pattern dispatcher under the covers with an ingenious filter mechanism for adding capabilities like XSRF protection, caching, and retries. The magic is possible thanks to RestyGWT’s generator classes which transform your annotated service methods into MethodRequest classes that get sent by a dispatcher. To add caching capability to all your APIs, for example, just put this in your GWT onModuleLoad():

DispatcherFactory factory = new DispatcherFactory();
org.fusesource.restygwt.client.Defaults.setDispatcher(factory.cachingDispatcher());

For a look at some of the other dispatcher options, have a look at the RestyGWT User Guide and the example DispatchFactory class.

Tips for success with Jersey + RestyGWT

Just a couple final notes to ease your migration to Jersey + RestyGWT.

If you’re using RestyGWT with Jersey + Jackson on the server, you’ll want to set the default date format to avoid Date serialization errors. Put this in your GWT’s onModuleLoad():

Defaults.setDateFormat(null);

Similarly, recent versions of Jersey complain about “self-referencing cycles” or some such with Objectify’s Ref properties, so you’ll probably want to annotate those in your server-side entities with @JsonIgnore. If you need the id of a related object, use a differently named getter instead. Jersey will create a corresponding JSON property. Example of an Objectified entity on the server:

package com.my.server.domain;

@Entity
public class UserPrefs {
    @Id
    private long id;
    @JsonIgnore
    private Ref<User> ownerKey;
    ...
    public long getOwnerId() {
        return ownerKey.get().getId();
    }
}

Then on the client you can reference the ownerId property:

package com.my.client.domain;

public class UserPrefs
{
    public Long id;
    public long ownerId;
    ...
}

I would prefer to use the same object representation on client and server, but RestyGWT makes using DTOs about as painless as possible. The client-side object is a class not an interface, so you can create a new instance anywhere you need it vs. having to GWT.create(BeanProxy.class). I could probably hack Jersey and maybe objectify-appengine to automatically replace all Refs with a long id, but honestly, it’s easy enough just to copy the server entities and replace Refs with long ids on the client.

Also, here’s a pair of classes you may find useful for sending any type of list from server to client. For security purposes, a JSON response should always return a root object, not an array directly. The ListWrapper (server) and ListResponse (client) serve as this root object, having a single field containing the list.

package com.my.server.domain;

/**
 * Wraps a List<T> in a JSON root object.
 */
public class ListWrapper<T> {
    private List<T> list;

    public ListWrapper(List<T> list) {
        this.list = list;
    }
    public List<T> getList() {
        return list;
    }
}

Use it like this in one of your Jersey-annotated methods:

    @Path("all")
    @GET
    public ListWrapper<Subscription> findAll() {
        User user = AuthFilter.getUser();
        List<Subscription> userAll = this.listByOwner(user);
        return new ListWrapper<Subscription>(userAll);
    }

Using ListWrapper this way has the additional advantage of causing Objectify to fetch any lazily loaded properties such as Refs used in the getOwnerId() method above while producing the JSON. If you were to return the List directly, this would not occur!

Here’s the corresponding representation on the client:

package com.my.client.domain;

public class ListResponse<T> {

    public List<T> list;

}

See the RestApi example earlier for how it’s used.

Summary

Jersey and RestyGWT in combination make a powerful and easy way to create a REST API for your GWT client and mobile apps, combining all the benefits of a request dispatcher with straightforward service interfaces and minimal boilerplate.

Happy coding!

Posted in AppEngine, Google Web Toolkit | 9 Comments »

GWT, App Engine, maven, and… IntelliJ!

Posted by David Chandler on June 5, 2014

I’ve been putting off migrating to Android Studio for a while now because I’m frankly loathe to learn a new IDE. I’ve used Eclipse for a decade and grew to become very productive in it. But a completely different team at Google may have just well forced me into it.

When I joined the GWT Developer Relations team in 2010, I worked closely with the Google Plugin for Eclipse team to get maven support into GPE. With their excellent work, we eventually achieved the holy grail: you could import a POM containing maven-gae-plugin and gwt-maven-plugin into Eclipse and all the GWT + GAE stuff from GPE like launching dev mode would just work (well, if you had the right supporting plugins like m2e-wtp). So the other day, I picked up a POM that worked in those days and tried it out on Kepler + GPE + m2e-wtp. Amazingly enough, the GWT stuff still works. Google now supports its own appengine-maven-plugin, so I swapped out the old GAE plugin for the new. It’s supposed to work with GPE and WTP, but so far no dice. It will be great when they get the kinks worked out. Funny thing is, I remember seeing exactly the same problem with GAE-maven integration way back in my GWT days. Only then, I could just walk over to Rajeev’s desk and he would fix it :-)

In the mean time, imagine my surprise to discover that I could just import the POM into IntelliJ (full edition) and everything works. The maven project imported, I can launch GWT dev mode, set breakpoints and debug, etc. It’s funny to me that JetBrains can keep up to date with Google App Engine better than the GPE team, but that’s how big companies move sometimes….

So I’m off to learning new keyboard shortcuts (I could use the Eclipse keymap, but I had customized Eclipse, too. It’s easier than I thought to learn new tricks). And I have to say, the performance of IntelliJ is impressive. A couple of my teammates way back at Intuit will no doubt be glad to hear that I finally came around to maven + IntelliJ, not to mention some of my Android DevRel mates. We’ll see how this goes, but so far, I’m impressed.

 

Posted in AppEngine, Eclipse, Google Web Toolkit | 1 Comment »

Android + Cloud with Mobile Backend Starter

Posted by David Chandler on May 21, 2013

The Google Cloud Solutions team has made it easier than ever to create a cloud backend for your Android application. Brad Abrams and I presented it last week at Google I/O, and Brad has published a gigantic step-by-step blog post with copious screenshots on how we built our sample app, Geek Serendipity. Here’s all the related content in one handy list:

Posted in Android, AppEngine | 8 Comments »

Troubleshooting App Engine-connected Android projects with c2dm

Posted by David Chandler on May 18, 2012

Google Plugin for Eclipse has a wizard to create an Android project with App Engine back end including client push (c2dm). The official tutorial is quite good, but after spending the better part of a day trying to get the Cloud Tasks sample up and running again, I thought it might be useful to post a few tips.

General Tips

  • Sign up for c2dm first. Remember the package name and use it when running the Eclipse wizard! The package name is required, and once you choose the package name on c2dm sign-up, it is nowhere visible later. Be patient–it may take an hour or even a day to activate your account.
  • Sign in to Eclipse with the c2dm account name you’ll use (see lower left corner–if not visible, you need Google Plugin for Eclipse). It’s easy to forget this, but Debug As | Local App Engine Connected Project requires you to be signed in.

Run on a Phone With the GAE Dev Server

In this scenario, the phone and dev server must be on the same network! Easiest way is to connect to the same wi-fi router. USB cable does nothing. Be aware that most public wi-fi routers don’t allow devices to contact each other, so it will probably work best at home or on your company’s guest wi-fi network.

  • Ensure that phone and dev server are on the same wi-fi network. Disconnect both from any VPNs.
  • Add -bindAddress 0.0.0.0 to your Web project’s Run Configuration | Program Arguments. This will start the server listening on all addresses, not just the default localhost (which the phone can’t see).
  • When you choose Debug As | Local App Engine Connected Android Application, GPE hardcodes your machine’s IP address in assets/debugging_prefs.properties. This is where the Android app looks up the address of the server.
  • You can launch/debug the Android app and the Web app independently; however, if your server IP address changes, you must again Debug As | Local App Engine Connected Android Application in order to update the IP address in assets/debugging_prefs.properties.
  • In case you wondered, the Android and Web projects are associated in a file in the Android project. You can find it in .settings/com.google.gdt.eclipse.mobile.android.prefs. To associate a different App Engine project with an Android project, simply edit this file and refresh the project.
  • If you’re using the GWT dev mode browser plugin, you can still connect to 127.0.0.1. If you connect to the address shown in Development Mode tab (your machine’s network IP address), you may need to allow the address in the dev mode browser plugin. Click the GWT icon in the address bar to do this.
  • Make sure you log in to the Web app as the c2dm user, not the default test@example.com.

Run on a Phone Against the Production App

In the production scenario, you’re using real c2dm so the phone doesn’t have to be on wi-fi at all.

  • Edit Setup.java and change APP_NAME and SENDER_ID to match your GAE app id and c2dm account.
  • Change app id in appengine-web.xml to match your GAE app.
  • Deploy to App Engine with your c2dm account.
  • Force sign in to GAE app by going to [prod_url]/tasks/. This triggers the security constraint in web.xml to get logged in to GAE. After logging in, you’ll get an error page, but that’s OK. Then go to [prod_url]/ and Say Hello to App Engine. If logged in, you should see a success message.
  • If you’ve previously connected an account on a phone to this production app, go to GAE datastore viewer and delete all records.
  • In Eclipse, Debug as | Remote App Engine Connected Android project
  • On the phone, you’ll be prompted to connect an account. Use your c2dm account. If you have previously connected a different account, go to Menu | Accounts | Disconnect in the app and reconnect.

The CloudTasks project still has a few issues. Feel free to log them in the tracker.

Posted in Android, AppEngine, Google Web Toolkit | 7 Comments »

App Engine, Dart, and Android at DevNexus

Posted by David Chandler on March 23, 2012

DevNexus keynote

DevNexus keynote

First, a huge thank you to the volunteer organizers and sponsors of DevNexus! The Atlanta JUG‘s annual not-for-profit conference attracted 500 mostly Java developers this year and remains one of the best values of any developer conference in the world.

I gave an update on Google App Engine for Java (see slides) and a pitch for HTML5 + Dart (see slides).

There were a lot of talks on mobile / Web development this year, including:

  • JQuery Mobile framework (Raymond Camden, Adobe)
  • Intro to PhoneGap (Raymond Camden, Adobe)
  • Java-Powered Mobile Cross-Platform Mobile Apps with Phone Gap (Andrew Trice)
  • Web vs. Apps keynote (Ben Galbraith)
  • Easy Mobile Development: Intro to Appcelerator Titanium (Pratik Patel)
  • What’s New in Android (Robert Cooper)
  • Spring for Android (Roy Clarkson, Spring)
  • Mind-Blowing Apps with HTML5 Canvas (David Geary)

All the mobile / Web talks that I attended were standing room only (70-130 people, depending on room capacity). It seems that mobile in the enterprise is red hot and enterprise apps are good candidates for Web apps vs. native. I was particularly impressed with

Thanks again to Gunnar, Vince, Sudhir, and the gang for making DevNexus such a high-quality event.

Posted in Android, AppEngine, Dart | 6 Comments »

Will it play in App Engine page moved

Posted by David Chandler on December 21, 2011

Just noticed that the old “Will it play in App Engine Java” page has moved to the appengine wiki. This was necessary because Google Groups deprecated groups pages. It’s also very out of date.

Posted in AppEngine | 1 Comment »

App Engine code download saves the day

Posted by David Chandler on August 26, 2011

While I was working on the sample project for yesterday’s post, I accidentally deleted the src and target folders in my project. I’m not entirely sure how this happened, but my source files were neither in the Trash folder nor Eclipse local history (except for the Maven POM, which I had just been working on). I was just getting ready to check into SVN but hadn’t done it yet, so it was beginning to look like a total loss.

Fortunately, I had just deployed my project to App Engine, which now allows you to download your deployed application. Java sources don’t get deployed so I had only WEB-INF/classes to work with, but a quick trip through JAD recovered the sources also (albeit sans comments–sorry about that, folks). I like this new GAE feature. In the future, I’ll at least create a local git repo before I start mucking with the Eclipse Run configs…

Posted in AppEngine | 1 Comment »

Using the GAE App Identity API and OAuth2

Posted by David Chandler on August 25, 2011

One of the challenges of cloud computing is interoperability: how can an application hosted on one cloud service authenticate and communicate with another cloud app or service? Within the enterprise, IT departments often utilize a central identity store like PingFederate from Ping Identity. How to manage identities in the cloud, however?

Google App Engine recently unveiled one solution: the Application Identity API, which provides your app with public key infrastructure managed by Google. Each App Engine app has a unique identity in the form of an email address and public/private key pairs which are rotated every few hours. The API lets your app discover its own identity, obtain the current public keys, and sign an object using the private key. The beauty of the API is that you the developer don’t have to manage the keys at all. Google handles all the PKI, crypto, rotation, and security.

Recently, at the Cloud Identity Summit sponsored by Ping Identity, I presented an example of using the Application Identity API to connect from App Engine to Google Storage as part of a 3-hr workshop on Integrating with the Google Cloud (see slides 149-153). I’ve now published a sample project showing how a GAE app can authenticate to the Google Storage API using OAuth2 and the App Identity API.

How to call Google Storage API from App Engine Java

Let’s say you’ve created a Google Storage account (free within limits) and want to access it from your Java code in App Engine. First, you need to set up your Google Storage account appropriately:

  1. Add your GAE app’s unique ID (aka “robot” email address) to Team in Google APIs Console.
    • to find it, call ApplicationIdentityService.getServiceAccountName() in your GAE app
    • assign write or owner access
    • use in bucket ACLs also
  2. Note your Google Storage project ID (click Google Storage in the left nav of the Google APIs Console)

Now in your Java code, you’ll

  1. Obtain your app’s unique ID with ApplicationIdentityService.getServiceAccountName()
  2. Create a JWT token with issuer ID = your app’s unique ID
  3. Sign it with ApplicationIdentityService.signForApp()
  4. Exchange it for an OAuth token by calling Google’s OAuth endpoint (which, in turn, calls App Engine infrastructure to verify your app’s identity).
  5. Include the OAuth token with each request to Google Storage API

Two different sample projects illustrate how to do this. You might want to start with GoogleStorageImpl.java showing all the low-level calls to create a JWT token and exchange it for an authentication credential.

Once you understand how it works, you can extend AbstractTwoLeggedFlowServlet to handle the token exchange for you. This is the approach I’ve taken in the App Identity Sample with OAuth2 and Google Storage. As an aside, the Maven POM in this project is a good starting point if you’re looking to call Google APIs from Java. There are lots of jars associated with Google APIs Client Library for Java available in Maven Central, and it can be confusing to figure out which ones are really needed.

Note: the sample projects above will only work if you actually deploy the code to App Engine. Google’s OAuth2 endpoint only works with production infrastructure (not your local dev server).

This article is about application identity, but if you’re also interested in user identity, be sure to check out the Google Identity Toolkit, which makes it easy for you to support login to your app from other identity providers (Yahoo, Hotmail, AOL).

Resources

Posted in AppEngine | 11 Comments »

Sending HTML emails with App Engine and Velocity in a Maven project

Posted by David Chandler on July 6, 2011

I’ve been using the App Engine Mail API for a while now to send emails in plain text format, but recently decided to beef it up to allow HTML formatting. In addition, I wanted a template-based approach so I could more easily modify the email contents. To achieve both purposes, I chose Apache Velocity, an easy-to-use, general purpose framework that lets you easily create anything textual from a template (text, email, XML, even Java code once on a previous project).

The main subtlety to observe when using Velocity with App Engine is that by default, Velocity uses the filesystem resource loader to load templates. This works fine in the App Engine development server and should also work in production as long as you put your templates in the WAR. Since my project is using Maven, however, the standard way to do this is to put templates in src/main/resources and let Maven automatically copy them to WEB-INF/classes during the build. In order for Velocity to see them on the classpath, you need Velocity’s ClasspathResourceLoader, so I created a helper class that lazily initializes the Velocity engine with the required resource loader:

/**
 * Singleton that initializes Velocity engine instance with the classpath resource loader
 * 
 * @author David Chandler
 */
public class VelocityHelper {
  private static VelocityEngine velocityEngine;

  static VelocityEngine getVelocityEngine() {
    if (velocityEngine == null)
      init();
    return velocityEngine;
  }

  private static void init() {
    velocityEngine = new VelocityEngine();
    Properties velocityProperties = new Properties();
    velocityProperties.put("resource.loader", "class");
    velocityProperties.put("class.resource.loader.description", "Velocity Classpath Resource Loader");
    velocityProperties.put("class.resource.loader.class",
        "org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader");
    velocityEngine.init(velocityProperties);
  }
}

To use the template, first populate the VelocityContext with variables to be replaced, then merge the template:

VelocityContext context = new VelocityContext();
context.put("date", today);
context.put("username", "johndoe");
VelocityEngine ve = VelocityHelper.getVelocityEngine();
// Finds template in WEB-INF/classes
template = ve.getTemplate("emailTemplate.vm");
StringWriter writer = new StringWriter();
template.merge(context, writer);

Since we’re sending an HTML email, the template is just an HTML file named emailTemplate.vm (by convention–.vm stands for Velocity macro). The template merge substitutes variables like ${date} with their values from the VelocityContext populated above. You can also loop over Collections with #foreach ($item in $list) … #end.

Now let’s send the email. There are a couple things to observe here.

First, webmail readers like Gmail write the HEAD and BODY tags to the page, so the browser will likely ignore these tags in your HTML template. This means you can’t link a stylesheet in the HEAD section of your HTML and expect it to work across email clients. The most compatible approach is to use inline styles, that is, the style=”” attribute on various HTML tags right in your template.

Second, in order to send HTML email, you must set the MIME type correctly. Here is the code that takes the template output, sets the MIME type, and sends the email:

import java.util.Properties;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;

...

Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
try
{
	Message msg = new MimeMessage(session);
	msg.setFrom(from);
	msg.addRecipient(Message.RecipientType.TO, to);
	msg.setSubject(msgSubject);
	if (msgBody.startsWith("<!DOCTYPE html"))
		msg.setContent(msgBody, "text/html");
	else
		msg.setText(msgBody);
	Transport.send(msg);
}
catch (AddressException e)
{
	throw new ServletException(e);
}
catch (MessagingException e)
{
	throw new ServletException(e);
}

That’s it in a nutshell. Spam away…

/dmc

Posted in AppEngine | 7 Comments »

Configuring the App Engine development server

Posted by David Chandler on June 30, 2011

Last week, I posted a way to preserve the local Datastore when using Maven by setting a system property datastore.backing_store=path. Although it’s not explicitly documented, it appears that many dev mode services can be configured this way. The configuration flags are documented with the local unit testing Javadocs like this one for LocalDatastoreService. Click on the property, then the link to Constant Field Values to find the associated String you can use in a -D argument as shown in last week’s post. Note that unit tests can configure services directly by calling setters on the test helper like LocalDatastoreServiceTestConfig.

Happy tweaking…

Posted in AppEngine | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.

Join 227 other followers

%d bloggers like this: