TurboManage

David Chandler's Journal of Java Web and Mobile Development

  • David M. Chandler


    Web app developer since 1994 and former Developer Advocate with Google now residing in Colorado. Besides tech, I enjoy landscape photography and share my work at ColoradoPhoto.gallery.

  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 231 other followers

  • Sleepless Nights…

    November 2009
    S M T W T F S
    « Oct   Dec »
    1234567
    891011121314
    15161718192021
    22232425262728
    2930  
  • Blog Stats

    • 965,068 hits

Archive for November, 2009

AppEngine needs an AppStore

Posted by David Chandler on November 30, 2009

I continue to be blown away by the power and EASE of Google AppEngine: one-button deploy from Eclipse, all the admin tools built right in (log viewer, data viewer, task queues, cron jobs, etc.), a graphical dashboard for monitoring performance and usage quotas, and even a system status page that lets me know how the Google infrastructure is performing.

I’m convinced this is a killer platform on which to deploy applications, especially for small businesses already moving into the cloud with Google Apps. And wouldn’t you know, you can already link AppEngine and Google Apps in two important ways:

  1. Your AppEngine apps can show up on your Google Apps home page. This happened for me automatically when linking my AppEngine app to a custom domain name using my Google Apps account.
  2. Your AppEngine apps can authenticate Google Apps users.

The only problem is, many of the small businesses likely to use Google Apps are doing so precisely because they don’t have an IT staff to maintain servers, email, etc. So they’re certainly not going to have software developers on staff to write custom AppEngine applications for their businesses. But there are plenty of folks like myself willing to write such apps, and AppEngine is already well suited for delivery of pre-packaged applications. It’s already got the control panel and admin tools built in, doesn’t require any database set up, and it’s easy to upload code.

All that’s needed is a marketplace for these applications. That’s where an AppStore comes in. Imagine if you could just sign up your business for a Google Apps account and select from thousands of pre-built applications for managing your business with the same ease as downloading an app for your iPhone. As a developer, I’d love to market such applications, and what better place to reach them than their Google Apps control panel? I’d be ecstatic to give Google a cut of the action! Besides their cut of the application purchase, Google gets more Google Apps sign-ups and sells more AppEngine resources. Google is already a market maker for small business applications through the core search engine and accompanying ad revenue, but a Google AppStore would be far more efficient for both buyer and seller. An AppStore would make many niche applications economically viable by eliminating fixed costs up front and enabling super-efficient marketing, thus realizing the much-written-about “long tail” of software development.

Google, are you listening? You’ve got something here! AppEngine is fantastic, and an AppStore would take it through the stratosphere!

Advertisements

Posted in AppEngine | 10 Comments »

AppEngine task queue problems resolved for now

Posted by David Chandler on November 27, 2009

I’ve been able to get Deferred.defer() working by Base64 encoding the serialized object stream for the task before queuing it. I think something in the task queue processing chain doesn’t handle raw binary data. I’ll post a link to the code once smarter folks than I have had a chance to discuss this and review it.

You can follow the discussion on the AppEngine forum.

 

Posted in AppEngine | Leave a Comment »

Problems with task queues

Posted by David Chandler on November 25, 2009

As I hoped (sort of), the weird serialization error I wrote about a couple days ago came back, and I don’t think it’s related to a field name. Might do some more testing after turkey tomorrow…

Posted in AppEngine | Leave a Comment »

How to reduce startup time with gwt-presenter

Posted by David Chandler on November 24, 2009

I’ve finally figured out a way to trim down the initial load time for my app. I’m not quite ready for GWT 2, runAsync(), or the new Place API in gwt-presenter, but I’ve been able to cut my initial load time significantly (from 14s to 2s) by doing some lazy loading within my presenters and views. In a nutshell, here’s how it works:

  • I now do nothing in the constructor and bind() methods of each presenter and view.
  • Instead, I’ve created a new method onFirstRequest() in my base presenter. It gets called on the first invocation of onPlaceRequest(), which simply increments a counter in a class field to keep track.
  • I’ve added an init() method to the base presenter’s Display interface. It gets called by onFirstRequest(). I moved all widget creation in my views from the constructor and onBind() methods into this init() method instead, thus delaying all widget creation until the first time the view is accessed.
  • Following a suggestion from Per Wiklander in a recent comment to this blog, I noop’d the addWidget() method in my WidgetContainerDisplay class. This method gets called for each presenter that you add to a WidgetContainerPresenter, which doesn’t make sense for the way I’m using the container (I show only one view at a time). The addWidget() method is not used for anything else.

Since this particular scheme utilizes onPlaceRequest() to keep track of the first invocation, it will only work if you first reveal a view by firing the corresponding PlaceRequentEvent, either in code or by visiting the place URL.

Posted in Google Web Toolkit, Model-View-Presenter | 6 Comments »

GWT-RPC serialization gotchas

Posted by David Chandler on November 23, 2009

In gwt-dispatch, Action and Result classes extend java.io.Serializable; however, I have  bumped up against this error a few times:

Type 'com.your_company...' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded.

I get this error for one of two reasons, and wanted it on my blog so I can search it next time 🙂

1. I forgot to include a no-arg constructor.

2. The type of one or more class fields in the Action or Result class is not Serializable (usually a domain object for which I’ve forgotten to implement Serializable).

As a last resort, delete the *.gwt.rpc files in your war/app directory, clean and rebuild. But the problem is usually one of the above…

Posted in Google Web Toolkit, Headsmack | 4 Comments »

A comprehensive PC backup strategy

Posted by David Chandler on November 21, 2009

“The Saturday Evening post”

Have you ever thought about exactly why you make backups? What are you trying to protect against? Fire? Thieves? Accidental deletion? A few months ago, a friend’s desperate Saturday night phone call vividly reminded me of why I make backups. “David? Um, my daughter was finishing her 200-page novel that’s due Tuesday, and our computer has just died. As in won’t boot. Is there anything you can do?” Of course, they had no backups.

I don’t have the data in front of me to prove it, but experience suggests that the most likely reason you’ll ever need a backup is hard drive failure, with malware /virus infection probably running a close second. Here’s the strategy I’ve developed over the years:

  1. Put all your data files in one directory structure so you can easily copy it. I use a folder named “C:\@My” with subfolders for documents, music, pictures, etc. The @ causes it to show up first in alphabetical directory listings, which is convenient. In Windows Vista (and maybe XP?), you can use this system and still retain the convenience of your Windows user home folder (containing Documents, Music, Pictures, etc.) that shows up in all the standard file dialogs. Simply right-click on, say, David\Documents, click Properties, select the Location tab, and point it to  C:\@My\Documents instead. The reason I don’t just back up my Windows user folder is because Windows stores the whole user profile there, including a lot of stuff that’s not really data per se.
  2. Forget about backing up all your applications. Just keep your original CDs and serial numbers somewhere you can find them. The PC is expendable–your data isn’t. I’m sure there are solutions that will backup the whole operating system and all installed software, but for me it’s not worth the extra time or space it takes to run backups. The exception is applications you’ve purchased via download. I keep these in @My/Downloads along with a text file for each containing the installation key (or find it in Gmail, since it was likely emailed to me when I purchased it).
  3. Each year, less and less of my critical data exists solely on my PC. Nowadays it’s on my iPhone or GMail. I moved my iTunes folder under my @My directory so the phone gets backed up–see Edit | Preferences | Advanced, and I trust Google not lose my mail, though I’m not completely sure why–what if someone hacked my account and deleted all my mail?!?!#@? I guess I really should set up the POP3 account for Gmail with a local email client like Outlook and back that up, too.
  4. At any rate, let’s get to backing up. I back up smaller files like Quicken data (be sure to move this under @My, too), documents, and photos, on a nightly basis using an online backup solution, iDrive.com. With iDrive, I can store up to 2GB for free, and it’s only $50/yr for up to 150 GB. This is my first line of defense against hard drive failure. My data goes off site every night, and I don’t have to think about it.
  5. For larger files like RAW photos and video (as well as small ones), I backup the whole @My folder to an external hard drive every few days using Karen’s Replicator as I blogged about last week. The name of the game here is speed, and for that, you’ll want a hard drive (and computer) with an eSATA interface. I’ve been very happy with my LaCie 1TB external drive. On a recent typical backup run, Karen’s Replicator processed 62,702 files and copied 400 new or modified files totaling 5.5 GB to the LaCie in 8.5 minutes. Most HP and Toshiba laptops now have an eSATA port. You could backup RAW photos and other large files online, too, but DSL upload speeds are so slow that it may be impractical depending on how much new data you create each day.
  6. For large items of sentimental value like photos and videos that are not backed up online, I burn a DVD once a year and drop it in my bank’s safe deposit box in a Ziploc bag (think sprinkler system malfunction). That way, even if I lose both my PC and external drive, I won’t lose more than a year’s worth of digital memories.

Following this strategy, all your data except RAW photos and video always exist in three places: your PC, external hard drive, and in the cloud (online). For extra peace of mind for larger files not backed up online, you could get a second external hard drive and always keep one in your bank’s safe deposit box in a Ziploc bag (think sprinkler system malfunction).

As of this writing, all of my accumulated digital assets take up 168 GB. That already fits easily on a portable hard drive (though I haven’t yet found one with the faster eSATA interface). Within a few years, it will fit on a USB flash drive, I expect. In fact, it would almost fit on this kind of flash drive today. Buy a couple of these along with an eSATA hard drive enclosure from Fry’s and you’ll probably spend more time swapping backup drives at the bank than you will backing up your whole file system every day…

By the way, my friend’s story had a happy ending. I dropped the hard drive containing the novel into a USB hard drive enclosure I bought for $13 at my local Fry’s, plugged it into the USB port on another PC, and recovered all the data. External hard drive enclosures are cool. Fry’s is cool, too.

Posted in PC Tech | 2 Comments »

Weird serialization error in AppEngine

Posted by David Chandler on November 21, 2009

My previous post discussed a very cool way to do background processing in AppEngine using Deferred.defer. Unfortunately, the AdminEmailTask example I posted results in a cryptic error when the task is dequeued and the Deferred servlet attemps to deserialize it:

com.newatlanta.appengine.taskqueue.Deferred deserialize: invalid type code: 00

After nearly a day’s worth of experimentation, I am really weirded out by what I’ve found: the problem seems to be the name of the private field “msgSubject” in AdminEmailTask. I tried shorter names, longer names, and other names without any problem. I changed the class name, moved it into a different package, and generally pulled my hair out until I simply tried changing the name of the field. At first, I thought it was this JDK bug, but I wasn’t using an array type. Nevertheless, I subclassed ObjectInputStream and overrode the resolveClass method as per the bug’s workaround, only to find out that AppEngine’s implementation of ObjectInputStream doesn’t call the overridden resolveClass and throws a security exception on enableResolveObject(true).

I can only guess that the field name “msgSubject” results in a binary sequence that is somehow special to AppEngine’s implementation of ObjectInputStream, or perhaps it’s a bug in the JDK.

At any rate, if you get the java.io.StreamCorruptedException with invalid type code: 00, try renaming your class fields! I really, really hope I’m mistaken about this.

Posted in AppEngine, Headsmack | 1 Comment »

Deferred.defer() is a thing of beauty

Posted by David Chandler on November 20, 2009

How do you do background processing in an AppEngine environment? The AppEngine experimental Task Queues API provides a way using Web hooks, but it’s a bit of a pain because you have to package each background task as a servlet.

Comes Vince Bonfanti of New Atlanta to the rescue with a Java implementation of Nick Johnson’s original Deferred.defer implementation in Python. Now queuing a task for background processing is as simple as creating a task and calling Deferred.defer(task). Here’s an example task to send an email to the application administrator:

package com.roa.server.service.task;

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.util.Properties;

import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import javax.servlet.ServletException;

import com.newatlanta.appengine.taskqueue.Deferred;

	public class AdminEmailTask implements Deferred.Deferrable
	{
		private String msgBody;
		private String msgSubject;

		public AdminEmailTask(String msgSubject, String msgBody)
		{
			this.msgSubject = msgSubject;
			this.msgBody = msgBody;
		}

		@Override
		public void doTask() throws ServletException, IOException
		{
			Properties props = new Properties();
			Session session = Session.getDefaultInstance(props, null);

			try
			{
				Message msg = new MimeMessage(session);
				msg.setFrom(new InternetAddress("admin@example.com", "admin"));
				msg.addRecipient(Message.RecipientType.TO, new InternetAddress(
					"admin@example.com", "admin"));
				msg.setSubject(msgSubject);
				msg.setText(msgBody);
				Transport.send(msg);
			}
			catch (AddressException e)
			{
				throw new ServletException(e);
			}
			catch (MessagingException e)
			{
				throw new ServletException(e);
			}
			catch (UnsupportedEncodingException e)
			{
				throw new ServletException(e);
			}

		}
	}

And here’s an example of queuing the task inside a gwt-dispatch ActionHandler:

package com.roa.server.handler.admin;

import java.io.IOException;

import net.customware.gwt.dispatch.server.ActionHandler;
import net.customware.gwt.dispatch.server.ExecutionContext;
import net.customware.gwt.dispatch.shared.ActionException;

import com.google.inject.Inject;
import com.newatlanta.appengine.taskqueue.Deferred;
import com.roa.admin.shared.rpc.AdminEmailAction;
import com.roa.admin.shared.rpc.AdminEmailResult;
import com.roa.server.service.task.AdminEmailTask;
import com.turbomanage.gwt.server.PMF;

public class AdminEmailHandler implements ActionHandler<AdminEmailAction, AdminEmailResult>
{

	@Inject
	private PMF pmf;

	@Override
	public AdminEmailResult execute(AdminEmailAction action, ExecutionContext ctx)
		throws ActionException
	{
		AdminEmailTask adminEmailTask = new AdminEmailTask(action.getEmailSubject(), action
			.getEmailBody());
		try
		{
			Deferred.defer(adminEmailTask);
		}
		catch (IOException e)
		{
			throw new ActionException(e);
		}
		return new AdminEmailResult();
	}

	@Override
	public Class<AdminEmailAction> getActionType()
	{
		return AdminEmailAction.class;
	}

	@Override
	public void rollback(AdminEmailAction arg0, AdminEmailResult arg1, ExecutionContext arg2)
		throws ActionException
	{
		// TODO Auto-generated method stub

	}

}

You might ask why you’d use a deferred task like this to send an email? In this example, there really is no reason to, but imagine you were sending 1000 emails at once. In order to guarantee you won’t exceed AppEngine’s free quota of 8 recipients / minute, you could set up your deferred queue in queue.xml with that rate and then call Deferred.defer() for each recipient rather than calling the mail API directly 1000 times in a loop. That way, the AppEngine task queuing facility throttles your mail API calls.

Here’s a link to the whole discussion thread and Vince’s code.

If you’d like to see this added to the Task Queues API, please star this issue.

Posted in AppEngine | 6 Comments »

Tips on organizing GWT modules

Posted by David Chandler on November 19, 2009

My current project is at the point where I’m starting to work on an administrative UI. I wanted to package it as a separate GWT module in order to not clutter up the main application module. There is some code such as the domain model that is shared between the main app and the admin code, so the question is how to organize the modules in order to share code between them.

My first attempt was to have the admin module simply inherit the main app module named ROA in admin.gwt.xml, like this:

	<inherits name="com.roa.app.ROA" />

This had two undesired results:

  • Both modules declare an entry-point, so GWT tries to load both, beginning with the inherited module.
  • At GWT compile time, all the code from the inherited module was duplicated under the admin module in the war structure.

My next attempt was to have the admin and app Java packages each point to a third sister package called “common” containing the domain model and other common code. Java is happy with this, but in order for GWT to compile the common code into JavaScript, you must make it a module. So now there are three packages, each with its own module:

admin.gwt.xml
app.gwt.xml
common.gwt.xml

Admin and app each have entry points and inherit the common module, which does not define an entry point. This works fine.

In the common module, I use the source tag to include multiple directories instead of just the default client directory. Note that if you specify one or more source directories, GWT no longer compiles the client directory by default, so you have to explicitly include it, as well:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC "-//Google Inc.//DTD Google Web Toolkit 1.7.1//EN" "http://google-web-toolkit.googlecode.com/svn/tags/1.7.1/distro-source/core/src/gwt-module.dtd">
<module>
	<inherits name="com.google.gwt.user.User" />

	<source path="client" />
	<source path="domain" />
	<source path="presenter" />
</module>

Also, I moved the “public” directory containing CSS and images from the admin and app packages into the common module, and discovered that GWT does not create a “common” directory in the war, but rather copies the contents of the public folder from the inherited common module into both the admin and app module directories. That’s OK, though, as my CSS and images now exist in only one place in source control.

A final note: thank goodness for Eclipse refactoring tools. The easiest way to move a package (say, com.roa.app.domain to com.roa.common.domain) is to select it in the Project Explorer, then right-click, Refactor, and Rename… It’s a bit non-intuitive to use Rename rather than the Move command, but the latter appears to work only for moving packages to another project or source folder, whereas rename allows you to “move” it within the current source folder. Eclipse automatically fixes all the package and import declarations in affected code (whew!).

Posted in Eclipse, Google Web Toolkit, Headsmack | 14 Comments »

The ISV is coming back (thanks to AppEngine)

Posted by David Chandler on November 18, 2009

Someone at last night’s AJUG meeting articulately expressed what I’ve been thinking for the last several weeks as I’ve been learning Google AppEngine: this platform gives the independent software vendor (aka “the computer guy at your church”) a new lease on life. Why will AppEngine bring back affordable custom programming for small business?

  1. There are no servers to configure. Sys admin and tech support are the bane of every developer’s existence, which makes AppEngine a developer’s dream. Code, test, and deploy. No sys admin needed.
  2. There’s nothing to buy up front: no servers, software, firewalls, backup solution–not even hosting fees until you exceed AppEngine’s generous usage limits.
  3. You can log on to your AppEngine application with your Google account (Gmail) or Google Apps login. For many businesses, Google Apps is a compelling alternative to running your own file and document servers and paying for MS Office on every desk. If everyone in your business already has a Google Apps account, it’s a no-brainer to use AppEngine for your custom apps, too.

As more folks learn how to create AppEngine apps in Python or Java, I expect there to be low-cost applications you can purchase for your business as well as pre-packaged applications that developers can buy and customize for your needs. The fabled long tail of software niche marketing is set to EXPLODE.

Shameless plug: if you’ve always dreamed of having software that could do (whatever) for your business, but thought it was probably too expensive, drop me a note. The game has changed.

Posted in AppEngine, Business of Software | Leave a Comment »

 
%d bloggers like this: