The vicious cycle of remote procedure call APIs

Lately I’ve been working with remote procedure calls (RPC). My experience is that they tend to be more trouble than they are worth, but I thought I’d give it another look. RPC is a way to program in which you write a program as if it were run on one computer, but some of the commands are run on a remote computer.

At its most basic level, RPC is just message passing. You send the message “what is 2+2?” to a computer, and you get back the message “4.” In fact, message passing is all computer networking ever is. Message passing isn’t hard, it simply doesn’t look like normal programming. So RPC decorates message passing to look better. And when you are working with complicated data, it can make life easier.

The problem is that there seems to be an inevitable slippery slope of feature creep, which ends up making RPC unusable. The cycle seems to be:

  1. Start with general-purpose network protocol du jour, sending messages from a client to a server.
  2. That’s not enough like regular programming, so add wrappers to let you send arbitrary commands to the server.
  3. Oh, and the server should be able to send commands back to the client.
  4. But wait! That’s not secure! Add security protocols.
  5. (Optional) We don’t want to be restricted to one programming language; make it multi-language.
  6. How do we know what the server can do? Let’s add a directory for finding services. Better yet, let’s add a directory for finding all servers– and maybe a directory for finding other directories!
  7. This is unusable. Let’s start over with the new general-purpose protocol du jour.

Case in point #1 CORBA. Before CORBA, you used specific protocols (FTP, SNMP, etc.) which could be coerced to trigger remote actions. Even today, you can unsubscribe from an email list by sending “unsubscribe” email to a particular address. If those weren’t good enough, you passed C structures (massaged to be network-safe) from a custom client to a custom server. Then CORBA came around, and over time it obtained all of the features listed above. To make it multi-language, the CORBA spec required you to name your commands commands so they looked funny in most languages but would work in every language. And most implementations were buggy and expensive. But the killer for CORBA was that it required ORBs (service directories) on specific ports, and those ports are blocked on firewalls by default. When I worked at Net Perceptions, we discovered that many of our customers couldn’t use our product because the IT department which controlled the firewall was five layers of management away.

The solution was to piggy-back on port 80, the web server port. That is, run RPC through a web server. Someone had recently written an article on how much easier RPC was if it was done as XML messages through a web server. And that’s exactly what we did. Worked like a charm.

Next thing you know, a committee was formed to standardize SOAP (Simple Object Access Protocol) for XML over RPC. Sun and Microsoft announced that Java and Windows would both support SOAP. Everyone joined the bandwagon. Pretty soon SOAP was just as unwieldy as CORBA. Talking to eBay from Java yields 700+ auto-generated classes!

The fact of the matter is that for most things you want to use RPC for you use (1) a server you write yourself talking to (2) a client you write yourself, with (3) a small set of commands, which are known in advance. Directory services are almost universally a boondoggle– yet they are nearly always a required part of the protocol. Multi-language support is rarely needed, and if you do need it, the easiest way to do it is through a simple, clear message protocol; RPC auto-translation of one language’s structures into another yields ugliness in both languages.

My prediction is that in the next iteration someone will discover that you can use AJAX for general-purpose RPC. The new protocol will use all the so-called Web 2.0 technologies, including RSS feeds. A committee will be called together to standardize mash-up-style RPC. JavaScript (ewww…) will be used for sending arbitrary commands hither and yon.

Side note:

I’ve just spent a week trying to coerce Java RMI (Remote Method Invocation) to do what I want. As RPC goes, it’s better than most. But there are a few special requests I need to make of it, each of which doubles the complexity. And now I’m at a pont where I’ll have to add a special security policy to the Java runtime, which just plain isn’t feasible for this project. I could do it, but it would make our web server far more fragile. That or spend another day or two learning the bowels of Java security. So I’m about to rip out all my new code and go back to a simple socket-based message passing protocol.