Server/Client apps as an abstraction exercise

My last couple personal/work projects have involved creating remote services and local clients (as well as interfacing with other remote services).  It’s been an interesting exercise in creating well-abstracted interfaces, because 1) network transfer is slow, so you want to limit the amount of data you send, and 2) serializing objects for transfer across processes is much more limiting than calling methods in process (especially the lack of ability to pass lambdas/methods/callbacks easily).  So there’s a lot of pressure to develop well abstracted public APIs for a service.  Here’s what I’ve found:

1. The client shouldn’t have to replicate the server.  IE, the client could ask for every piece of data the server has and run its own queries.  But that would be terribly inefficient.  The server, instead, needs to offer a balance between getting raw content and having methods for common queries (like everything with a certain tag or category).
2. So you end up with a ‘dirtier’ service API than if you were designing a single class- the APIs are inherently larger and have more ‘helpers’ than an in-process class, where the consuming code could just query the data itself to get what it needs (all other things being equal).  There are ways to combat this- like the exposed API being on multiple classes- but the easiest way is just to have the API on one class (the implementations, of course, can and should be broken up to adhere to the Single Responsibility Principle as much as possible).
3. Asynchrony is difficult between server/client.  At least initially, consider making a synchronous service API and hide the asynchrony behind a background thread.  This isn’t ideal because your threads are blocking.  But it is much simpler.  Once things are stable and working, consider making an asynchronous service API.
4. Pick the right framework/setup.  Know your needs.  Is this an internal system?  Is it a local-only service?  Must it communicate across languages or just within one language?  Service/client frameworks are very complex and the simpler you can make your needs, the better.

In the end, the ideal is to have an API that provides everything the client needs but only what the client needs.  “Everything should be made as simple as possible, but not simpler,” says Einstein.  It would also be useful to read up on RESTful APIs (the Wikipedia article is good and probably covers a lot of what you’d learn by trial and error), as well as understanding how things like XMLRPC, JSONRPC, sockets, and CGI work, even (and maybe only) in just an introductory sense.