From: "The Role of Distributed State" by John K. Ousterhout, Computer Science Division Electrical Engineering and Computer Sciences, University of California Berkeley, CA 94720

Why Is Distributed State Good?

Distributed state can be used to provide three benefits in a distributed system:performance, coherency, and reliability. Distributed state improves performance because it makes information available immediately; there is no need to send a mes-sage to a remote machine to retrieve the information. For example, a local table containing name-to-address mappings makes it unnecessary to contact a central nameserver each time a textual name must be mapped to its corresponding address. Or, if a machine caches a remote file in its main memory then the file can be read withoutre-reading the file from disk and potentially without even contacting the server to which the disk is attached.

The second potential advantage of distributed state is coherency. For machines(or people) to work together effectively, they must agree on common goals and coordinate their actions. This requires each party to know something about the other. Forexample, if a host keeps a sequence number identifying the most recent byte of data it received from some other host, and if each arriving packet contains a sequencenumber identifying the first byte of data in that packet, then the receiver can compare sequence numbers to detect when packets are duplicated or arrive out-of-order.Without the sequence number there would be no way to detect these common error conditions, and it would be much more difficult for machines to communicate.Another example is the one from above, where a file server keeps a table of file usage: if one workstation is about to write a file that is cached on several otherworkstations, the file server can notify the other workstations so that they don't use ``stale'' data from their caches.

The third potential advantage of distributed state is reliability. If a particularpiece of information is replicated at several sites in a distributed system and one of the copies is lost due to a failure, then it may be possible to use one of the othercopies to recover the lost information. For example, if a file server crashes but a workstation has one of its files cached, it might be possible for the workstation tomake the file available to the rest of the system while the server reboots; after the server has rebooted it could reclaim jurisdiction over the file.