Novel Stigmergy Structures in the Internet
The Web is a vast collection of software multicellular organisms. Novel stigmergy structures are growing in the Internet and exerting their organizing power in new ways.
The vast messaging structure called the Internet harbors many
self-organizing
multicellular digital structures. More evolve every year.
Linux source code, for example, can be thought of as a software termite mound, that is, a new kind of digital stigmergy structure. Linux's source code is organized by the “blessed” Concurrent Versions System (CVS) repository that is under the control of the Linux “inner circle” (Linus Torvalds, Alan Cox, etc). Clearly, the Linux CVS code tree is both built by, and helps to organize the efforts of, the worldwide community of Linux programmers. Classic stigmergy!
Before the Open-Source revolution, it was pretty much taken for granted that large-scale, robust software had to be built by a corporation dedicated to its success. The corporation provided a co-located stigmergy structure of servers, programmers, buildings, marketing arms, PR flacks and large financial resources. However, Linux, Apache, MySQL, Mozilla/Firefox and OpenOffice all directly compete with key portions of Microsoft's well funded Windows monopoly. Clearly, an open, distributed software stigmergy structure offers advantages that were previously underestimated.
The Internet supports many other public stigmergy structures that are collectively managed and used by humanity[1]. The entire Web is itself a distributed stigmergy structure constantly being modified and constantly influencing the organization of the humans and computers that use it. The Web as a whole is a living digital stigmergy structure in a sense analogous to the Gaia hypothesis that the entire Earth is a living biological/chemical stigmergy structure.
Examples of emergent digital stigmergy structures in the Internet include:
- Google and other search sites -- consisting of crawlers, databases and servers. These search sites create and maintain meta-level stigmergy structures that adapt to the changing content of the Web. These constantly changing structures in turn affect both the browsing and the remodeling of the Web. The task of keeping up-to-date metadata is massive. Google manages their metadata stigmergy structure(s) on upwards of two million stripped-down Linux servers organized in a multi-cellular (distributed) architecture. The distributed architecture includes apoptosis mechanisms for identifying and removing failed servers.
- Twitter -- a case of a self-organizing system built upon a preexisting data and communication structure. It is an organization layer based, at least initially, upon cell phone SMS text messages. Texting itself is an organization layer proposed in 1985 over an already existing secondary low bandwidth radio channel between cell-phones and cell towers that had been used only to communicate data to phones about reception strength and to supply them with information about incoming calls. In 1985, the GSM telephony standards committee, led by Friedhelm Hillebrand, proposed the 160 character text message protocols. Twitter is now used over the Web on media other than just cell-phone texting. And a large searchable database of tweets has been accumulating -- it is, in effect the digital world's diary. It will be archived forever by the US Library of Congress. On April 14, 2010, @librarycongress tweeted, “Library to acquire ENTIRE Twitter archives -- All public tweets, ever, since March 2006!” The archives are quite large. Twitter processes more than 50 million tweets every day.
- MMORPGs -- Massively Multiplayer Online Role Playing Games such as World of Warcraft, EverQuest, Second Life, and many others are based on servers that maintain "models" (in the Model-View-Controller sense) acting as the stigmergy structures around which tens of thousands of online gamers organize their play. The game model is altered in real-time by the behavior of the players and the player's behavior is altered by the constantly changing model. WoW has at least 6.5 million users and claim that they have supported over 250 thousand simultaneous users.
- Instant messaging communities (AOL, MSN, Yahoo, and others) where the stigmergy structure is the “presence registry” that tracks who is online at any given moment. By the act of logging-in to an Instant Messaging community, each user deposits information into the stigmergy structure: the IP address of the users machine and perhaps other status information e.g., do-not-disturb. Their community of friends can know that they are now accessible. People who use instant messaging recognize how their behavior is modified by being aware in "real time" of their friend's and colleague's online presence.
- Public databases. For example, most medical and post-genomic bioinformatics research world-wide is completely dependent upon public databases such as GenBank, PDB, Pub Med, and BIND[2]. Most academic and/or public affairs disciplines, where data is usually assumed to be a public good, have similar sorts of communities organized around major databases. These databases, are stigmergy structures in which the data is updated by the users and, in turn, helps to organize the subsequent behavior of those same users. The databases often support easy-to-use, yet sophisticated domain-specific search functions. Social incentives, such as mandates set granting agencies are encouraging researchers to put their new data into these databases and most researchers focus much of their activity around use of that public data.
- Free
VOIP and video conferencing services (e.g., Skype) are similar to the
instant
messaging communities except that they communicate with higher
bandwidth voice or video rather than text messages. The stigmergy
structure
includes not only a presence registry, but also may use the combined
bandwidth
of the users machines in a peer-to-peer model.
- Peer-to-peer (P2P) file sharing networks (e.g., Gnutella, Freenet, and BitTorrent), construct a virtual distributed stigmergy structure supported by a constantly shifting collection of client machines with the appropriate set of P2P message APIs. Each participating machine makes available some files. Together they collaborate to maintain an ever-changing distributed presence registry that enables individual machines to search out desired files. Music file sharing communities, together with the new end user iPod/iPhone devices, are reshaping the way people relate to music and thus reshaping the entire music industry. They threaten to do the same for the movie industry.
- Blogs and Wikis are other examples of public stigmergy “selves.” Wikipedia, for example, is an open encyclopedia with over 1.1 million entries in the English version that is created and maintained by users all over the world. It's stigmergy structure evolves minute-by-minute, 24 hours a day, as the human "termites" add and modify its pages. The content is supported and served by more than 120 servers running Linux and Apache.
- Social networking sites such as MySpace and Facebook help form and organize human digital social groups.
- "folksonomy" sites such as Flikr and del.icio.us collect and organize tags for unstructured data. Folksonomy stigmergy structures consists of both the data (e.g., photos or bookmarks) and the database of tags that support search. Thus, while the data and the tags are each stigmergy structures, their interdependence strengthens both.
Note that all of these novel communities are organized around new stigmergy structures, i.e., new digital selves, of a sort that didn’t exist previously. Their worldwide scope, speed, and enablement of new kinds of interaction emerge out of the scope, speed, and protean flexibility of the Internet. They also typically operate in a climate that is diametrically opposed to the tendencies corporations have to hoard and leverage secret or proprietary knowledge.
Open Internet-based systems mutate and grow by the self-organizing activity of large numbers of users that share a stigmergy structure. They mutate and evolve more rapidly than corporate IT infrastructures. For example, P2P networks grow organically because their infrastructure grows at the same rate as their “customer” base. Each user contributes some resources to the network in exchange for participating in the network. Thus Skype, a free VOIP network, grew very rapidly in its early stages with almost no expenditure of its own resources while the Telcos struggled to adapt. Similarly, public databases, once accepted in their relevant communities, can grow much more rapidly than can a competing proprietary database. Many a bioinformatics startup company has found it impossible to provide proprietary for-fee databases in competition with free public databases.
[1] “humanity” is an overstatement. Only the digitally connected can participate. But cell-phones are making inroads even in rural Africa and India, and smart-phones will eventuall follow. The reach of the net is threatened, however, by attempts to balkanize the Internet into censored and controlled subnets, e.g., in countries such as Singapore, China, Saudi Arabia, North Korea and Iran.
[2] GenBank manages DNA sequence data, PDB manages protein structure data, Pub Med manages biological and medical publications, and BIND manages protein-protein interaction data. There are hundreds of other useful bioinformatics databases freely available on the Web.
Contact: sburbeck at mindspring.com
Last revised 6/11/2012