[NEC] 1.1 - Communities, audiences, and scale

[list-replies@shirky.com]

NEC @ Shirky.com, a mailing list about Networks, Economics, and Culture 

           Published periodically / # 1.1 / April 6, 2002 
              Subscribe at http://shirky.com/nec.html

In this issue:

 - Introduction: Welcome to NEC
 - Essay: Communities, Audiences, and Scale: Limits on community size
   in online environments
  (Also at http://www.shirky.com/writings/community_scale.html)
 - Things worth reading
 - Questions I'm asking myself

* Introduction ======================================================

Welcome to the inaugural issue of NEC. This will be a low-volume
periodical, with my writings on various aspects of networks,
economics, and culture, as well as pointers to other things I think
are worth reading, and reader comments on the material presented here.

The first issue is devoted to the question of communities, audiences,
and scale: what effects can we expect to see from growth in the size
of online groups, given that the internet supports both broadcast and
communal patterns of communication?

This is a big subject, one that touches several ideas about the shape
of human networks, including especially David Reed's theories of group
value on networks, and Duncan Watts' work on Small Worlds networks.
What follows is not an attempt to bring all the the possible models
into a single essay. Instead, it's an exploration of one basic but
important effect: Audiences scale, communities don't.

-clay
clay@shirky.com

* Essay =============================================================

COMMUNITIES, AUDIENCES, AND SCALE
  (http://www.shirky.com/writings/community_scale.html)

Prior to the internet, the differences in communication between
community and audience was largely enforced by media -- telephones
were good for one-to-one conversations but bad for reaching large
numbers quickly, while TV had the inverse set of characteristics. The
internet bridged that divide, by providing a single medium that could
be used to address either communities or audiences. Email can be used
for conversations or broadcast, usenet newsgroups can support either
group conversation or the broadcast of common documents, and so
on. Most recently the rise of software for "The Writable Web",
principally weblogs, is adding two-way features to the Web's largely
one-way publishing model.

With such software, the obvious question is "Can we get the best of
both worlds? Can we have a medium that spreads messages to a large
audience, but also allows all the members of that audience to engage
with one another like a single community?" The answer seems to be "No."

Communities are different than audiences in fundamental human ways,
not merely technological ones.  You cannot simply transform an audience
into a community with technology, because they assume very different
relationships between the sender and receiver of messages.

Though both are held together in some way by communication, an
audience is typified by a one-way relationship between sender and
receiver, and by the disconnection of its members from one another --
a one-to-many pattern. In a community, by contrast, people typically
send and receive messages, and the members of a community are
connected to one another, not just to some central outlet -- a
many-to-many pattern [1]. The extreme positions for the two patterns
might be visualized as a broadcast star where all the interaction is
one-way from center to edge, vs. a ring where everyone is directly
connected to everyone else without requiring a central hub.

As a result of these differences, communities have strong upper limits
on size, while audiences can grow arbitrarily large. Put another way,
the larger a group held together by communication grows, the more it
must become like an audience -- largely disconnected and held together
by communication traveling from center to edge -- because increasing
the number of people in a group weakens communal connection. 

The characteristics we associate with mass media are as much a product
of the mass as the media. Because growth in group size alone is enough
to turn a community into an audience, social software, no matter what
its design, will never be able to create a group that is both large
and densely interconnected. 

- Community Topology

This barrier to the growth of a single community is caused by the
collision of social limits with the math of large groups: As group
size grows, the number of connections required between people in the
group exceeds human capacity to make or keep track of them all.

A community's members are interconnected, and a community in its
extreme position is a "complete" network, where every connection that
can be made is made. (Bob knows Carol, Ted, and Alice; Carol knows
Bob, Ted, and Alice; and so on.) Dense interconnection is obviously
the source of a community's value, but it also increases the effort
that must be expended as the group grows. You can't join a community
without entering into some sort of mutual relationship with at least
some of its members, but because more members requires more
connections, these coordination costs increase with group size.

For a new member to connect to an existing group in a complete fashion
requires as many new connections as there are group members, so
joining a community that has 5 members is much simpler than joining a
community that has 50 members. Furthermore, this tradeoff between size
and the ease of adding new members exists even if the group is not
completely interconnected; maintaining any given density of
connectedness becomes exponentially harder as group size grows. As new
members join, it creates either more effort or lowers the density of
connectedness, or both, thus jeopardizing the interconnection that
makes for community. [2]

As group size grows past any individual's ability to maintain
connections to all members of a group, the density shrinks, and as the
group grows very large (>10,000) the number of actual connections
drops to less than 1% of the potential connections, even if each
member of the group knows dozens of other members. Thus growth in size
is enough to alter the fabric of connection that makes a community
work. (Anyone who has seen a discussion group or mailing list grow 
quickly is familiar with this phenomenon.)

An audience, by contrast, has a very sparse set of connections, and
requires no mutuality between members. Thus an audience has no
coordination costs associated with growth, because each new member of
an audience creates only a single one-way connection. You need to know
Yahoo's address to join the Yahoo audience, but neither Yahoo nor any
of its other users need to know anything about you. The disconnected
quality of an audience that makes it possible for them to grow much
(much) larger than a connected community can, because an audience can
always exist at the minimum number of required connection (N
connections for N users).

- The Emergence of Audiences in Two-way Media

Prior to the internet, the outbound quality of mass media could be
ascribed to technical limits -- TV had a one-way relationship to its
audience because TV was a one-way medium. The growth of two-way media,
however, shows that the audience pattern re-establishes itself in one
way or another -- large mailing lists become read-only, online
communities (eg. LambdaMOO, WELL, ECHO) eventually see their members
agitate to stem the tide of newcomers, users of sites like slashdot
see fewer of their posts accepted. [3]

If real group engagement is limited to groups numbering in the
hundreds or even the thousands [4], then the asymmetry and
disconnection that characterizes an audience will automatically appear
as a group of people grows in size, as many-to-many becomes
few-to-many and most of the communication passes from center to edge,
not edge to center or edge to edge.  Furthermore, the larger the
group, the more significant this asymmetry and disconnection will
become: any mailing list or weblog with 10,000 readers will be very
sparsely connected, no matter how it is organized. (This sparse
organization of the larger group can of course encompass smaller, more
densely clustered communities.)

- More Is Different

Meanwhile, there are 500 million people on the net, and the population
is still growing. Anyone who wants to reach even ten thousand of those
people will not know most of them, nor will most of them know one
another. The community model is good for spreading messages through a
relatively small and tight knit group, but bad for reaching a large
and dispersed group, because the tradeoff between size and
connectedness dampens message spread well below the numbers that can
be addressed as an audience.

It's significant that the only two examples we have of truly massive
community spread of messages on the internet -- email hoaxes and
Outlook viruses -- rely on disabling the users' disinclination to
forward widely, either by a social or technological trick. When
something like All Your Base or OddTodd bursts on the scene, the
moment of its arrival comes not when it spreads laterally from
community to community, but when that lateral spread attracts the
attention of a media outlet [5].

No matter what the technology, large groups are different than small
groups, because they create social pressures against community
organization that can't be trivially overcome. This is a pattern we
have seen often, with mailing lists, BBSes, MUDs, usenet, and most
recently with weblogs, the majority of which reach small and tightly
knit groups, while a handful reach audiences numbering in the tens or
even hundreds of thousands (e.g. andrewsullivan.com.)

The inability of a single engaged community to grow past a certain
size, irrespective of the technology, will mean that over time,
barriers to community scale will cause a separation between media
outlets that embrace the community model and stay small, and those
that adopt the publishing model in order to accommodate growth. This is
not to say that all media that address ten thousand or more people at
once are identical; having a Letters to the Editor column changes a
newspaper's relationship to its audience, even though most readers
never write, most letters don't get published, and most readers don't
read every letter.

Though it is tempting to think that we can somehow do away with the
effects of mass media with new technology, the difficulty of reaching
millions or even tens of thousands of people one community at a time
is as much about human wiring as it is about network wiring. No matter
how community minded a media outlet is, needing to reach a large group
of people creates asymmetry and disconnection among that group --
turns them into an audience, in other words -- and there is no easy
technological fix for that problem. 

Like the leavening effects of Letters to the Editor, one of the design
challenges for social software is in allowing groups to grow past the
limitations of a single, densely interconnected community while
preserving some possibility of shared purpose or participation, even
though most members of that group will never actually interact with
one another.

-=-

- Footnotes

1. Defining community as a communicating group risks circularity by
ignoring other, more passive uses of the term, as with "the community
of retirees." Though there are several valid definitions of community
that point to shared but latent characteristics, there is really no
other word that describes a group of people actively engaged in some
shared conversation or task, and infelicitous turns of phrase like
'engaged communicative group' are more narrowly accurate, but fail to
capture the communal feeling that arises out of such engagement. For
this analysis, 'community' is used as a term of art to refer to groups
whose members actively communicate with one another.

2. The total number of possible connections in a group grows
exponentially, because each member of a group must connect to every
other member but themselves. In general, therefore, a group with N
members has N x (N-1) connections, which is the same as N**2 - N.
If Carol and Ted knowing one another count as a single relationship,
there are half as many relationships as connections, so the relevant
number is (N**2 - N)/2.

Because these numbers grow exponentially, every 10-fold increase in
group size creates a 100-fold increase in possible connections; a
group of ten has about a hundred possible connections (and half as
many two-way relationships), a group of a hundred has about a thousand
connections, a thousand has about a million, and so on. The number of
potential connections in a group passes a billion as group size grows
past thirty thousand.

3. Slashdot is suffering from one of the common effects of community
growth -- the uprising of users objecting to the control the editors
exert over the site. Much of the commentary on this issue, both at
slashdot and on similar sites such as kuro5hin, revolves around the
twin themes of understanding that the owners and operators of slashdot
can do whatever they like with the site, coupled with a surprisingly
emotional sense of betrayal that the community control, in the form of
moderation. 

(More at kuro5hin, http://www.kuro5hin.org/story/2002/1/17/21155/1564, 
and slashdot, http://slashdot.org/comments.pl?sid=26315&cid=2850660)

4. In _Grooming, Gossip, and the Evolution of Language_ (ISBN
0674363361), the primatologist Robin Dunbar argues that humans are
adapted for social group sizes of around 150 or less, a size that
shows up in a number of traditional societies, as well as in present
day groups such as the Hutterite religious communities. Dunbar argues
that the human brain is optimized for keeping track of social
relationships in groups small than 150, but not larger.

5. In _The Tipping Point_ (ISBN 0316346624), Malcolm Gladwell detailed
the surprising spread of Hush Puppies shoes in the mid the '90s, from
their adoption by a group of cool kids in the East Village to a
national phenomenon.  The breakout moment came when Hush Puppies were
adopted by fashion designers, with one designer going so far as to
place a 25 foot inflatable Hush Puppy mascot on the roof of his
boutique in LA. The cool kids got the attention of the fashion
designers, but it was the fashion designers who got the attention of
the world, by taking Hush Puppies beyond the communities in which it
started and spreading them outwards to an audience that looked to the
designers.

-=-

Thanks to NYU's Information Law Institute, http://www.law.nyu.edu/ili/, 
who commented on an earlier version of this argument, and to Jessica
Hammer for editorial review.

* Things worth reading  =============================================

- Journalistic pivot points, Dan Gillmor's observations on
  participatory journalism, sparked by blogging at PC Forum in real
  time, and being corrected live, from the stage, by a panelist who
  was reading his blog.

  http://www.siliconvalley.com/mld/siliconvalley/business/columnists/dan_gillmor/ejournal/2946748.htm

- Duncan Watts work on Milgram's "Small Worlds/6 Degrees of
  Separation" experiments:

  His book: _Small Worlds_, 1999, ISBN 0691005419

  His current re-creation of the Milgram experiment:
    http://smallworld.sociology.columbia.edu/index.html

  Evidence that the Milgram experiment was more anomalous than it
  first appeared to be:
    http://smallworld.sociology.columbia.edu/history.html

- Interesting speculation about slashdot's further transformation into
  a more traditional news outlet, with the libel laws as a possible spur.  
    http://slashdot.org/comments.pl?sid=28874&cid=3100604

* Questions I am asking myself  =====================================

- What shape will the blogosphere take?

The blogosphere is already resolving itself into a power law
distribution, with many small blogs (small in terms of readership),
some moderately sized blogs, and a handful of highly trafficked
blogs. This in turn recreates all the difficulties the original web
users had in locating content, which created niches for search
engines, directories, portals, et al. Many of these solutions don't
work well in the blogosphere, however, because weblog content is
time-sensitive.

What shape will the blogosphere take? Will blogdex and daypop be
enough, or will readers flock towards trusted meta-blogs that filter
and organize, or will other models of collaborative filtering arise?

- What killed Usenet? 

So many people predicted usenet's demise for so long that its
hardiness became a running joke: Death of Usenet Predicted; Film at
11. Then it died.

Despite its technological hardiness, and the dozen years during which
it was so important that journalists routinely mistook it for the
internet itself, it is now awash in spam and irrelevant for all but a
few uses, in all but a few groups. What killed it?

- What questions are you asking yourself?

* End ===============================================================

Copyright 2002, Clay Shirky
Feel free to reprint, quote, or forward, so long as you credit me.