Re: <nettime> DARPA digests openBSD [cramer, hwang]

[Florian Cramer <cantsin@zedat.fu-berlin.de>]

Francis Hwang <sera@fhwang.net> worte:

> It's an interesting wrinkle, DARPA funding OpenBSD research. Makes me 
> wonder what the folks over at the NSA thinks about this -- they were 

(See SELinux...) 

> A bit of background might be useful for the less tech-obsessed on this 
> list. (Others please correct me if I err.) BSD, like Linux, is a Unix 
> variant, 

Yes and no: BSD ("Berkeley System Distribution") was first released in
1977 by Bill Joy, author of the vi editor and more recently of the essay
"Why the Future Doesn't Need Us". Initially, BSD was a patched version
of AT&T's original Unix operating system, extending it - among other
things - into a network operating system with a TCP/IP stack, Sendmail,
network printing, DNS/bind etc.. Many of those extensions
became software infrastructure of the emerging Internet, 
much of them were also gradually merged into the official AT&T Unix
releases.

>From the beginning on, BSD code was released under the liberal (almost
public domain) license you described. Ironically or not, its freedom
was an immediate effect of its DARPA funding. According to Brett
Glass, "Darpa's research grants meant that if you could pay for the
computer tape, you could get the BSD portion, with source code, for
free." <http://www.extremetech.com/article2/0,3973,555443,00.asp>.
As BSD co-developer Marshall Kirk McKusick writes it
in his informative essay "Twenty Years of Berkeley
Unix - From AT&T-Owned to Freely Redistributable"
<http://www.oreilly.com/catalog/opensources/book/kirkmck.html>, DARPA
continued to fund BSD even in the 1980s: "DARPA was sufficiently
satisfied with the results of the first contract that a new two-year
contract was granted to Berkeley with funding almost five times that
of the original." This contract paid the work to turn BSD into an
ARPAnet/Internet operating system, according to McKusick: "To assist in
defining the new system, Duane Adams, Berkeley's contract monitor at
DARPA, formed a group known as the 'steering committee' to help guide
the design work and ensure that the research community's needs were
addressed."

However, BSD wasn't usable as an operating system of its own without the
Unix code owned by AT&T until the early 1990s; according to McKusick,
"up through the release of 4.3BSD-Tahoe [in 1988], all recipients of BSD
had to first get an AT&T source license."

As McKusick and
<http://mail-index.netbsd.org/netbsd-advocacy/2001/01/18/0017.html>
tell, the first distribution of BSD code that could be used without an
AT&T license was 
the "Networking Release 1" from 1989. It didn't contain a full operating system,
but primarily the TCP/IP stack:  "With the increasing cost of the AT&T
source licenses, vendors [...] requested that Berkeley break out the
networking code and utilities and provide them under licensing terms
that did not require an AT&T source license." Among those "vendors" who
used the BSD TCP/IP stack in their own operating systems were IBM with
OS/2 and Microsoft with Windows.

>From this point on, the BSD developers of the Berkeley Computer
Systems Research Group - first of all, Keith Bostic - pushed hard to
develop a free release of BSD code which would be fully usable as a
Unix-compatible operating system. Quote McKusick:

  "Undeterred, Bostic pioneered the technique of doing a mass net-based
  development effort. He solicited folks to rewrite the Unix utilities
  from scratch based solely on their published descriptions. Their
  only compensation would be to have their name listed among the
  Berkeley contributors next to the name of the utility that they
  rewrote. The contributions started slowly and were mostly for the
  trivial utilities. But as the list of completed utilities grew and
  Bostic continued to hold forth for contributions at public events such
  as Usenix, the rate of contributions continued to grow. Soon the list
  crossed one hundred utilities and within 18 months nearly all the
  important utilities and libraries had been rewritten."

In 1991, the work was done except for six files that were still
needed from AT&T Unix:  "Proudly, Bostic marched into Mike Karels'
and my office, list in hand, wanting to know how we were doing on
the kernel. Resigned to our task, Karels, Bostic, and I spent the
next several months going over the entire distribution, file by file,
removing code that had originated in the 32/V release. When the dust
settled, we discovered that there were only six remaining kernel files
that were still contaminated [with AT&T-copyrighted code] and which
could not be trivially rewritten." Six months later, Bill Jolitz had
supplied the six missing files, compiled the resulting operating
system for PCs and released it as 386BSD (see the Wikipedia entry
<http://www.wikipedia.org/wiki/386BSD>).  The company BSDi was spun off
by 386BSD developers to sell a commercial, non-free version of BSD,
while community bugfixes of the operating system soon amounted to the
NetBSD project (from which Theo de Raadt later spun off OpenBSD).  

With BSDi marketing its product as "Unix" in 1992, the Unix division
of AT&T sued both the company and the University of California for
violation of its intellectual property. The lawsuit continued until
1994 and ended with a settlement soon after AT&T had sold off its Unix
division to Novell (which later sold it to SCO which, on the height of
dotcom hype, was bought by Linux distributor Caldera, until Caldera
changed its own name to SCO and now sues IBM for allegedly violating
SCO's "intellectual property" in Unix with its contributions to the
development of the Linux kernel).

The development of Linux began, as we know, independently from BSD
in 1991 because Linus Torvalds was frustrated with the limitations
of Minix, an educational Unix-like mini operating system by computer
science professor Andrew Tanenbaum. While the AT&T lawsuit questioned
the future of BSD, prevented its user adoption and stalled its
development, Linux kernel hacking took off in the same years. With the
GNU software of the Free Software Foundation, free development tools and
a Unix-ish userland had been already available. 

Both the free BSDs and general-purpose Linux distributions significantly
rely on the GNU software the Free Software foundation developed since
1983 as replacements of proprietary Unix components. GNU provides the C
compiler, debugger, manpage/troff formatter and some base commandline
programs (awk, bc, dc, [ef]grep, g[un]zip, zcat) of the free BSDs and
Linux distributions alike.  Beyond that, "Linux distributions" depend on
the entire GNU toolchain; their base Unixish commandline tools like cat,
ls, cp, mv, find, tar, man, sed... are GNU programs, and the software
they contain is compiled against the GNU C library.

> and the BSD family is quite similar to Linux in system 
> architecture and overall philosophy. The difference is negligible 
> enough that a user in one can become familiar with the other quickly -- 
> comparable, I suppose, between the difference between Windows 2000 and 
> Windows XP. 

Since Windows 2000 and Windows XP are two subsequent releases of the
same operating system (similar to FreeBSD 4.x and FreeBSD 5.x, or Debian
woody and Debian sarge), I would rather compare it to the different,
partly independently developed versions of the ancient DOS, i.e. MS-DOS
vs. IBM PC-DOS vs. DR-DOS vs. FreeDOS, or, to use a contemporary
example, the Java Virtual Machines/Runtime environments of Sun vs. those
of IBM and the Free Software project Kaffe. 

The differences may be too subtle to be noted by casual users - a
KDE desktop with OpenOffice is the same on GNU/Linux and the free
BSD, for example -, but can be significant for people doing low-level
administration and.

> (Mac OS X is built on a BSD core, though on some level it's 

Actually on a Mach microkernel running a BSD "personality".

> quite different because of all the stuff Apple added.)

Most prominently the proprietary graphical subsystem and user interface.

> BSD source code licensing is different from Linux licensing. Linux is 
> licensed under the GPL, which is viral in nature: Use any of the code 
> from Linux in a project of your own, and you're required to release 
> your project GPL. 

Sorry, this is notreally correct; the way you phrase it, your statement
very much resembles the Microsoft's FUD on "Linux" (while meaning GPLed
software in general).

First of all, it's GNU licensing rather than Linux licensing. Large
amounts of standard "Linux" (or rather: free Unix-compatible) software,
such as XFree86, Apache, Samba for example, are released under BSD-style
licenses. The GNU General Public License and its derivative, the GNU
Lesser General Public License applies to the Linux kernel, all GNU
software and such well-known programs as KDE and Gnome. By and large,
the GNU licenses are probably the most popular Free Software licenses,
because they prevent that free code gets incorporated into proprietary
code, possibly with proprietary, incompatible extensions (the way
the Kerberos protocol and the BSD TCP/IP stack got incorporated into
Windows). Interestingly, large companies - such as AOL/Netscape with
Mozilla, Sun with StarOffice/OpenOffice, Silicon Graphics with its XFS
file system and IBM with its JFS file system - prefer to release free
code under the GPL because it prevents that their competition takes
proprietary advantage of it.

Your assumption "Use any of the code from Linux in a project of your
own, and you're required to release your project GPL" is not correct, but
(sorry!) Microsoft FUD. You are only required to release your project
under the GPL if both of the following two conditions are met:

(a) you have _incorporated_ somebody else's GPLed code.  

    You can, of course, use Free Software to develop proprietary
    software. For example, you can code your program in GNU Emacs and
    compile it with the GNU C Compiler, and sell it as a proprietary
    product.

(b) if you publicly _release_ the program with the incorporated GPLed code 
    as a product.

    Non-public/in-house projects which incorporate GPLed code
    _don't_ need to be put under the GPL as long as they are not publicly
    distributed.

A prominent case of a GPL violation was NeXT, the company bought up by
Apple and whose NeXTstep operating system has been rebranded MacOS X.
NeXTstep is written in "Objective C", an object-oriented spin-off of C
programming language. Instead of building an Objective C compiler from
scratch, NeXT took the GNU C compiler, extended it for Objective C, but
released the result as its proprietary, closed-source project. Legal
action by the Free Software Foundation then forced NeXT to put their
modifications under the GPL.

Still, companies can sell GPLed software commercially, or they can
dual-license their code under the GPL and a non-free license (like, for
example, Trolltech's licensing of the Qt GUI library, Sun's licensing of
OpenOffice/StarOffice and AOL/Netscape's licensing of Mozilla/Netscape).

> BSD licensing allows you to incorporate BSD code into 
> your closed-source project if you want. 

- Yes, in which case the Free Software community wouldn't have got 
Objective C compiler extensions from NeXT.

> As regards quality: The quickest way to waste time is to ask a group of 
> sysadmins which is better, but the security guys I know mostly tell me 
> the BSDs are easier to lock down than Linux. 

Particularly OpenBSD, which provides the functionality out-of-the-box.

> One of the reasons they 
> cite is that although both projects are open-source, BSD is controlled 
> more tightly by a handful of fairly hard-to-please module owners. 

In the rarest cases, bugs in the kernel cause security flaws; the most
frequent and dangerous ones are those in network servers like Sendmail
because they can be remotely exploited (and which affect GNU/Linux and
*BSD alike). Comparing "BSD" and "Linux" in that respect is, IMHO,
like comparing apples and oranges. I assume SELinux or Trustix are, by
default, more secure than a default FreeBSD or NetBSD install while
OpenBSD is, by default, more secure than any mainstream GNU/Linux
distribution.

> BSD developer communities even have a slight reputation for being
> elitist, impatient, and generally dismissive of programmers who aren't
> complete geniuses.

The official tagline of the German newsgroup de.comp.os.unix.bsd is
"God's own operating system", and they are serious about that! - This
motto lately created an inspired theological debate in
de.comp.os.unix.discussion on whether God still has a root account for
this world. -
 
> possible. You're still racing to get your code good faster than the 
> next guy, 

Applies to proprietary software development much more than to Free
Software development. Watching a TV documentary about ex-Microsoft
employees taught me a lot in that respect. The company has excellent
programmers, including ones with decent architectural visions (after
all, Microsoft Windows 2000/XP is a microkernel-based OS with a modular,
component-based userland, an advanced journalling file system with
access control lists etc.etc.), but is dictated by its marketing and
sales people - on top: Steve Ballmer - forcing insane feature lists and
deadlines on the programmers who thus crank out code like mad. 

Compare that to the 4 1/2 years it took from Netscape's announcement to
free its browser in January 1988 to the release of Mozilla 1.0 in June
2002...

> and if you're making something mainstream like a web browser, 
> the vast majority of users won't care one bit whether or not your code 
> is GPLed or BSDed or owned by Bill Gates. They just want it to work.

Disagree. They also want to be safe that it doesn't contain spyware,
adware cookies or other malicious code.

> Some open source projects try to let the organization's priorities 
> bubble up from below, others rule from above. ( "Le code c'est moi" ?) 
> Open source doesn't automatically introduce you to this amazing world 
> of decentralized anarchy-in-action, though it does make it a little 
> easier to defect and start your own commune.

Indeed - unfortunately, Eric S. Raymond's writings ("The Cathedral
and the Bazaar") have falsely associated free/open source
software development with a certain development methodology or
politics. Prominent Free Software projects like GNU, XFree86 and the
BSDs are developed in cathedrals, and while this style seems to have
harmed the XFree86 project lately, it seems to work fine at least for
the BSDs. Then there also exists a popular confusion of Free Software
with open standards, but that's another story...

-F


-- 
http://userpage.fu-berlin.de/~cantsin/homepage/
http://www.complit.fu-berlin.de/institut/lehrpersonal/cramer.html
GnuPG/PGP public key ID 3200C7BA, finger cantsin@mail.zedat.fu-berlin.de

#  distributed via <nettime>: no commercial use without permission
#  <nettime> is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: majordomo@bbs.thing.net and "info nettime-l" in the msg body
#  archive: http://www.nettime.org contact: nettime@bbs.thing.net