Re: <nettime> VW

[dan@geer.org]

http://new.spectator.co.uk/2015/10/we-let-programmers-run-our-lives-so-hows-their-moral-code/

   We let programmers run our lives. So how's their moral code?

   When unethical behaviour is embedded in software, as it was at
   VW, bosses often don't have a clue

   Rory Sutherland
   10 October 2015

   A few years ago, in the week before Christmas when supermarket
   sales are at their highest, staff at one branch of a leading
   British chain regularly did the rounds of local competitors'
   shops buying up their entire stock of Brussels sprouts. It
   was, in its ethically dubious way, an interesting experiment.
   You might assume frustrated shoppers would merely buy all
   the other things on their list and then go somewhere else
   for their sprouts. They didn't. As the perpetrators suspected,
   spending 30 minutes in a shop knowing that you'll eventually
   have to make a separate trip to buy sprouts feels like wasted
   time -- so people promptly left to find a shop where they
   could buy everything in one place. Their branch, with its
   sprout monopoly, enjoyed record-breaking sales that year.

   But here's the thing. They only did this once. Wiser heads
   prevailed. Perhaps the management were keen ethicists who
   realised that sprout-hoarding would violate the categorical
   imperative or (more likely, I think) they were afraid they'd
   get rumbled. Either way, they instinctively felt the activity
   was wrong.

   Some people will blur ethical lines occasionally; what's
   rarer is when bad behaviour becomes widespread. This is what
   is so bizarre about the VW emissions affair. For a car company
   to tweak cars before measurement (`teaching to the test' as
   it's known) might be expected; when officially testing a
   car's mpg, you give it your best shot by turning off the
   air-conditioning and pre-charging the battery. I understand
   this. But to extend the ruse to a point where millions of
   cars carry `defeat devices' seems astonishing. Why did no
   one in the car industry cry foul? Contrary to what journalists
   think, people in large businesses usually span a wide spectrum
   of political views and opinions; Germans are high-minded
   about environmental issues to the point of sanctimony (remember
   the Brent Spar affair). This is a company so Teutonically
   perfectionist that for many years it lost countless sales
   in the US by refusing to add cup-holders.

   Perhaps everyone believed, rightly or wrongly, they had the
   tacit approval of regulators for what they were doing? Or
   was it a case of pathological altruism, where engineers felt
   they were doing God's work by reducing CO2 emissions with
   smaller diesels, and believed any price in other emissions
   worth paying? Or perhaps very few people knew what was going
   on?

   This last possibility seems implausible -- until you remember
   one thing. The `defeat device' is not a device at all: it
   is some lines of additional software in the engine control
   unit. The addition of a physical device to cheat the test
   would have required approval from many different divisions
   of the company, some of whom would have spotted the ethical
   and reputational risks. But if you want to rig up some dodgy
   ECU software, all that's theoretically needed is one or two
   unscrupulous people and a keyboard. The software they produce
   is largely incomprehensible to anyone else.

   We are unwittingly delegating a huge amount of unscrutinised
   power to programmers. Once, computers largely performed dull
   repetitive tasks with great speed and precision; today we
   are delegating complex human judgments to software and to
   the small clique of atypical people who write code. Most do
   a fine job, but among them must be a few rogue actors who
   wouldn't see anything wrong with buying up everyone else's
   sprouts.

   The truth is that most companies have little clue what is
   inside the software they produce. Do any of the editors of
   the newspapers reporting this affair have the slightest idea
   about the inner workings of their publications' mobile
   applications, or how they handle readers' data? I suspect
   not.

   Rory Sutherland is vice-chairman of Ogilvy Group UK.


#  distributed via <nettime>: no commercial use without permission
#  <nettime>  is a moderated mailing list for net criticism,
#  collaborative text filtering and cultural politics of the nets
#  more info: http://mx.kein.org/mailman/listinfo/nettime-l
#  archive: http://www.nettime.org contact: nettime@kein.org