The Wiki Man

We let programmers run our lives. So how’s their moral code?

When unethical behaviour is embedded in software, as it was at VW, bosses often don’t have a clue

10 October 2015

9:00 AM

10 October 2015

9:00 AM

A few years ago, in the week before Christmas when supermarket sales are at their highest, staff at one branch of a leading British chain regularly did the rounds of local competitors’ shops buying up their entire stock of Brussels sprouts.

It was, in its ethically dubious way, an interesting experiment. You might assume frustrated shoppers would merely buy all the other things on their list and then go somewhere else for their sprouts. They didn’t. As the perpetrators suspected, spending 30 minutes in a shop knowing that you’ll eventually have to make a separate trip to buy sprouts feels like wasted time — so people promptly left to find a shop where they could buy everything in one place. Their branch, with its sprout monopoly, enjoyed record-breaking sales that year.

But here’s the thing. They only did this once. Wiser heads prevailed. Perhaps the management were keen ethicists who realised that sprout-hoarding would violate the categorical imperative or (more likely, I think) they were afraid they’d get rumbled. Either way, they instinctively felt the activity was wrong.

Some people will blur ethical lines occasionally; what’s rarer is when bad behaviour becomes widespread. This is what is so bizarre about the VW emissions affair. For a car company to tweak cars before measurement (‘teaching to the test’ as it’s known) might be expected; when officially testing a car’s mpg, you give it your best shot by turning off the air-conditioning and pre-charging the battery. I understand this. But to extend the ruse to a point where millions of cars carry ‘defeat devices’ seems astonishing.

Why did no one in the car industry cry foul? Contrary to what journalists think, people in large businesses usually span a wide spectrum of political views and opinions; Germans are high-minded about environmental issues to the point of sanctimony (remember the Brent Spar affair). This is a company so Teutonically perfectionist that for many years it lost countless sales in the US by refusing to add cup-holders.

Perhaps everyone believed, rightly or wrongly, they had the tacit approval of regulators for what they were doing? Or was it a case of pathological altruism, where engineers felt they were doing God’s work by reducing CO2 emissions with smaller diesels, and believed any price in other emissions worth paying? Or perhaps very few people knew what was going on?

This last possibility seems implausible — until you remember one thing. The ‘defeat device’ is not a device at all: it is some lines of additional software in the engine control unit. The addition of a physical device to cheat the test would have required approval from many different divisions of the company, some of whom would have spotted the ethical and reputational risks. But if you want to rig up some dodgy ECU software, all that’s theoretically needed is one or two unscrupulous people and a keyboard. The software they produce is largely incomprehensible to anyone else.

We are unwittingly delegating a huge amount of unscrutinised power to programmers. Once, computers largely performed dull repetitive tasks with great speed and precision; today we are delegating complex human judgments to software and to the small clique of atypical people who write code. Most do a fine job, but among them must be a few rogue actors who wouldn’t see anything wrong with buying up everyone else’s sprouts.

The truth is that most companies have little clue what is inside the software they produce. Do any of the editors of the newspapers reporting this affair have the slightest idea about the inner workings of their publications’ mobile applications, or how they handle readers’ data? I suspect not.

Got something to add? Join the discussion and comment below.

Rory Sutherland is vice-chairman of Ogilvy Group UK.

You might disagree with half of it, but you’ll enjoy reading all of it. Try your first 10 weeks for just $10

Show comments
  • Leroy_Jenkins_01

    Programmers tend to be very literal, you have to be when speaking to a computer* so I’d imagine what happened here, if you are correct about the “defeat device” being solely down to the coders I’d guess they read “We need to make the software tweak then engine so these cars will pass the emissions test” and took the last part of that to literally apply to only when the test was running.

    * Bonus Joke: A programmers wife tells him to go to the shop and pick up a loaf of bread, and if they have eggs get a dozen. The programmer returns home with 13 loaves of bread.

    • rorysutherland

      Interestingly, I initially thought it was odd – and inherently dodgy – to program the device so that it would spot when it was on a test-rig.

      However, having investigated this a bit more, it seems that the on-board computer of a modern car has to know when it is on a test rig: otherwise, when the Dynamic Stability Control system notices that the drive wheels are moving and the other (rear) wheels are not moving at all, it will go into complete spasm, assuming the car has completely lost traction. Hence it needs to enter a separate mode when on a rig in order to behave normally.

      • Conrad Brown

        I put it to you that upwards of 50 people will have been aware, including management, architects, coders, testers, QA. Some of them will not have fully appreciated the implications, or thought they were building to the design, some will have not cared very much one way or the other and they would let senior management take responsibility. It will have also been a gradual process, with each subsequent release gaming the system more.

    • Stan Gad

      should be 12 loaves of bread
      that’s more logical

      • Leroy_Jenkins_01

        Nope, the first statement “pick up a loaf of bread” stands alone, the loop initiated after that using a previously defined variable 😛

        • Stan Gad

          not if he ate one on the way home

          • Leroy_Jenkins_01

            Ha ha ha, OK I’ll give you that one.

        • Paul B

          I disagree. The “and” obviously ought to be read as “else” not “additionally”.

  • caycepollard

    I don’t support the lone programmer theory.

    • rorysutherland

      Nor me. I merely think it is one of a few plausible possibilities – one of them being downright corporate greed. But it is much more easy for this thing to happen when it is empowered by software than hardware. All kinds of reasons – invisibility, overoptimisation, the need for fewer people to get involved – and the fact that once written software is forgotten.

      • Snibbo

        Programmer here. Yes, whoever is responsible, the main point is that it could not have happened with hardware. Software, unless it is integrity checked, is completely malleable and can be altered without raising any alarms. Automated digital integrity checks are usually applied only to safety critical software, such as flight control systems; in this case the “check” was probably made by the programmers/engineers/managers themselves (in Germany engineering managers usually are engineers, so they would have no excuse for not knowing).

        • uberwest

          I don’t see how software can control CO2 levels or whatever in vehicle emissions without hardware to talk to. This implies that the hardware is capable of doing what the software tells it to, it also implies that the hardware engineers were in on it.

    • Caractacus

      Not in this case – the simple fact that other car manufacturers are caught up in this with similar cheats, disproves it.

      However, Rory Sutherland has nonetheless put his finger on a very important point that should not be ignored simply because it doesn’t apply here.

  • Huperniketes

    This breach of public trust is similar to that of Bernie Madoff’s programmers, whose creation of software to mislead auditors and investigators permitted their firm to defraud investors of US$17 billion.

    It is time programmers should be required to meet professional workmanship and ethical standards as are required of other engineering disciplines, lawyers, medical practitioners and even hairdressers.

    • uberwest

      They would only have done as instructed.

    • MikePage

      What does the law say? That it is illegal to code? No – that it is illegal to offer sub-standard goods for sale. Bosch gave clear account of their actions – what they had done and that it was not for market. VW are the marketing entity and are responsible, full stop.

      • Huperniketes

        Workmanship and ethical standards for the software profession are the codification of what is legal, and providing goods which are sub-standard would result in penalties for the engineers found responsible.

        For that reason, I assert that such obligations on the profession would seriously hamper teams or companies from claiming that code permitting emissions exceeding the government’s standard was intentionally crafted by a rogue programmer for “whatever reason”.

        • MikePage

          Technology is a team effort. I design meters for electricians although I am not a practicing, licensed electrician and depend on “Marketing” to provide me a suitable specification and not over-sell the product once it’s ready.

          This is quite difficult in practice because of sales culture (think “Glengarry Glen Ross”) and the tendency for the less technical to gravitate towards marketing.

          Sorry if I have misread your reply: I’m not quite connecting your first and second paragraphs. It is quite an old thread …

  • uberwest

    I can’t see the software developers taking it on themselves to do that. They would need approval from higher up. After all, it’s not their responsibility to ensure that the cars meet the company’s pollution targets. I think the company have found scapegoats to cover the arses of the big nobs.

    • Karol Stasiak

      I can’t see the software developers taking it on themselves to do that, either, but for a totally different reason: they’re too lazy to bother unless being told to. Most likely, the idea came from some medium-level manager.

      • uberwest

        Spoken like a true manager. Developers are ‘lazy’ if they don’t come up with ways of fraudulently enhancing the reputation of their company’s product.

        • Karol Stasiak

          “Should I bother writing this piece of code that will take my time, may cause legal trouble if discovered, and no one is telling me to write? Nah, why bother, I’ll browse Reddit for two hours more instead.”

          • serguei_p

            If only. Then there would no bugs created by developers who decided to refactor code unnecessary…

            Also in this case I think a middle manager was involved.

    • MikePage

      They only think they have, but their slur against technical staff has so many holes, it’s insupportable.

  • Fasdunkle

    “the small clique of atypical people who write code”

    Ooh, that makes me sound exciting and edgy

  • Clive

    It is certainly not ‘coders’ who did it – not in the modern world of computing. There would have to be a design input.

    A phenomenon which no-one ever seemed to get to the bottom was the design of derivatives that caused the financial crash of 2008.

    As I understand it – and my knowledge is very limited – Mortgage Backed Securities (MBS) were deliberately designed and bundled so that high value ‘safe’ mortgages were together with risky mortgages. The credit rating agencies gave them top ratings anyway so that they counted in the banks’ capital.

    That looks a lot like gaming the system – which is what this is all about.

    It is the moral hazard which Daniel Hannan has spoken about of replacing each individual’s ethics with imposed regulation. The regulation replaces the person’s ethics so that the person games the regulatory monitoring system instead of using their own ethical judgment.

    • MikePage

      A team has to pull together, and regulation facilitates that. Who writes and imposes the regulation, though? They bear the far greater responsibility.

  • Fasdunkle

    Programmers are a small link in the chain for systems used in engine management – they follow a detailed specification and the code is subject to code review and extensive QA.

    • Paul B

      The code in an EMU is not written by common and garden programmers but more likely by mechanical engineers who program. I think it quite possible no one actually reviewed the code. But what I can’t accept is that non-programmers would not spot that the engines have better emissions when tested. Or that engineers with any interest in physics or chemistry would not question the result.

      No, the problem is that one is allowed to cheat at the tests in Europe, there is widespread cheating, everyone knows mirrors are removed, gaps sealed, alternator disconnected, tyres pumped. It’s legal in Europe, there are no penalties.

      In the USA they don’t cheat because of the penalties. But no one told the engineers in Germany who shipped the software to the USA. Perhaps obliviously, as I suspect.

      • Fasdunkle

        It will be written by embedded systems programmers. Many of them are from electrical engineering backgrounds

      • rorysutherland

        This is a very good explanation.

      • MikePage

        Speaking generally, EMU code is subject to higher standards both in terms of source code quality – since MISRA is a normal requirement – and performance, since there are safety critical aspects.

        It is unfeasible the Germans didn’t know about relevant US standards, which are famously demanding and expensive. And it may be some of the European test practices are considered “cheating” but if the test procedure allows it (and everyone does it), how can it be? It is the test itself which is at fault.

        Yes it’s true “people” would note obvious differences in performance when testing vs real world. How that is explained internally is where the deception has occurred, before being released into the wild.

    • Caractacus

      Yes, in this case this scenario is simply not feasible. That doesn’t mean it is never feasible. Say I, completely code ignorant, employ someone to write me a website. How do I know that the coder is legit and hasn’t embedded a malicious code on the site?

      • Fasdunkle

        you don’t know. However you can get the code independently reviewed or use a tool such as Fortify

        • Caractacus

          Useful info about Fortify. Thanks.

      • MikePage

        Cui bono?

    • serguei_p

      The only people who can review the code are other programmers who work on this code.
      Unless you are a member of a team that works with the code every day, you would have a lot of trouble understanding it even if you are a good programmer yourself.

      • Fasdunkle

        That’s not how code reviews work

        • serguei_p

          And how do they work? A manager with an MBA looks into C code and make clever pronouncements 🙂 ?

          If you have never seen a code of a particular system before, even if you are a programmer, it will take you quite some time to get up to speed, before you understand what is going on.

  • leed25d

    This is one of the reasons why open source is so important.

  • Paul B

    As I see it there is no one overwhelmingly plausible explanation. Yet. But I have a better one that the lone programmer theory.

    Sure, perhaps one lone programmer could have inserted the code “if (hooked up to test device) then (lower emissions)” but VW is a firm of engineers. Some group of engineers at VW must have responsibility for engine performance and another or the same group of engineers for emissions compliance. They need to talk amongst themselves. How is it, they must have asked each other, that this small diesel is so much better than that small diesel? I think several engineers and their management must have known about the programming trick.

    In Europe all kinds of tricks are employed to pass the tests. Wing mirrors are removed, gaps between body panels are taped, alternators are removed, tyres are pumped to bursting point. None of this is secret, it’s very well known and all manufacturers do it. There are no legal penalties in Europe for passing the tests this way.

    There are severe penalties in the USA for acting this way. That’s what no one told the engineering team and their bosses in Germany.

    • MikePage

      What do you think of the Bosch memo?

      • Paul B

        I have no inside track on any of this. But there is a lot of nonsense being written and spoken about this, patently. The lone programmer theory always seemed a nonsense to me, and the Bosch memo does seem to demonstrate that management knew.

  • Edward Newgate

    You really don’t seem to understand how embedded software of that scale is developed. There is a very specific process that includes many reviews of the actual source code, it’s conformance with MIRSA C standards, as well as QA and security reviews. The fact that they shipped something like that, is a strong indication that the decision came from higher in their corporate “chain of command”.

    Really weak article.

    • serguei_p

      Do you really think anybody who never programmed in C would understand what any of MIRSA C rules mean?

  • carl jacobs

    The German media is reporting that a subcontractor (Bosch) delivered the defeat device to VW in 2007 along with a disclaimer that it was only to be used “for testing.” So VW must have contracted for it. VW asked a subcontractor to develop a piece of SW that specifically detects a specific test profile in order to change the behavior of the engine when that test profile is detected. I’m sitting here wondering what tests would require such a device.

    If I was a cynic, I would think that Bosch knew exactly what VW was doing, and included the warning to protect itself.

    • Yes, I would find it incredible and difficult to believe that Bosch would knowingly be complicit in development of such software, at least without heavily covering it’s arse and getting supreme sign off.

      • carl jacobs

        We’ll see how good their lawyers are.

        I can understand the position in which Bosch could have been placed. A major customer wants X. “Give us X or we take our business elsewhere.” VW even provides assurance that this capability is only “for testing.” It’s not really the business of Bosch to regulate VWs ethical behavior – assuming Bosch itself isn’t thereby exposed to legal liability.

        The easy solution would have been “Sure, we’ll do this, but we want this capability to be public knowledge. Its existence needs to be self-reported to regulatory agencies.” If VW was only using it “for testing” then this theoretically shouldn’t have been a problem. Except it would have created all sorts of difficult questions about proving it wasn’t in production links, and what tests required the capability in the first place. The thought of public exposure would have caused crucifixes and garlic to be displayed on doors at VW.

        So everyone kept silent, and the lawyers scribbled furiously.

      • rorysutherland

        Just to clarify, all cars need the ECU (Engine Control Unit) software to detect when the vehicle is on a test-rig, otherwise when one set of wheels (the drive-wheels) are moving and the other wheels are static, the car’s dynamic stability control system will go haywire, assuming that the drive wheels have completely lost traction. So spotting test-mode is not as abnormal as it may seem.

        Without wishing to seem a corporate shill, I am always a little suspicious when journalists quote figures such as “over 10x accepted limits” without quoting actual figures. Typical NOx emissions from a modern, very green diesel in normal road conditions might be 0.28g/km, say. If a VW in urban driving registers 0.8g/km that sounds terrible when expressed as a multiplier. But I wonder what the NOx emissions are from, say, a 1974 Routemaster bus? I’d like a sense of wider comparison before expressing a sense of wounded outrage. After all, someone in a new VW diesel is likely a good deal cleaner than someone in a 2005 Merc – but it is politically impossible to penalise the drivers of older cars.

    • WTF

      As always, follow the money. Bosch supplies the same parts to many manufacturers and wouldn’t jeopardize its sales by treating anyone differently so I’m certain it would cover its ass with a disclaimer and I don’t blame them.

  • I take your wider point, there does need to be more interest higher up about what’s going on under the hood (literally in VW’s case)… but CEOs are not programmers, so they rely on testers, security experts, auditors and quality assurance processes. Where were they at VW? It cannot feasibly be claimed that all of those areas were vacant during the development and shipping process of this software, otherwise we have a bigger safety issue on our hands than at first thought. So it can only reasonably be presumed that management not only knew about it, but probably commissioned it too.

    • WTF

      That’s not how it works at product review meetings and this is the usual discussion.

      Engineer: If we are to meet pollution levels we have to reduce economy and performance.

      Marketing: What if we program the vehicle to sense a rolling test bed for emission control and adjust the engine management to pass it, can you do that ?

      Engineer: Yep, that shouldn’t be a problem as the driving pattern is completely different.

      Marketing: OK, program it to pass the rolling test bed emissions but switch to performance mode on the road.

      Engineer: OK but isn’t that breaking the rules.

      Marketing: Let me worry about issues like that, your only concern is to get it to pass the rolling test bed tests. Oh, and by the way, this is sensitive internal information to us (nudge, nudge, wink, wink, say no more)

      • MikePage

        That’s how it starts, with someone trying to be clever. News today about how many execs knew, and the CEO jumped pretty sharpish, didn’t he?

      • You know too much.

  • I’m sympathetic to the car-makers. We in the no-idea-how-it’s-made world expect miracles, and we expect them to be economical. We expect too much for the money we pay. And Leftists, in their quest for the Golden Tomorrow that no one has even seen, expect more than too much. I blame the Leftists.

    • WTF

      Why are you sympathetic with the car makers ?

      If they all said these emissions were impossible to meet without sacrificing performance and economy AND the buying public were informed this due to bodies like the EU, it wouldn’t happen.

      • Ever heard of the idea of ‘diminishing returns’?

  • TheCarlos

    You are absolutely correct that bosses don’t have a clue. Such is the extent of the cluelessness that the vice-chairman of a marketing firm could actually believe that a fraud of this nature could happen without direct managerial instruction at the highest level. To *this* engineer, the attempt to frame the situation as a “rogue” incident from a “rogue” engineer beggars belief. But then again, I *do* have a clue. Not because I belong to the atypical clique of people who write code, but because I belong to a *discipline*. This is not a case of programmers going rogue, this is a case of programers who should have refused to go along when their *bosses* went rogue.

    • MikePage

      Exactly. Cui bono? (Follow the money.)

  • WTF

    Lets correct a few mistakes here.

    Top management does not usually have a clue on programming any device or car they sell.

    Top management in marketing most certainly does understand trades offs that are presented to them affecting emissions vs performance and economy in the case of cars.

    Having worked in engineering where we had to meed EMI emissions (Radio Interference) laid down by the EU a ‘lowly’ engineer knows full well has to meet those standards. There is ALWAYS a problem (usually cost) in meeting the regulations as it usually means extra hardware, shape & size or some other factor that marketing don’t want to hear about. The engineer at product review meetings will raise these issues and I’m 99% certain that in this case VW marketing picked up on the aspects here and suggested this fiddle with a nod and a wink. Its a brave engineer who will go against a top exec.

    Its just like banking chiefs and staff in the local branches, most ‘low’ level staff are not crooks but just want to do a good job plus there isn’t any monetary incentive like stock options or bonus’s that affect their work ethics.

    • MikePage

      Agree – and it’s not illegal to code, but to sell sub-standard goods. All the engineers need to be is honest about their work.

  • WTF

    The question that hasn’t been asked by this article is motive –

    Whats in it for these ‘rogue’ programmers ?

    Were they given mega bonus’s, loads of stock options, paid for vacations, free cars, It certainly doesn’t seem like it.

    Now ask the same question of VW execs ?

    Remember all the banking scandals of the past decade, you can see where real motivation exists for crooked behavior and its not the counter staff !

  • JSC

    I’ve been a programmer for 20 years, let me tell you how this kind of thing occurs
    01. Boss asks programmer for a fuel emissions system.
    02. Programmer produces the system.
    03. Boss doesn’t like the performance of the fuel emission system. Boss demands [insert completely unrealistic goal] of the fuel emission system.
    04. Programmer tells boss their demands are unrealistic.
    05. Boss tells them to implement them anyway.
    06. Programmers improve system significantly.
    07. Boss still doesn’t like performance of fuel emission system. Boss still demands [insert completely unrealistic goal] of fuel emission system.
    08. Programmer tells boss their demands are unrealistic and require the suspension of the laws of physics.
    09. Boss tells them to implement them anyway.
    10. Programmer fudges system to appear that bosses demands are met.
    11. Boss is satisfied with system.
    11. No acknowledgment is given to the programmer that they have apparently defied the laws of physics.

    Funny, but 100% true.

    • sidor

      Science is a miracle. Science sold for money is witchcraft. The very act of selling miracles is a sin. This is like selling love for money. No ethics is applicable.

    • You’ve got a bug at line 11 😉

      Actually I would insert a new point after 11. Marketing comes along having been absent all this time and says the software needs to feature dancing kittens too.

  • MikePage

    I thought the lone programmer slur had been busted ages ago?

    It’s easy to regard with suspicion what you don’t understand, but it’s also kinda cheap and lazy, sneering even. And so Sutherland completely fails to make his point.

    • uberwest

      It’s pretty rich for a journalist to question the moral code of anyone, to be honest.

    • rorysutherland

      I wish I had had a few hundred more words to make my point better. I am not casting aspersions against coders at all. (I am always much more inclined to believe the cock-up view of history than the conspiracy view). My point was simply that invisible things which replicate easily are significantly different to mechanical devices, which cost money to reproduce and which are visible.

  • Ade

    Witchcraft! Burn the programmer! And anyone else, doing things we don’t understand…

  • Betty Hernandez

    Here is something worth attention , an opportunity for work for those who want to use their free time to make money using their computers… I have been doing this since last two years and I am making 40 to 70 dollars per hour … In the last week I have made 12,245 for almost 18 hours sitting ….

    ?There are no special skills required just basic typing and an internet connection ….

    ?There are no time constraints … You may do this any time when you are free ….

    ?Here is what I’ve been doing….

    < ->>w­w­w­.­c­h­a­n­g­e­y­o­u­r­d­e­s­t­i­n­y­f­o­r­b­e­t­t­e­r­.­b­l­o­g­s­p­o­t­.­c­o­m >


  • sidor

    Rory, would you ask about moral standards of a surgeon saving your life? You depend on him, he doesn’t depend on your opinion. You are not in a position to ask questions: you can only thank. That man is God’s grace for you.

    The same is your situation with respect to programmers and other people who produce technology providing your existence. You need them, they don’t need you. Be humble and grateful.

  • David Gillies

    The idea that a couple of rogue hackers deep in the bowels of VW’s software division took it on themselves to write a test defeat mode into the engine management software code without sign-off from management is so stupid as to beggar belief. This isn’t some Easter egg in a video game. I don’t know what the Ogilvy Group makes but I bloody well hope it’s not software.

    • Ogilvy Group is a marketing company and, like most marketing companies, they are pretty good at fudging the realities of the products they push on behalf of their clients. It wouldn’t surprise me if the author of this piece was trying to appear sympathetic to poor VW’s ‘plight’ in an attempt to win them as a client in the near future.

    • serguei_p

      They were not “rogue hackers”, they were simply people who’s loyalty was a bit misplaced and also influenced by an interesting technical task (figuring out how to tell if the car is undergoing the test).
      A product manager probably knew at some point that there was a trick in software, but probably did not fully understand what it did (this is how IT works – you have to be the programmer who changes the code level to know what REALLY is there, managers’ job is different and code control is not what they do).

      • David Gillies

        Yeah…no. That’s not how IT works, at least in a large organisation like VW. You don’t just commit stuff to the mainline source code repository and hope that no-one will notice. Before anything even advances to unit test it will have undergone code review. You potentially need to be able to justify every line of code, every choice of data structure, algorithm or even variable name. There will be regression tests to make sure it doesn’t break anything, all signed off by people who are at least as knowledgeable as the primary coders. Then it has to go to acceptance testing and QA. Unless the test-defeat mode was part of the specified behaviour of the EMS it would have failed at this point.

        Industrial-grade software development is a bit different from the idea of the lone nerd typing away in his basement. It’s as tightly managed as any large-scale engineering process. I actually do this sort of stuff for a living, so I’m not just hand-waving here.

        • serguei_p

          And who would do the code review? They are the guys from the same team – because nobody else would know what the code does.
          It is not like nobody notices. It is more like developers at some point in the past understood that “special functionality for test” was what the Product Manager asked for, Product Manager probably did not pay much attention to the details (or preferred not to). As years pass after that the developers changed, management changed and everybody assumed that what the code did was what is supposed to do.

          • rorysutherland

            I think this is a very good explanation indeed.

        • serguei_p

          David, the people who notice are fellow developers. If other developers assume that the change is required then there would be no red flag raised.

          Management does not read the code and unable to do code reviews, so a lot depends on how “the nerds” understand the requirements.

          I can’t comment on the particular process in VW (I never worked for them) but I have seen enough IT from inside :).
          I suspect VW coding process is more on a bureaucratic side meaning that the coders themselves might not even know the answer to “why” (especially if they were somewhere in India), so a lot depended on whoever was the technical leader writing detail technical spec (effectively a “nerd” with a higher status who knows how to talk to people).

          • David Gillies

            I’m sorry, that’s just rubbish. You ‘suspect’ VW’s process is bureaucratic. Of course it’s bureaucratic. Intensely so. Deliberately so. It always is in large industrial environments, especially when creating mission-critical, real-time embedded systems. Peer developers will not be privy to code outside their areas of responsibility. The development life cycle will proceed according to rigid design criteria leaving very little scope for deviation. Code will need to satisfy stringent best practises (do you know what MISRA C is?). Great gobs and scads of documentation will be generated at every stage of the process (this is the bane of the systems engineer’s life, but it is a requirement.) We’re talking about an Engine Management System, for Pete’s sake. Every byte of code in it will have been analysed, scrutinised and documented. That’s just the way things are.

            To suggest that any of this is not the case is to suggest that VW’s software engineering practises were sui generis.

          • serguei_p

            MISRA C is something only programmers will understand. Managers can demand it, but unless a programmer is involved they would not know if it is adhered to or not.

  • informatimago

    About the ethics, I feel that there’s actually more ethics in the programmers than in corporations in general. The only thing that holds the corporations back, is the fear of lawsuits and the corresponding economical hit they might take.

    But in any case, you how can you trust software and firmware? The programmer is only one amongst the earliest steps in chain of creation of the software or firmware. While in small shops, it’s often the programmer who will produce the binaries for distribution, in bigger shops and notably when the software is enclosed in firmware, the original programmer is several steps removed from the actual production of the firmware or distribution media (eg. signing and uploading on AppStore), and at each of those steps, the software could be tampered with, if it wasn’t defective at the origin. This is not to exonerate programmers (eg. in the case of the Toyota firmware, it was the programmers who produced the awful and fatal firmware), but to mention that a lot of parties, both inside and outside the corporation selling the software can introduce bugs or malware.

    Therefore you need 4 things:

    – you need to trust the hardware itself. If you’ve not built your microchips yourself from sand, can you trust Intel? Can you trust a Chinese chip? Can you trust a Russian chip? A couple of years ago, there was some scare from a paper describing some Chinese spying chip (it might have been a fake, but it’s perfectly possible).

    – you need to trust the OS (BIOS, system, tools), and the compiler. (cf. Ken Thomson’s “Trusting Trust” and “Reflections on Trusting Trust”, and Schneier’s “Countering ‘Trusting Trust'”). The point here, is that you have to have the sources to all those software components, and you have to compile them yourself, using a trusted compiler (and how do you get a trusted compiler? It’s not entirely trivial, even if Scheneier gives a solution).

    – now you need all the sources of the software and firmware of the concerned devices, so that you may audit them (otherwise, you might be using for decades software with obvious bugs, backdoors or other malware, cf. recent examples as the bash shell bug, the openssl bugs, the toyota firmware problems, and indeed, the VW hack, etc),

    – and you need to compile and integrate/install it intot he device yourself, using the trusted OS and compiler obtained at step 2.

    Now, of course, it’s not the final user who can do all that. But as the countries have administrations to test and check the properties of drugs (eg. the FDA in the USA), and allow the drugs to be on the market only after strict checks and validations, countries could forbid the sale of any hardware containing software or firmware, and of any software, that:

    1- is not provided in source form with all the required instructions and dependencies to compile and install it in the devices they want to sell, and

    2- has not passed a full, precise and exhaustive “code review” mandated by a FSA (Federal Software Administration), and received thus an authorisation of sale, along with a FSA distribution of that software.

    Now, for the state it would be ok to use that FSA distribution, but how a final customer (persons or corporations), could trust the FSA of their country. After all, their NSA could have introduced new backdoors, or the FSA could be corrupted by lobbies too! Therefore the alerted user or the concious corporation will want to have the sources and compile and install them themselves too, after an audit (which would be lighter, including eg. only a comparison of those sources with the one from the original company).

    In conclusion, what you want is a law that forbid the sale of non-free(dom) software or firmware (because, if anybody introduced a backdoor in the sources, you want to be able to remove it for your own usage, or that of your friends or customers).

    Forbid the sale of any non-free(dom) software or firmware!

  • serguei_p

    What is true is that the businesses are very much dependant on what programmer produces without really understanding how it works.
    In my career I came across companies that would ask you to develop software calculating price with only an Excel spreadsheet as a “documentation”. The spreadsheet would then be lost and the only spec on where the values come from are in the code.
    What is even more warring is that these days the programmer who actually does the coding sits somewhere in Bangalore.

    Generally the businesses are lucky that the programmers today (at least in the Western countries) are mostly quite idealistic and most of them tend to care about what they do.

  • iconoclast73

    I’ve written software for devices set to be sold in the thousands or greater. It has occurred to me, (but I never gave into this temptation) that I could easily put in a back door that could give me control over thousands of devices. Perhaps there should be a law that programmers could be held accountable if they leave in secret functionings in the code.