Agree without reservation. and on your "Things that annoy me post" too.

PB

I disagree. The manufacturers spent thousands if not millions developing that code and there is no good reason to force them to give it away. There is nothing that can be learned by looking at the source code that cannot be determined by testing the operation of the devices.

Even if the manufacturers are in league with the police departments and have some code like this:

if ((BAC >= 0.06) && (BAC < 0.08))
{
BAC = 0.08;
}

The best way to determine whether they did this is not to look at the code, but to do an independent test on the device (what do you get out when the device is blown with a known 0.07 BAC). The lawyers are simply making demands that they know the manufacturers will balk at. Hopefully, the judge will see through this ploy.

Sissyphus-

I agree - testing is essential. But so is code review. How do you know without reviewing the code that there isn't some 'feature' like 'hold down the zero key for three seconds and the reported BAC on the next test will be > 0.08'.

For a feature like that to be useful, the officers would have to be in on the conspiracy.

It would be much simpler to just create an identical looking breathalyzer that you could dial to any BAC you wanted, and then after the reading is made, swap it with a code reviewed model.

Sisyphus,

You're far, far too optimistic on the quality of code that comes out of small manufacturers or of equipment with small runs. The code is rarely of very high quality unless the manufacturer takes extraordinary steps. Do you remember the recent case of release of criminals because of a bug in software code (http://www.wlns.com/Global/story.asp?S=4004197)? The stories of folks killed by medical software because of bugs? The catastrophic tales are legion, and that's for software that's been reviewed much better than this.

Me? If I'm accused by a machine I want to be sure that the machine has been reviewed for quality, which far too little software has been.

Oh, and it's C that's nothing more than a super macro assembler. C++ is more reasonable, although I still don't think that most programmers should be allowed anywhere near it. As Ted T'so commented on the linux kernel list, "...handing C++ to the average programmer seems roughly comparable to handing a loaded .45 to a chimpanzee."

"C++ - all the power of assembler, with all the ease-of-use of assember".

In embedded systems C++ would cause a lot of bloat, its a real memory hog. C is much more efficient, but it requires programmers who (a) know what they're doing [usually not a problem with C] and (b) have the time to code and test properly [this is the killer].

It all comes down to how much quality control management can support, and in a competitive environment history has shown that quality loses to "good enough but cheaper and sooner" when it comes to software development.

Nerdbert,

I agree that far too much bad code is written, but I don’t agree that code reviews by lawyers will significantly improve it. I believe that the FDA reviews the code for medical devices; maybe we should have something similar for law enforcement tools. That would be preferable to effectively making someone’s code public.

I also disagree with you on C++. I have worked with a lot of embedded code written in both C and C++. The C++ code is almost always better designed and easier to write, debug, and maintain. Of course, someone who doesn’t know what they are doing can make a big mess, but if a programmer is incapable of learning how to properly use C++, I wouldn’t trust them near my assembly or C code either.

Bill, I disagree with you on the C vs C++ bloat, not if you have a good compiler anyway.

Sisyphus-

>For a feature like that to be useful, the officers would have to be in on the conspiracy.

Want a related, conspiracy free scenario?

---------------------------------------------

The Chief of Police (CoP) goes to the big equipment trade show. He meets with a rep from the BAC tester company.

CoP: We need new BAC tester.

rep: Got just the thing right here.

CoP: Does this one make it obvious? The old one printed a slip that just said 'pass' or 'no pass' in tiny letters on a slip of paper, anlong with the BAC.

rep: Let me show you. Blow in here... Okay - here's your result. See how it says 'Tea Totaler' in big letters on top with your BAC underneath?

CoP: Great, but what if I'd had a six martini lunch?

rep: Glad you asked! Blow in here again while I hold down this key... Okay - now here's your result. See how it says '!!!Drunken Fool!!!' across the top with a BAC of 0.10 underneath? Thats what it would look like.

CoP: Great. Even a bleary-eyed rookie on his 13th consecutive hour on-shift wouldn't miss that!

---------------------------------------------

That demo code would never find it's way into a production machine, would it? No....

That feature would never be left in intentionally as a training tool, would it? No...

No one would ever set accidentally lean against the keyboard would they? No....

Keyboards never get a stuck key, do they? No...

No conspiracy needed. Just a poorly thought-out feature.

God help us all if there were an actual bug. Nerdbert is right - people have been killed by bugs in code that is stringently controlled and reviewed by hundreds of people.

Mr. Cranky,

Your example does not make a very good case for forcing manufacturers to provide their code for review. It would be far easier to find the problem you describe through testing (or just reading the manual) than reviewing code.

You admit that code reviewed by hundreds of people still has been found to have bugs, yet apparently super genius lawyers can find these problems. Companies should hire these magical lawyers to review code BEFORE the evil incompetent software engineers kill or falsely imprison their users.

I better stop now before I really piss everyone off by expressing my esteem for Microsoft’s software development process.

Sisyphus,

"I agree that far too much bad code is written, but I don’t agree that code reviews by lawyers will significantly improve it."

Who said lawyers would review it? That's why they hire expert witnesses at exorbitant rates. And it's good work when you can get it.

"I believe that the FDA reviews the code for medical devices; maybe we should have something similar for law enforcement tools."

If I may quote the FDA itself: "The FDA’s analysis of 3140 medical device recalls conducted between 1992 and 1998 reveals that 242 of them (7.7%) are attributable to software failures. Of those software related recalls, 192 (or 79%) were caused by software defects that were introduced when changes were made to the software after its initial production and distribution." (http://www.fda.gov/cdrh/comp/guidance/938.html) Using this as a guideline, I'd have to say that the folks in FL have a valid complaint if the code in the devices has been changed after the devices were certified.

It also points out a fundamental problem in software in general with verification: it's shortchanged. If after FDA rules have been followed and all the independent reviews done and there's still a failure rate this high, do you still maintain that independent inspection is not likely to reveal flaws in the system? Having dealt with FDA reviews and the immense time it takes to push electronic equipment through it and seeing the number of flaws that still come out, I'm far more sceptical that you wouldn't find something in there that couldn't be called a bug and a potentially serious one.

But I agree that C++ compilers have become almost as good at C compilers in the last few years and the memory bloat is now down to minimal levels for the more advanced ones: I really like Intel's, for example.

Nerdbert,

Okay, I shouldn’t have said “super-genius magical experts hired by lawyers” instead of “super-genius magical lawyers”.

Perfection cannot be reached; we can only approach it asymptotically. The more people who review code and test the product, the closer we get, the question is how close is close enough. We are in agreement that in health and law enforcement, the standard ought to be highest, and in all industries it should be higher than it is.

In the breathalyzer case, we want to know whether it is giving accurate readings. There may very well be a problem with the software, but it is possible to ascertain this by testing the breathalyzer as a black box.

"...the officers would have to be in on the conspiracy."Let me explain how this works from the point of view of someone who has experienced it.

The police pull you over (or come upon you after you called for help because you slipped off the icy road in February in Minnesota) and smell alcohol when they speak with you. They run you through the road-side stand on one foot and say the alphabet backwards while touching your nose in sub-freezing temperatures. Miraculously, you pass. But they "smelled alcohol". Now your rights are forfeit. They do not have to read you your rights because they "smelled alcohol". They can sieze your vehicle with no other explanation other than they "smelled alcohol". They tell you, according to the law, that you will either submit to a breathalyzer test or forfeit your driver's license for a minimum of one year because they "smell alcohol". You haven't even left the roadside yet.

When they get you, handcuffed, to the station you are put in a cell for some time while they go through your wallet and get some coffee. Then you are taken into another room with the big, desk sized, Intoxilyzer machine. They put forms in front of you and tell you to sign them or you will be presumed guilty and sent to the judge. You already know that unless this machine clears you, you will loose your license for a year. You're already guilty, the machine is there to give them the justification.

You might request to speak to an attorney before agreeing to the test. They have to let you do this, so they give you a phone book and tell you you get one call. You can use the phone over there in the corner of the holding cell (which is nearly full). Ever try to get a lawyer worth speaking to at 11:30 pm on a Friday night?

So you get some answering machine and the troopers are getting pushy. They know your body is processing whatever alcohol they thought they smelled when they first spoke with you. they need to get the machine to take a reading, and soon, or they've got extra paperwork and no revenue to show for it. Bad numbers are bad for their job, just as they are for yours.

So one of the guys is being nice, saying just sign the paperwork and then they'll get you processed and out of there. The other one is already yelling and threatening to lock you up for the next 3 days. They say you've already had your opportunity to call someone, tough shit if you didn't get through. They can't wait any longer.

So you think you're fine. No one got hurt. You had 3 drinks in just over 2 hours. It's been 2 hours since your last drink. You should be fine. You agree to take the test.

Damn, how did that come back 0.108 ?

"In the breathalyzer case, we want to know whether it is giving accurate readings. There may very well be a problem with the software, but it is possible to ascertain this by testing the breathalyzer as a black box."

Certainly. But the testing procedure for such tests is complicated and exceptionally long in the case of a complicated device such as this. And then you need to repeat it many times for each upgrade/patch (if you read the article, the folks doing the complaining were the ones tested by versions of the device that had *not* been certified -- they were later versions that had not undergone certification). Even in the simplest devices you wind up getting stateful sequential operations that must be performed just so to cause errors, and in the case of the justice system and the consequences of DUI these days I don't believe that failure is an option. A DUI can cut short many careers and socially these days it's worse than adultery.

The real problem is that code these days just doesn't go out with enough testing. Much of the culture of the software world is pretty slap-dash and patch oriented. Even in the medical devices field where it is someone's life we're talking about they can't even get it right. Given the culture and results from the software world and the consequences we're talking about here, I don't think it's unreasonable to demand more supervision.

Nerdbert,
We are agreed on the damage done by a false DUI. I would just throw out any readings not made on certified equipment, and once the software was changed, I would consider it out of certification. I am skeptical about breathalyzers in general -- I question whether there is an exact consistent relationship between alcohol in the breath and alcohol in the blood (although that is just a hunch, I've never investigated it).

We are also agreed in the need for better software testing and release procedures in general. I think the only place we differ is that I don't believe that forcing companies to turn over their code to defense lawyers (absence some evidence of fraud) will improve anything. It will make getting into the business less financially attractive to those who are good enough to do something else, decreasing rather than increasing quality.

If the software could be examined in such a way as to guarantee that proprietary information would not be released, I would not oppose it. But, I am skeptical.

"I think the only place we differ is that I don't believe that forcing companies to turn over their code to defense lawyers (absence some evidence of fraud) will improve anything."

You are code monkey at Bust'em, Inc. Your job is to write the code for your latest analyzer. As usual, the hardware wankers took too long to finish the design and you're under the gun to meet the deadline. Do you (a) bang out something that'll work to meet the schedule and fix the inevitable bugs in updates and patches or (b) push back hard on getting more time to get it done right? I can tell you which management will tell you to do...

Now the gov't demands that any code you write gets put out for review by guys who are actively trying to discredit your work. Suddenly (a) doesn't look as attractive to you or your company. If you don't write it well your name is Mudd in the industry, and the customers will have your head because they'll be losing their cases. Heck, even if you did a good job but didn't follow "best practices" your life will be miserable. Your competition has the same problem, so they'll have to wait, too.

Personally, I prefer to make guys who hold my freedom and future in their hands toe every line and take every precaution. And if code review by antagonistic outsiders is one club with which to beat them, I say whack away!

"If the software could be examined in such a way as to guarantee that proprietary information would not be released, I would not oppose it. But, I am skeptical."

Happens all the time. Look at the electronic CAD industry: tons of suits, tons of infringement claims, etc. Code gets distributed to independent experts (NOT the opposing company), reports are made to the courts, etc. Heck, even Brian Kernighan's done that gig in the SCO v. IBM case, which you may have heard of. There's a special dance you have to do when you do these things, but it's not rocket science and it's done constantly.

What's happening in this case and the Diebold cases are that companies are refusing audits at any cost -- they'd rather lose the business. That alone should tell you something about how they feel about independent experts examining their code.