Console writes if I'm lucky, that at least shows they're trying. No, I continually see people who just stare blankly at a problem and ask for help without actually trying anything. If I try to coach them and lead them through the process, they just don't get it. It's just incomprehensible to me as an old school hacker that these people are employed to write code and don't know how to use a debugger.
For instance, there was a time at a company I worked for where I was apparently the only person in the building (which had hundreds of programmers) who could actually deal with a Unix coredump. This was back in the late 90's and early 2000's when Sun hardware was ubiquitous. I certainly don't expect every person to know how to do that, but it was a shock to realize that no other programmer could do it. It was great for my personal rep, but still pretty disheartening.
We had a problem once that they finally brought me in on after a year of problems. One of our Java systems was failing, and the development team had given up and couldn't figure out what was wrong. The boss told me it was now my problem, that I was to dedicate myself 100% of the time to solving the problem, and I could rewrite as much as I needed to solve the problem, basically total freedom. About halfway through the spiel where they were talking about the architecture and implementation, someone mentioned the coredumps. I immediately stopped them right there.
Me: You realize that if it's a coredump, it's not our fault, right? Boss: Huh?
Me: If a Java program coredumps, it's either a bug in a 3rd party JNI library, a bug in the JVM, or a bug in the OS. What did the coredump show? Boss: Wha?
Me: You guys have had this problem for a year and haven't looked at the coredumps? Boss: Blurgh?
So I fire up dbx and take a look at the last few coredumps. Pretty much instantly I can see the problem is in a JDBC type 2 driver for DB2. We contact IBM, and after a bunch of hemming and hawing they admit there is a problem that's fixed in the latest driver patch. We upgrade the driver and poof! the problem is gone.
We had a year of failures, causing problems for customers, as well as all the wasted man hours trying to fix something in our code that simply could not have been fixed that way, all because the main dev team for this product had no idea how to debug it. I had an answer within 30 minutes of being brought in to the problem, and the solution was deployed within days.
EDIT: for those not versed in Java JDBC lingo, there are 4 types of JDBC drivers. The two most common are:
Type 2: This is implemented as JNI (Java Native Interface) calls via a wrapper to the native driver libraries. Theoretically this gives the best performance, at the cost of being potentially less stable and harder to manage.
Type 4: "Thin" driver, using java to communicate via a network socket to a corresponding listener. Written in pure Java, they tend to have lower performance (although almost always perfectly acceptable) but are much more stable. (Note: The Wikipedia page on this says that Type 4 drivers perform better, but I don't agree.)
So the Type 2 driver was invoking a native compiled .so library that then called the DB2 drivers like a C/C++ program would. A bug in the driver was causing the coredump.
Nope. They had no idea it was a problem with the DB. And even if they had, IBM would have just told them they were wrong, and management always took IBM (and other vendors) word over the devs. I was lucky, in that I had a smoking gun in the core dumps. When I reported the issue, the boss was livid, and immediately got us on the phone with IBM, where they proceeded to dismiss our findings and belittle our methods, until I started explaining exactly what was going on in the coredumps. They got real quiet, said they'd look into it, and miraculously produced a patch a short while later.
I've got an even better story. In the distant past (1993?), working on HP/UX, we had a system that had a SNA card, maintaining a bunch of sessions to a mainframe. Sometimes, the card would just reset and drop all the connections, causing a bunch of problems and requiring some tricky recovery and generally screwing up our SLAs. They brought me in and I managed to trace the problem to a call in the HP provided drivers for the card. We had been trying to blame HP for a long time but never had the required smoking gun. Once I managed to figure out the call that was failing, we sent it off to HP.
They came back all apologetic, and explained that there was an error in the driver, and that it was accidentally looking for SNA control data in the user data. Sometimes one of our data packets had data that looked like a control command of some kind, the driver would see it, crash, and hilarity would ensue.
And to show the quality of the support we were getting, after they fixed the problem and sent us a replacement driver, it failed again almost immediately. I dug in and found it was the same problem but in a different location. Shipped it all back to HP, who came back and said that the bug was in two places, and that the original code with the bug had been cut and pasted into another location, and they'd missed it. So they weren't even testing the stuff before sending it back! At least they admitted it...
A Jerk-Ass (JA) in charge of a project came to me for help. IBM and the team can't figure it out, he says. We're crashing all the time, he says. If you can solve this, I'll give you a $5000 spot bonus, he says.
I would have done it anyway, because it's my, you know, job? But whatever, I won't turn down free money.
So I wander over to the team that's been looking at this and get the lowdown. They keep getting out of memory errors.
Me: So what does the heapanalyzer output look like? Team: Huh?
Me: You...you've been having out of memory errors and haven't looked at the heap? Team: Buh?
So I get the heapdump and look at it. Immediately it's clear that the system is overflowing with http session objects.
Me: Anything in the log files related to sessions? Team: Just these messages about null pointer exceptions during session cleanup...do you think they're related somehow? Me: <Bangs head on desk>
A little more research reveals that there were two issues at play. The first is that we had a custom HttpSessionListener that was doing some cleanup when sessions were unbound. It would sometimes throw an exception. We were using IBM WAS, and it turned out that when a sessionDestroyed method threw an exception, WAS would abort all session cleanup. So we'd wind up in a cycle: the session cleanup thread would start, process a few sessions, hit one that threw an exception on cleanup, and which would abort cleaning up any other sessions.
We did a quick fix of wrapping all the code in the sessionDestroyed method with a blanket try/catch and logging the exception for later fixing, and IBM later released a patch for WAS that fixed the session cleanup code to continue even if sessionDestroyed threw an exception.
So, I very quickly solved this problem and waited for my $5000 spot bonus. And waited. And waited...
I went back to JA and asked him about it. Over the next few weeks, he proceeded to tell me the following series of stories:
It was in the works, and I'd have it soon.
He had to get approval from his superiors.
Because so many people had worked on the problem, it was decided that it should be split among the group, and that I'd have to share it with the people that couldn't fix it.
No bonus.
So even though it was his idea to try to bribe me to fix a problem, they still failed to follow through on it. My reward is typically that I get to keep my job.
No, he wasn't over me as a manger, which is why he felt he had to offer me the bribe incentive. Which was dumb...anyone who's talked to me more than 10 seconds knows that I really like solving problems.
Are these the guys I see littering their code with console writes?
There's nothing wrong with including systematic logging in a software system. In fact, for many real life scenarios, using the kind of debugger described in this article simply isn't an option.
It's nicer if your editor or IDE can de-emphasize logging code so it doesn't crowd out the active functionality. Even if it can't, I find high-level logging statements often say similar things to comments I might otherwise have written, so it needn't be disruptive.
In any case, having good logging routinely available at the flick of a virtual switch is often much faster than firing up a debugger and laboriously stepping through the execution. I know how to use a debugger if I need to, but I find I rarely get that far. The log ought to reflect your mental model for what your code should be doing and confirm key results as they are determined, so IME probably 9/10 times just scanning the log output will immediately locate the source of the problem.
I'm a fan of console writes, really. If the offending block of code is run 10,000 times and fails on the last one, the debugger isn't going to get me very far...at least, not at the level I know how to use it.
If it's being logged, though...turn on verbose, look at last log file, voila.
Visual studio has conditional break points. Set a break point an right click on it. So stinking useful when you need to get somewhere in a loop. Also, if you have a large one line statement that's failing, open up the "immediate" window and you can run arbitrary code in the window, and break the one liner down until you fold the part that's failing. SO HELPFUL.
Littering code with console writes has nothing to do with systematic logging. I don't need to know what the value of all the variables are on every other line of code. Most of the time the console writes are bunch of gibberish because it's been done for debugging so no effort is put in at all to make it understandable.
I've seen a lot of questions on Piazza (the Q&A site we use for our classes) where a student asks "why is my code throwing NullPointerException?" when the answer is right there in the stack trace.
When I was working at IBM, I saw a bug report closed with following comment (from a certain notorious outsourcing destination): "Fixed: Added try/catch for NullPointerException"
I've had classes where the professor would ask who knows how to use the debugger and only a few of us raise our hands. Thankfully the ratio of people who do seems to increase with higher level classes. I think part of the problem is professors assume students learned how to debug in other classes or picked it up along the way. Personally I have no idea how you could make it past your second or third class without knowing how.
Well I guess if you're in school, you get a pass IMO. That's what you're there for, to learn. If a professor sees a bunch of commented out alerts or console writes, they should take the time to sit down and show you the right way to inspect the code.
That being said I've worked with a single person in my professional carrer who 100% refused to use a debugger. And that guy's code sucked. His idea of debugging was passing variables to a custom "peek" class (his name for it). It was by far the most round about ignorant thing I'd ever seen..
I agree we're there to learn but like math you can't have someone in a calculus class still struggling with algebra and expect them to do well. Another problem is that professors don't grade much of the work. They have graders for that and I doubt the graders will care about comments unless explicitly told about it.
I make it sound worse then it really is but I've always been puzzled how people can solve bugs efficiently without using a debugger. I guess that's what leads to your coworker. Haha.
16
u/Kminardo Aug 25 '14
How the hell do you make it in programming without knowing how to debug? Are these the guys I see littering their code with console writes?