Scott Leberknight's Weblog

Dave Thomas at it again, giving a really interesting presentation on what exactly Open Source development is, how it works, and what can be learned from they way it is written. I think the main points in this presentation were that open source is driven by a need (e.g. a developer scratching an itch), is measured the value or utility it provides, and is controlled by evolutionary pressure. This last point is interesting, since it basically translates to survival of the fittest. The evolutionary pressure ensures that only the best open source software thrives, the average open source projects survive, and the worst die off and are never heard from again. This formula makes it "likely to produce high-quality, applicable products." As a result, many developers prefer the best open source products over the best commercial offerings.

Dave mentioned several interesting tidbits that jive well with the survival of the fittest in the open source world. Specifically, that open source projecs are not "utopian communes of loving developers" and are instead characterized by dictator-driven meritocracies where the best rise to the top; brutal honesty between developers; and a focus on delivering value. What a shame most companies refuse to learn from this model.

The last two sessions last Friday at NFJS were spent listening to David Geary explain the wonderous nature of JavaServer Faces (JSF). He started by giving a brief history of Java on the server-side focusing mainly on Servlets, JSP, and Struts. David is a member of the JSF expert committee, and so obviously likes JSF. I, on the other hand, do not like it at all.

According to Geary, JSF is both a "better Struts" and as "the standard web application framework". I think neither is true. First, in many ways JSF is similar to Struts. That's not surprising since Craig McClanahan, Struts' creator, and David Geary, the 2nd Struts committer, both helped to define JSF. That doesn't mean it's better though. Second, how is JSF "the" standard web application framework? Perhaps in Sun's distorted view it is, but I doubt it is to WebWork or Tapestry users, or how about .NET users? For example, though EJB is a standard, how many people have been running away from it at full speed towards lighter-weight frameworks like Spring and HiveMind?

JSF was designed by a committee, and it looks like it. JSF tries to satisfy a ton of requirements. It tries to define a flexible and extensible model for creating UI components. It tries to separate GUI rendering from the actual GUI components, to allow a component to be rendered into different view technologies. It has an event model, a validation framework, a navigation framework, and supports i18n. It doesn't seem to do any of these things particularly well, however, and seems overly abstract and convoluted right now. Perhaps it'll get better in the future but I'm not interested right now.

One thing about JSF that really irks me is the lack of useful GUI components, such as a DatePicker component or a powerful Table component that supports sorting, pagination, etc similar to DisplayTag. According to Geary, the JSF committee was a year and a half behind schedule because of infighting between Apache and Sun over the future of Struts versus JSF. After they resolved their differences they actually began doing something but then were pressured to "get something out". Due to this their strategy was to provide a flexible and extensible component model from which you can build complex component types. While it's nice to have that flexibility, I think it is absurd that they simply left out some of the most useful and needed components. I'd have gladly waited another year for them to put JSF out if they had included useful GUI components out of the box.

Another thing that is annoying is that JSF was designed with tools in mind, and in fact was designed (more or less) for vendors to create point-and-click GUIs such as Sun Studio Creator. Designing a framework specifically to permit tooling is not necessarily a bad thing, but I think a framework should be not painful to use without tools. JSF development without tools seems painful to me. Sun's position, as stated at JavaOne and by Geary at NFJS, is to use JSF and the supporting tools as a way to lure millions of Microsoft VB developers to Java development. Is that really a good thing?

The first session I attended at the Reston No Fluff Just Stuff conference was "State Machines: Moving Logic into Data" by Dave Thomas. It was, as pretty much all of Dave's presentations, very informative and well done. Dave started with a very simple state diagram and some Java code that represented the various states, transitions, and events. He showed how quickly you can get yourself into trouble using a code-driven approach rather than a data-driven approach, with masses of conditionals within conditionals, switch statements, etc.

We then saw some serious refactoring into a much more practical table-driven approach where the state machine is described in a database table. We then moved into a discussion of the GoF State Pattern and showed how that is not really the best way to implement state machines, and how you end up with some serious code bloat using the State Pattern. We then moved into State Machine Compilers, the specific example being SMC, the State Machine Compiler available on Sourceforge. Interestingly, SMC generates code that follows the State Pattern! Of course, since the code is generated for you, it doesn't matter all that much, precisely since the code is generated for you.

Dave ended the session with four hints. First, make life easy when possible by using strings rather than numbers for states and events. In addition, add debugging hooks on state transitions. Second, states are long lived, meaning state can be stored in a database for long-running business processes. A state is a "stable, quiescent place" until an event occurs which triggers a transition to another state or back to the same state. Third, state machines are business logic. If you put the logic in data not code it is easier to change the logic. This data-driven approach can also allow business users to change the logic by modifying data rather than requiring a code change and redeployment. Fourth, isolate and test. Specifically, separating your state machine from event generation permits separate unit testing of individual transitions using mock events.

Just got home several hours ago from the Reston No Fluff Just Stuff conference. It started on Friday and ended Sunday afternoon, and as usual was packed with high quality technical content and excellent speakers. I was too lazy to write any blog entries during the conference sesions, plus I probably would have gotten behind in the sessions trying to learn and write simultaneously! But over the next couple of days I'm going to try and write down descriptions of some sessions and conference highlights.

I'll start by mentioning the keynote that Dave Thomas (of the Pragmatic Programmers) gave on Friday night was really interesting. It was entitled "The Art in Programming" and talked about whether programming is art or engineering. He gave lots of really interesting examples of famous artists, showing their "sketches" and the final products. It was intriguing to see sketches made by Da Vinci, Michelangelo, and others as they tried to gain an understanding of a problem, and then see the final painting. The important point is that you cannot just start off doing the final product; you first have to try some things out, make some preliminary "sketches", figure out difficult parts of your problem before you can dive into creating the final product, whether it be software or a painting. He noted that in the software industry, though, developers are somehow expected to go straight to the final product without doing any sketches, without following "tracer bullets" to get closer to the target, without gaining a better understanding of their problem. This is exactly why most software projects fail. Many managers and the general culture in the software industry believes it is wasteful to do any "sketching" and that developers are supposed to be "professional" and "get it right the first time".

Another suggestion he made was the duality that without art, there is no engineering; while without engineering there is no art. I for one agree and enjoy the art part of software development just as much as the engineering part. It is really satisfying to start with a blank editor buffer, type some text, press compile, and then a program spring into existence which can perform all kinds of useful functions. I am in no way doing justice to Dave's presentation. Suffice it to say it was a great presentation. If you get the chance to see Dave speak, jump at the chance. He speaks regularly at the No Fluff conferences around the country.

One project I'm working is using Hibernate as the persistence framework. We are writing the Hibernate XML mapping files and then generating the POJO model classes. We initially began writing the mapping documents using the Hibernate built-in types like "string", "integer", and "float". A potential gotcha to watch out for is using primitives rather than the wrapper types for numeric fields. The reason is simple - there is no way to set a primitive to a null value since they always assume default values in Java. So if you have a table in your database with a numeric column that allows nulls and a corresponding POJO that uses a primitive to represent the field, when Hibernate tries to hydrate an object from a row in that table it will get really upset because you cannot set a primitive to null!

When generating the POJOs from the Hibernate XML mapping files, we found that some fields in the POJOs were generated as primitives instead of the wrapper types. I am not really sure how exaclty Hibernate decides whether to generate a primitive or a wrapper type, and it doesn't seem consistent as to whether a primitive or wrapper type is generated. Maybe it depends on the database dialect or something like that. In any case, our solution was to set the type attribute of the property elements to the fully-qualified class name such as java.lang.Integer and java.lang.Long rather than integer and long to force Hibernate to use wrapper types no matter what. For example:

<property name="quantity" column="quantity" type="java.lang.Integer"/>

This ensures that the field in the generated POJO is declared as a java.lang.Integer and can handle null values. I've read the Hibernate reference documentation, JavaDocs, and Hibernate in Action, and none of these resources seem to mention (that I've found) that you can do this. There is only one example in Hibernate in Action of using the fully-qualified class name as the type for property elements, and that is showing the output of using Middlegen to generate Hibernate mapping files from a database. I would think this would be a common issue and addressed in the books and documentation but apparently not.

I learned the other day not to inherit a "log" variable from a superclass, or else it really messes up your ability to adjust logging levels in Log4J. Normally I don't do this but the base class was a utility class in my "test" tree and would not actually be deployed to production, so I was being lazy. Without thinking too hard about it, I created a protected log variable in a base JUnit test class so other tests could use it if they needed to perform some logging:

protected final Log log = LogFactory.getLog(getClass()); // don't do this!

So in one test case derived from this base class, I used this log to do some debugging. Later I was messing around with logging levels in the Log4J configuration and I quickly found out that changing logging levels for various packages wasn't working as I expected. Now, I expected this to work fine since the call to getClass() will polymorphically return the actual type instantiated at runtime. But for whatever reason changing the logging levels wasn't working. The base test class resides in a different package than the derived test class. I didn't want any logging messages from the base test case but did want to see the log messages from the derived test. However, log messages displayed from both the base and derived class. The reason is simply because the runtime type does in fact return the type instantiated at runtime! The culprit is that the derived class is in a different package than the base class, and therefore changing the log level for the package the base class resides in has no effect.

I changed the code so that the base and derived test class each has their own private static final log variable, which is what I normally do in my classes. So the code looks like:

private static final Log log = LogFactory.getLog(MyClass.class)); // This works great!

Now the package of the logger can match those defined in the Log4J configuration, and we only have one log per class rather than one per instance of a class. This fixed everything and now I can selectively turn logging on and off for the various packages the way I would expect. Thinking back this all makes perfect sense and it was not a good idea to create that protected log which could be used by derived classes. A log should be a private component of a class anyway. The only thing I don't so much like is having to use MyClass.class in the log variable declaration, since getClass() is not static and cannot be referenced from a static context. But this isn't that big a deal since most IDEs will fix this reference if you rename refactor the class.

On a new project I'm working on, we are using Hibernate as the persistence framework and Spring for DAOs using its HibernateDaoSupport class, declarative transaction management, and integration with Struts, among other things. We wanted to effectively unit test the persistent classes without writing the boilerplate code in our tests to insert and clean up test data. In the past I've written my own DBUnit-like framework, nowhere near as extensive, but it did the job. This time I decided to try DBUnit.

It took all of about 15 minutes to download, install, and get DBUnit integrated into our development environment. In DBUnit you subclass DatabaseTestCase, override two methods, getConnection() and getDataSet(), and provide test data in XML files. I didn't want all our developers to have to actually implement these methods every time they write a database-driven test case, so I extended DatabaseTestCase with the help of Spring to create default implementations of these methods in a BaseDatabaseTestCase. Spring is used to mainly to configure a DataSource from which a connection is obtained, in a dbunitContext.xml file which resides in the same package as BaseDatabaseTestCase. So the getConnection() method simply obtains the DataSource from the Spring application context defined in the XML file.

This solves the connection problem for all tests, but what about the dataset needed by DBUnit to prime the database prior to test execution? The dbunitContext.xml permits you to define a standard set of files containing DBUnit datasets that all extending tests will get by default. This is optional and should not be used to have the entire database populated before each test. You could use it for populating some tables that are required to always be populated in your application, such as reference tables or user tables or something similar.

The BaseDatabaseTestCase implements getDataSet() to combine the DataSet found in the Spring application context with a default DataSet for the specific test. This default DataSet is defined in an XML file that resides in the same package as the test and is named the same as the test with the file extension .dbunit.xml. For example, suppose you have a test MyTest.java which extends BaseDatabaseTestCase. Then to define the default DataSet for this test, simply create a file named MyTest.dbunit.xml in the same package as MyTest. BaseDatabaseTestCase automatically looks for this file and loads the DataSet contained within it. If there are any standard DataSets defined in dbunitContext.xml, they are combined with the default DataSet to prime the database for the test.

This allows developers who need to create unit tests for database-driven code to simply extend BaseDatabaseTestCase, write an XML file containing the test data, and implement standard JUnit testXXX() methods. Pretty nifty.

I have a friend who works on a project with a manager who typifies the problems with the software development industry. We'll call this developer Bob and the manager Jim. Over the past couple of months Bob has mentioned some very disturbing things that Jim has said and done in running, if one could call it that, the project. For example, Jim started off by not allowing Bob, the lead developer, to use a persistence framework and instead mandated he use straight JDBC because of the "risks" of the frameworks. This is of course nonsense since the real risk is not using a tried and true persistence framework and rolling your own.

Another brilliant decree made by Jim was that developers cannot spend time developing unit tests because they will slow the development effort and are not really needed because the developers "are very careful to write bug-free code". Excuse me? Did I just actually hear that? Yes in fact I did. As a developer I am careful to not introduce bugs into my designs and code, but I in no way do I assume I write bug-free code. Unit tests of course are one way to ensure high-quality code and in fact speed development, since many bugs are caught and fixed much earlier than they otherwise would be. In addition, a quality suite of unit tests that can be run at the touch of a button are wonderful for regression testing new changes. Even worse, Jim told the customer that the developers were very careful to write bug-free code and that they would only need one week to fix any and all bugs that were found by the integration testers and the customers! Bob was not at all pleased to hear this of course.

Another fun thing Jim has done on several occasions is to inform the customer that the developers intended to be done early, so some additional new requirements could easily be added. In reality, Bob told me many times there was no way they were even going to meet their original deadline much less with loads of new requirements. Finally after recognizing they were not going to make the initial (unrealistic) deadline, Jim got the customer to extend the deadline several months. Then Jim did something that Fred Brooks would, I am sure, not advocate. Jim told Bob that since they had a few more months, this would be the perfect time to bring on five more developers to "speed up development". So much for Brooks truism that adding developers to a late project makes it later, whcich has been proven out over several decades.

In the end, Bob has started looking around for other projects and probably not be with that project much longer.

Our brand new baby, Sarah Jende Leberknight, was born September 17, 2004 at 9:09 PM. Check out some photos in our photo gallery. It was unbelievable to say the least, and just awesome! Ann Marie did great! She went into the hospital at around 4:30-5:00 and just about four hours later we had a new baby girl. And since we had not found out the sex, it was a complete surprise. Apparently I interpreted the cryptic sonogram correctly - the sonographer wrote LO and RO at one point, and I said that meant Left Ovary and Right Ovary so it must be a girl. I don't think that was actually true but I liked telling everyone that anyway!

We are now at home and Sarah and her mommy are both sleeping peacefully! Soon, however, I think things might get a little more hectic around here, though!

You know the type: the ones who send out mass emails containing a PowerPoint presentation detailing the entire infrastructure for a company on a couple of PowerPoint slides, complete with lots of rectangles and arrows connecting them. And of course it is that simple, isn't it? Draw some rectangles, label them with things like "Single Sign-On", "Enterprise Search", "Authentication and Authorization", "Enterprise Storage", "Dynamic Workflow Engine", etc. and Voila! you've got yourself a ready made enterprise architecture.

I cringe every time I see a new one of these presentations, because it reinforces in many non-technical managers' views how easy all this software development stuff is - simply draw some rectangles and maybe some ovals and cirlces too, label them with the appropriate buzzwords, and you're done. Now you only have the minor detail of turing that vision into reality in the form of working software. It is not that I have a problem with people having a vision or creating high-level architectural diagrams. In fact I create them myself and think they can be very valuable. The problem is when the peeople creating them have either never developed software or they have not developed software for so long they are completely out of touch with today's agile development environment.

Several years ago I worked with an IBM software architect who told me about an interesting policy IBM has. He said that all IBM architects are required to implement a core piece of the system they are architecting. This does several things. First and foremost, it ensures the architects cannot get away with PowerPoint Architecture since they have to implement a core piece of their architecture. Second, they get much better feedback on the quality of their designs since obviously they can tell whether the architecture makes sense and works well. Third, it helps the architects maintain their coding skills. And finally, it boosts morale of developers working on the project since they see the architect validate his or her position by writing real, production code.

I don't know if this is actually a real IBM policy or not, but I think it makes a lot of sense, and I wish more architects did actually wrote some code once in a while. Too many consider themselves above writing "low-level" code. These are usually the same architects who have never heard of the Gang of Four or Martin Fowler's Refactoring book, though!