Scott Leberknight's Weblog

One of the projects I am involved with a work recently had its design review with major stakeholders - the typical dog and pony show nonsense where people who have no business making technical decisions try to impact your project - and one of the outcomes was that an "architect" was concerned that some diagrams were not "standard UML." This person basically stated in an email after the review that we should "prefer" standard UML diagrams, apparently because standard UML is, I suppose, standard. There was no other reason given. He then attached a sample formal UML state diagram from some other project. I looked at it and could not even understand it due to the level of complexity.

I sent out an email to my team telling them to ignore what that "architect" said and to continue producing documentation that fits the intended target audience. In this case that meant not only technical personnel but also functional and business domain experts who do not know UML, have never probably heard of it, and do not care a lick about whether a diagram is UML. The only thing they care about is whether they can understand it in the context of their domain expertise. Period. The diagrams in question were simple screen flow diagrams done using PowerPoint. They consisted of squares with text inside (the screen name) and arrows going between the squares to show the navigation between screens. Nontechnical people can easily glance at this type of diagram and grasp it. Try taking a simple diagram like that and shoving a formal UML model in front of them. I am guessing most will immediately tune out as you start to explain how the little solid circle is the starting point and the circle with the solid circle inside is the end point. Or is the solid circle the end point? Well, before we can build our system we better make sure we get those correct, or else the code monkeys might just code everything in reverse, right?

The point is this: just as you should prepare a briefing according to your intended audience, you should do the same for software modeling. And you should only model as much as you need, using whatever notation is comfortable for your team, to understand your problem and begin implementation. No more and no less. If you need to show a flow diagram to a business domain expert, please stop trying to educate them on formal UML notations and just use simple notations so they can concentrate on whether your model is accurate, not on trying to remember what symbol means what. And even between developers, how many times do people get bent around the axle when whiteboarding some classes and their relationships? Does anyone really remember which diamond (solid or empty) is aggregation and which is composition? Does it even matter if your colleague understands what you mean? I vote "No." So, resist the formal modeling and resultant analysis paralysis and vote "No" on the formal UML referendum by your local PowerPoint Architect.

My question is this: who cares and why should we care if we can trace from a functional use case down to one or more classes, or even methods? Recently I have been discussing this issue, as I might be required to worry about this type of traceability on an upcoming project. My main problem with this whole idea stems from the fact that well-designed and abstracted OO code typically will not map in any sort of one-to-one fashion from a use case to code. This is mainly because of the fact that use cases are a functional breakdown of system requirements, while OO (and nowadays AOP) code that implements those requirements does not follow a structured, top-down breakdown. Thus there is not a one-to-one mapping of requirements to code. So that being the case, in my experience at least, what good does it do you to be able to show that a specific use case links to a whole mess of classes? Seriously, if someone can show that use case "Edit FooBlah" links to classes as well as other artifacts such as XML files, JSPs, properties files, etc. how does that truly help them? What is the purpose other than simply being able to do the linkage? The "Edit FooBlah" use case in say, a Struts-Spring-Hibernate typical web application crosses all tiers and touches a whole mess of artifacts ranging from Java classes to HTML, JavaScript, CSS, JSPs, Struts configuration files, resource bundles, Spring configuration files, Hibernate mapping files, utility classes, etc. etc. etc. If I've properly architected the application into well-defined layers, however, I already know exactly what artifacts make up a particular use case since almost all the typical use cases will be built in the same, consistent way.

I can clearly see the purpose for linking bug fixes, issues, etc. to the code that was altered to perform a fix. For example, if a bug is found, someone enters it into a tool like Atlassian JIRA. It then is reviewed, worked on, and code is checked in against that bug. The affected code tends to be concentrated (hopefully) into a small number of classes and is is easy to see how a particular issue was resolved. JIRA and similar tools are also useful to document new feature requests, improvements, etc. once a system is released into at least alpha or beta. In addition, tracking issues like this allows release notes to be easily assembled from reports run for a particular release number and to inform users about fixes, changes, and other things that might directly affect them. People can also search and find whether there is a workaround to a specific issue or whether someone else already reported the bug they found. There are lots of other good reasons why we use issue tracking systems like this.

But as for linking requirements, in whatever form they might be, directly to code, I do not see the tangible benefits. What if your requirements are on index cards in a XP-style development shop? Or if they are in textual use cases in a wiki? Is anyone really going to gain tremendous benefits from being able to tell you that a use case links to the following 27 different files? And the management of this information becomes much more complicated over time, as requirements change throughout the initial development as well as product maintenance. If a feature a product contains is fundamentally changed or improved, do I care that I can go back and trace the old requirements to code that has been removed from the system long ago? To me the most important thing is to have a set of automated tests that define whether a system does the tasks it is intended to perform. This can be unit tests, functional test suites, user acceptance tests, performance and load tests, and whatever other types of tests you want. But in the end they determine whether the product meets its users needs. Period. Users do not care to look at some gigantic stack of paper containing system requirements. They simply want a system they can easily use to perform their job. That's it. It's that simple. But for whatever reason many people in our young and immature industry still continue to believe that documentation such as requirements traceability, test plans, system requirements specifications, system design documents, system security plans, data migration plans, and umpteen other documents are the most important facet in developing software.

At JavaOne in 2004 I saw a demo of some products that claimed to easily and seamlessly (of course) provide linkage between requirements and code. So here's how it worked. You go into the requirements tool, select the requirement your class or (as the demo showed) method satisfies, and then drag and drop the requirement directly into your source code. The tool suite, which of course required you to have all the various parts of the suite, then created a code comment "linked" to the requirement and apparently was then able to track exactly what code linked to each requirement. Notwithstanding how brittle this solution was - just go into the source and delete the comment - it was also invasive and ugly. Assuming you had a sizable system, soon your developers would be spending all their time linking code to requirements, and trying to figure out which requirements some little utility class that everyone is using (e.g. StringUtils) maps to, which of course is an exercise in futility.

Many times you need to limit the number of results returned by a query, for example if you have the potential for a user-created query to return a very large number of results. Since this pretty much always happens in applications, you have to deal with it somehow. But most books and articles don't really talk about the gory details, since many times the mechanisms you must use are database and vendor-specific. The two databases I've worked most with are MySql and Oracle. Limiting results in MySql is ridiculously simple, and I really wish their syntax was part of standard SQL. For example, suppose you have a person table and you want to search on first and last name, and you need to order by last name then first name. In addition, you need to limit the results to 25 results. In MySql you can issue the following query:

select * from person where last_name = 'Smith' order by last_name, first_name limit 25

The key in the above is the limit clause which MySql provides. MySql applies the limit after it has retrieved the results and applied the order by clause as well. Trivial. Now let's consider how to do this in Oracle.

Oracle provides a pseudo-field called rownum which seems like a promising way to specify limits on result sets. Suppose we issue the following query in Oracle:

select * from person where last_name = 'Smith' and rownum <= 25 order by last_name, first_name

When you run that query, you find much different results than you expect. This is because Oracle performs the query and applies the rownum to each row in the results before applying the order by clause. So the question is how in the world do you use Oracle's rownum in concert with an order by clause and get the correct results. I cheated. I have been using Hibernate for a while on an Oracle database and know it is able to handle this exact situation - that is, limiting the number of results and applying an order by clause to a query. So I set the hibernate.show_sql property to true and looked at what it generated for a query. It turns out to be very simple and makes use of a subselect:

select * from (
  select * from person where last_name = 'Smith' order by last_name, first_name
)
where rownum <= 25

So Hibernate is simply wrapping the original query in another query which limits the results. Since the subselect is executed first, the rownum property of the outer select works properly. Pretty slick the way Hibernate does that but it would be much easier if Oracle had a syntax more like the elegant MySql limit clause.

Unfortunately, limiting the number of results for a query is only part of the equation, since you normally also need to provide some type of paging functionality. There are tools, such as the very popular DisplayTag custom tag written by Fabrizio Giustina, to help with the display of data in a nice table that provides paging and sorting. Unfortunately, most of them want to have all the results completely in memory for the paging to work properly. I hate doing this since you are just taking up memory by putting data in the user session that the user will probably never even access. Thus you are just wasted memory by storing the complete result set in memory. I have always tried to implement paging so that you go back to the database for each page of information and not store anything in the session. You think Google stores a couple million results in memory for each search? Doubt it.

So how can you accomplish paging in the database? Again you normally have to use vendor-specific syntax. For MySql you simply use the limit clause, but you specify two parameters: the start row and the number of rows to retrieve. For example, suppose we want the first page from the person search assuming a page size of 25:

select * from person where last_name = 'Smith' order by last_name, first_name limit 0, 25

Simple. Suppose you want the second and then third pages. The queries are:

select * from person where last_name = 'Smith' order by last_name, first_name limit 25, 25
select * from person where last_name = 'Smith' order by last_name, first_name limit 50, 25

Again, in MySql this is really simple to accomplish. In Oracle, not as simple since you now have to get additional operators involved, specifically row_number() and over() functions. More on that later...for now I suggest you migrate all Oracle databases to MySql. Wouldn't that be nice? :-)

One last thing. The really nice thing about Hibernate is that it understands how to limit result sets and perform paging operations in all the databases it supports. Your code does not need to care. You simply do something like:

Criteria criteria = session.createCriteria(Person.class);
criteria.addOrder(Order.asc("lastName"));
criteria.addOrder(Order.asc("firstName"));
criteria.setFirstResult(50);
criteria.setMaxResults(25);
List results = criteria.list();


Then, Hibernate will perform the appropriate vendor-specific query based on the dialect you configure Hibernate with. Thus, using a tools like Hibernate allows you to write queries at a higher level and completely abstract the actual query that is performed on your specific database at runtime.

Yes I have been watching Star Wars IV: A New Hope lately in case you are wondering about the reference to Vader's "I find your lack of faith disturbing" comment. Anyway, recently I've come across a couple of situations where I was doing some consulting work and found that people were using -- no, relying completely -- on their IDE to perform builds of J2EE applications. One even included an EAR, EJB-JAR, and WAR file all being built exclusively by the IDE. Then Novajug launched a poll yesterday on what people were using as their build tool. As of about 10 minutes ago people using their IDE stood at 20%. Now granted the number of people who have voted thus far is small and the 20% only translates to 11 people. But I find even that number of people using only an IDE to build applications alarming and disturbing. As for my recent consulting, I won't mention the IDE by name, but it is a commercial IDE that allows you to define, using mounds of dialog boxes, in excruciating detail everything about your build, which is where they had put everything! And all of those settings were checked into version control no less.

The reason I was helping them out was because one developer's IDE would simply not build the WAR file, instead giving a single one line error message: java.lang.ArrayIndexOutOfBoundsException. Nice. Well, with this amount of detail I naturally was unable to help him out, and suggested in passing that they try switching their build system to Ant rather than the IDE. And there really isn't much more to say in this paragraph about it either, so I'll continue to the next one.

I suppose the reason I thought about writing this is because I pretty much assumed everyone nowadays uses Ant to build J2EE applications. Apparently I am wrong. Not the first time. Anyway, another tidbit about that project I was helping -- they had no "test" tree in their project, thus no, nada, zero unit tests. Maybe bad practices run amuck and are concentrated on a small set of projects, such that if someone is using only their IDE to do builds, perhaps it's not all that surprising they aren't unit testing either. But that's just random speculation.

At this past weekend's No Fluff conference one thing was missing that has been at every conference I've been to for as long as I can remember...a session on Struts. There was not a single session on Struts, unless you count the session on Shale which I suppose is possibly the last gasp for Struts before it completely succumbs to JSF. Well, I suppose I don't really know enough (or anything) about Shale nor am I enthralled about learning, since I think most people will use straight JSF or Spring MVC or perhaps even Tapestry or WebWork. Though, other than Erik Hatcher I don't personally know anyone using Tapestry on a production project and I know only one person using WebWork. Erik loves Tapestry and though it looks pretty cool I am much more interested in Spring MVC simply because I am already a heavy Spring user. Everyone else is still on Struts it seems and a select few people I have talked to are using Spring MVC.

Just attended the Reston, VA No Fluff Just Stuff conference this past weekend. As usual there were lots of high-quality speakers and content. This time around there were a bunch of new sessions to choose from and I attended a bunch of sessions on topics that I don't know very much about like Swing GUI development, cryptography, Java Security and JAAS, even a session delving into the depths of JavaScript. I never knew how dynamic JavaScript actually is and some of the basic rules it uses when it parses and executes code. I plan to document all the sessions I attended over (hopefully) the next week or three.

IntelliJ 5.0

Suppose you have a web application with a field named "amount" on a form. Suppose also the field should convert its input to a number with at least 2 fractional digits. Ok, JSF provides a <f:convertNumber> tag that provides an attribute minFractionDigits. So far so good. We could put the following code in our JSF page:

<h:inputText id="amount" value="#{payment.amount}">
    <f:convertNumber minFractionDigits="2"/>
</h:inputText>

Makes some degree of sense. Whether I want to place this conversion directly in the view layer is debatable. Actually not really. I don't want to put it there but that's what JSF wants you to do, so for my example I did just that. Then I started playing around, submitting the form with various values. Entering valid numbers works fine, e.g. 75.00, 45, 21. So let's see what it does with something that is not a number. Entering 'abcd' produces a conversion error which gets output to the screen. The default error message is 'Conversion error occurred' and changing it to something like 'The amount could not be converted to a number' turns out to be exceedingly difficult in JSF on a per field basis, but that will have to wait for another blog. Ok, what about a mixture of numbers and letters? Entering '50xyz' produces...50.00. Um, wait a second. Apparently the invalid trailing letters are ignored. Last time I checked, if you try to feed that number to the standard factory method Double.valueOf() you get a NumberFormatException as you would expect. Why would JSF accept that value and convert it as a valid number? That just doesn't seem to make any sense at all.

Oh but wait. Before assuming this is purely the fault of JSF, I did a little more and wrote a JUnit test to test out parsing numbers using the NumberFormat.parse() class. The string "50xyz" is successfully parsed using this method. The JavaDoc for NumberFormat.parse() actually states "Parses text from the beginning of the given string to produce a number. The method may not use the entire text of the given string." That last sentence tells the story. The method doesn't necessarily use all of the input string, which I think is a bit odd. What if the input string were "50?75"? The parsed value is 50! This is not only odd, it seems just plain wrong to me. What if someone meant to type in "50.75" but typed "50?75" (since the "?" is next to "." on the keyboard? The parse does not flag an error and would then cause a wrong input value to be accepted. So apparently the culprit here is actually the JDK, not JSF. So this just illustrates that you need to thoroughly test your applications using many different forms of invalid data to ensure your application actually considers it invalid. But since JSF (apparently) uses NumberFormat during its conversion phase, this type of data will be converted successfully and not flagged as erroneous input.

Note that NumberFormat.parse() is not the only method with a lenient parse policy. The SimpleDateFormat.parse() method also does. For example, the string "4/2005edfnnas", when parsed with the pattern "MM/yyyy", is parsed successfully to the date 4/1/2005. And, the JSF <f:convertDateTime> tag probably uses SimpleDateFormat as well, since the above input string converts to "4/1/2005" in JSF. So if you are using JSF and its converter tags, be aware of the leniency during parsing.

So I deployed one of the Core JSF examples to Tomcat and tested it out. In case anyone actually cares, it is the "Deleting Rows" example in chapter five. Anyway, since I normally use Firefox that is what I was using and the example worked as advertised. I have no idea why, but I opened up Internet Exploder and tried the example. It didn't work properly. The example basically has several checkboxes that, when changed as detected by a JavaScript onchange event, called submit() to submit the form. (Whether clicking a checkbox should cause a round trip to the server is a different matter altogether, but that is what this example does.)

So in Firefox this worked great. When a checkbox was clicked, the onchange even handler fired and submitted the form. but in IE what happened was quite different...nothing at all actually happened when I clicked the checkbox. That is, until the checkbox lost focus by tabbing away or mouse clicking somewhere else. The solution is relatively simple. Just change the onchange event handler to onclick. When a checkbox is clicked, the handler fires and the form is submitted. This works in both IE and Firefox.

My initial reaction was to praise Firefox and then to curse IE, but I wondered whether what the HTML specification said about onchange. Here's what the HTML 4.01 specification has to say about onchange event handlers:

"The onchange event occurs when a control loses the input focus and its value has been modified since gaining focus. This attribute applies to the following elements: INPUT, SELECT, and TEXTAREA."

Well, well, well. Apparently I needed to check (no pun intended) that initial reaction and in this case state that it is IE that is following the specification to the letter, not Firefox. IE does in fact wait until the checkbox has lost focus and that its value has changed. Prior to posting this, I submitted the "fix" of changing to use onclick to the Core JSF website. Using that method works in both IE and Firefox, and therefore probably Mozilla and Netscape as well. Anyway, the point here is simply that not everything is as it seems: in this case IE and Microsoft got it right, and Firefox has added extensions or at least "enhancements" to the way onchange is supposed to work.

Stuff like this is incredibly annoying, though; you cannot simply take the specification at face value because different vendors have slightly different implementations. This is one reason I dread having to test web applications in every single browser. And it goes to show why having specifications that are followed and that you can count on is very important. After all, how much time and money has been wasted over the years on getting web interfaces to work properly in all browsers, and even different versions of the same browser? Even on my current project at work we found a difference in the way Netscape 7 and IE 6 handle a certain thing in JavaScript and had to write different code for Netscape and IE. This should not be required but apparently is still an issue. Not as much as several years ago to be sure, but it is still not perfect.

So if JSF forms only permit POST requests, how would you go about creating a link to submit your form instead of a button? Whereas you use the <h:commandButton> tag to create an HTML submit button, you use <h:commandLink> to create a link that will submit your form. So far so good, right? Not so fast. Because JSF only permits forms to be POSTed, the JSF generates an HTML link with an onclick even handler that sets some hidden form variables and then submits the form. Lots of magic incantations going on here that are out of your control. Check out the following example code.

Suppose you have a form whose id is "registerForm." Also suppose you want to have several text fields and a "register" link to submit the form. In your JSF page, you write this code:

<h:commandLink value="register"/>

I deployed the JSF page containing this code to Tomcat and then viewed the generated source code. Here it is:

<a href="#" onclick="document.forms['registerForm']['registerForm:_idcl'].value='registerForm:_id6'; document.forms['registerForm'].submit(); return false;">register</a>

In addition to the above, a hidden field was added:

<input type="hidden" name="registerForm:_idcl" />

Is this the kind of code you want your web framework generating? I think I'll pass.

Another fun thing about JSF's generated HTML code is the way it names HTML form fields. It uses the format formName:componentName where formName is the name of the form and componentName is the name of the HTML form control. So in my "registerForm" the code <h:inputText id="password"/> generates the following:

<input id="registerForm:password" type="text" name="registerForm:password">

This code has the nice little side effect that it cannot be accessed in JavaScript in a simple manner. You cannot simply write document.forms.registerForm.password. Instead, due to the colon in the element name you are forced to write the following:

document.forms.registerForm["registerForm:password"].value

This seems par for the course so far while learning JSF. Every time you turn around there is some other gotcha or misuse of technology or artificial restriction being placed upon you the developer. Have fun!