Agile Phil: September 2011

Monday, 26 September 2011

Get thee to sbt

I was struggling to work out all of what the maven prototype gave me, and then struggling with figuring out maven itself. My gripes against maven aside, I came across this which walked through a "from the ground up" Hello, World application. It's not the most complete "walk through", but it uses sbt (simple build tool).

New Skeleton Application

After following the instructions, my files look like:

project/build.properites is:

#Project properties
#Mon Sep 19 22:11:42 IST 2011
project.organization=philgra.org
project.name=contactdb
sbt.version=0.7.7
project.version=1.0
build.scala.versions=2.9.1
project.initialize=false

project/build/LiftProject.scala is:
import sbt._
class LiftProject(info: ProjectInfo) extends DefaultWebProject(info) {
val liftVersion = "2.4-M4"
override def libraryDependencies = Set(
"net.liftweb" %% "lift-webkit" % liftVersion % "compile->default",
"net.liftweb" %% "lift-common" % liftVersion % "compile->default",
"net.liftweb" %% "lift-mapper" % liftVersion % "compile->default",
"org.mortbay.jetty" % "jetty" % "6.1.22" % "test->default"
) ++ super.libraryDependencies
}

src/main/scala/bootstrap/liftweb/Boot.scala is:

package bootstrap.liftweb
import net.liftweb.http.LiftRules
import net.liftweb.sitemap.{SiteMap, Menu}
class Boot {
def boot {
LiftRules.addToPackages("org.philgra.contactdb")
LiftRules.setSiteMap(SiteMap(Menu("Home") / "index"))
}
}

src/main/org/philgra/contactdb/snippet/HelloWorld.scala is:

package org.philgra.contactdb.snippet
class HelloWorld {
def howdy = <span>World!! The current date is: {new java.util.Date}</span>
}

src/main/webapp/templates-hidden/default.html is:

<html xmlns="http://www.w3.org/1999/xhtml" xmlns:lift='http://liftweb.net'>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta name="description" content="" />
<meta name="keywords" content="" />
<title>Hello World</title>
</head>
<body>
<lift:bind name="content" />
<lift:Menu.builder />
</body>
</html>

and src/main/webapp/index.html is:

<lift:surround with="default" at="content">
<h1>Hello World Snippet</h1>
Hello <lift:HelloWorld.howdy />
</lift:surround>

I won't go through the explanation given by Sanj, you can follow the link and get his explanation which seemed reasonable enough to me :)

From the sbt console, running the "update" and "jetty-run" commands gave me a nice and happily working "Hello, World" application which I can wrap my head around. I'm sure that as I move into more complete things my LiftProject.scala will become more complicated, but as there are ample examples of folks using sbt to build their Scala projects I'll have good company.

And I won't have to contend with the farce that is XML-as-a-build-language.

Post Script: this didn't quite work. I just realized that I ended up with the correct "index.html" but the HelloWorld.howdy piece ("World!! The current time is blah") and the Menu piece (basically the lift:surround piece) don't seem to have worked. I will update this post when I get to the bottom of it...

Diagnosing the Problem
Looking at the HTML source coming through to the browser, we can sort of see what has gone wrong:

<lift:surround with="default" at="content">
<h1>Hello World Snippet</h1>
Hello <lift:HelloWorld.howdy /> </lift:surround>

Basically, the Lift code results aren't getting injected into the HTML. Why not?

Saturday, 17 September 2011

Exploring the Skeleton Application

In Starting the Project, I discovered that the skeleton application provided by the basic maven archetype "magically" gave me a working application. So...what's lurking in here?

When I get down to it, the first thing I'll want to do is write some tests. First, though, let me see what I can figure out. The page has some text "Welcome to your project!". Where is that?

src/main/webapp/index.html

<div class="lift:surround?with=default;at=content">
<h2>Welcome to your project!</h2>
<p>
<div class="lift:helloWorld.howdy">
Welcome to contactdb at <span id="time">The current time</span>
</div>
</p>
</div>

So this is Lift at work. I'm no UI or Web developer, but this is clearly HTML using div tags (rather than tables) to control layout. Both of these specify a class.

The first specifies lift:surround which ultimately is going to reference the content of main/webapp/templates-hidden/default.html . More on that file later, but for now it should be sufficient to accept that this is a way to give a common look-and-feel to all pages within the application (including the menu).

The second references lift:helloWorld.howdy which is talking about the class (and method) specified at ./main/scala/org/philgra/snippet/HelloWorld.scala :

package org.philgra {
package snippet {

import _root_.scala.xml.{NodeSeq, Text}
import _root_.net.liftweb.util._
import _root_.net.liftweb.common._
import _root_.java.util.Date
import org.philgra.lib._
import Helpers._

class HelloWorld {
lazy val date: Box[Date] = DependencyFactory.inject[Date] // inject the date

// bind the date into the element with id "time"
def howdy = "#time *" #> date.map(_.toString)

}
}
}

I'm a bit surprised to find so many imports, particularly boiler-plate looking things. Without (yet) getting into the nuances of Scala we can sort of see how the HTML <span id="time"> links into the method howdy and a binding. I have to be honest, this doesn't make a huge amount of sense to me at this point: it seems that the developer of this is trying to be too clever by half. That or Lift is goofy :)

So, that's about a wrap for now...I'll next look into the tests. There's also User and DependencyFactory classes, but at least we have the jist of how the HTML is being built.

Tuesday, 13 September 2011

Determining Value in Agile

The Problem
One of the central tenets of Agile is that "value" should be delivered as soon as possible. That sounds great on paper, but how can one value software code?

For functional user stories, this can be easy. If you believe that some sort of enhancement will allow you to increase the profitability of your product, the value calculation is straight-forward: the "value" of the enhancement is the increased profitability. But, it can also be hard: a new report might give you a new insight on a new (more profitable) way to operate...but you won't really know until you see the data in the new report. In this case, the user story has the potential for increasing value but you won't really be able to put an accurate figure on it until the deed is done.

For non-functional user stories, the question of "value" starts to become impossible to determine. An example of a non-functional story might be enhancing your logging architecture to be able to better troubleshoot problems. (Of course, you're writing perfect software with no problems in the first place, so this is a silly example...but humour me please!)

You would only pursue this a story if the current logging architecture makes it difficult to troubleshoot problems. But how much does the "difficulty" cost? And if you're writing this software for customers you don't yet have, how much does it cost? This user story falls into what I would call "avoiding the loss of value"...it's defensive in nature. Scalability, performance, operability...all of these (and many more) are important non-functional capabilities and attributes of good software, but how do they get built if you can't gauge their value?

I suspect this is a problem that a lot of Agile ISVs (Independent Software Vendors, or people who write and sell software for use outside of their own company) must struggle with. The most vocal stakeholders (the sales team and sales-oriented product managers) demand functional enhancements: they're the sexy bits that sell software. But there are a whole bunch of stakeholders on the operations and support side who aren't as vocal...and these are the folks who would derive great value on stories that avoided the loss of value.

One Idea
So, an idea I've had floating about in my head is this...what if the stakeholders in your product had to put their money where their mouth is? I think the executives of ISVs (at least those that I would want as an employer) "get" the importance of non-functional capabilities. Customers need a good ownership experience, but this is made difficult if they're always "losing value" because of poor quality, poor performance, or difficulty in its operation.

I might be so bold as to suggest that they could probably give a pretty decent guess as to how much they would like to have invested in functional versus non-functional enhancements.

So, let's assume that I give the executive team 1,000,000 gold dubloons. (Avast! Thar be maths a'comin'!), and let's say in the next release they want to see invest 70% of these invested in functional enhancements and 30% in non-functional enhancements.

The game is this...and this is really just a brain dump so it might not read that well:

Stakeholders (who represent the Agile customer interests) have authored Stories (which include both enhancements and defects). It is possible to be a Stakeholder without having authored a story, more on that later.
Each Story must be well-articulated in the sense that it outlines what the story should do to be considered "complete", and is well-groomed in the sense that the story can be earned (written, accepted, and tested) within a single iteration.
Each Story also has a "value pitch" which outlines how the story adds (or avoids the loss of) value, using qualitative and/or quantitative language.
It is up to the product management team to ensure these stories are "good".

All good stories are placed on the product backlog...REGARDLESS of what the product managers believe the value of these stories to be.

Assuming total executive dubloon allocation is D, the total number of Stories on the backlog (up for consideration) is N, and the total number of Stakeholders is S.

Each story author is automatically entitled to allocate 20% x D / N dubloons.
Story owners are therefore automatically Stakeholders.

This leaves 80% x D dubloons to spread across all Stakeholders.

One does not need to be a story owner to be a Stakeholder. Anybody with dubloons is considered a Stakeholder.
Because the executives desired a 70/30 split between functional and non-functional value-add, the remaining 80% of dubloons are allocated along these lines. The primary functional product manager is therefore given 56% x D dubloons, and the primary non-functional product manager is given 24% x D dubloons.

Any individual Stakeholder can hold at most 2 x D / S dubloons.

The product managers who have been given such a generous pile of dubloons will therefore need to seek out others who they feel can represent their respective interests. These could be other product managers, architects, sales team members, support team members...whoever they think would be able to make a value determination.
As more Stakeholders are added, the quantity 2 x D / S decreases, meaning that some stakeholders may need to give up some dubloons.

Next, Stakeholders allocate dubloons to the stories on the backlog.

Stakeholders should be independent; there should be no coercion or lobbying for one story over another. The "value pitch" is the story authors sole means of trying to attract dubloons.
A Stakeholder can allocate their dubloons however they want. They could put all their dubloons onto a single story, indicating that they believe ALL of the value of the backlog is embodied in this single story.
A Stakeholder can "spend" the dubloons in their hand, but no more. They must spend all their dubloons or give them to another person (who might suddenly find themselves as a stakeholder).

Upon completion of a Story, the development and test teams get to keep all of the dubloons allocated to the story.

They will therefore be highly motivated to complete the stories that have attracted the most dubloons, and will not be highly motivated to complete the stories with very few dubloons.
There should be some sort of real-life compensation for holding a dubloon. Maybe this is money, some sort of time-in-lieu, gift certificate, stock...something. This is to give the development and test teams a real incentive to complete stories. There ends up being a real-world budget for this effort...maybe it's $0.01 per dubloon ($10k for the release)...it doesn't need to be much. Perhaps it is some number of dollars per developer+tester.
The ratio of dubloons between developers and testers should vary by completed story. Basically, developers should be incented to get it right the first time...if they do they get a disproportionate fraction of the dubloons (maybe 120% of what they would if it were divided evenly). BUT, if testers reject a story because of a legitimate problem, the balance shifts...maybe the first retest developers get 100% of an even division, and if there is a second retest it falls to 70%. Basically, this is to incent the testers to find problems.

The stakes here aren't huge, and the development manager would be the person to police abuse of the system.

So...what does this complex game achieve? Quite a few things actually...

The executives "created" dubloons because they wanted to unlock additional value in the product.
They had an idea about the sort of value they wanted to see added...70% functional and 30% non-functional. They were able to achieve this by giving 80% of the dubloons to the two individuals who represented functional and non-functional interests.
Story authors (who are the proxy for customers) get a say too...they have 20% of the dubloons to allocate as they see fit.
Because of the "wealth cap" a wider set of individual value judgements are made. Basically, this "crowd-sources" value determination of stories.
Priority rankings now become completely trivial: development will now do whatever they can to earn as many dubloons as possible. Because dubloons are a proxy for value...value is delivered as quickly as possible.

So...am I nuts? What other ways are there to solve this problem?

Monday, 12 September 2011

Starting the Project

Setting up Version Control

First things first, we need a home:

$ mkdir agilephil
$ cd agilephil
$ git init
Initialized empty Git repository in /Users/phil/agilephil/.git/

Okay, one repository, created! No immediate plans to put onto github, though that's really more to do with the fact that I have nobody asking for it. Next, a branching model. The one proposed by Vincent Driessen at nvie.com is nicely explained, so I'll be aiming to follow this one.

$ cat <<EOF > README
This is a CRM-type of application I've created to learn Scala and Lift. It is super-basic and won't win any awards. Probably.

Follow along with me at agilephil.blogger.com.
EOF
$ git add README
$ git commit -m 'initial project version'

$ git checkout -b develop master
Switched to a new branch 'develop'
$ git checkout -b features/MultiUser1 develop
Switched to a new branch 'features/MultiUser1'

Creating the Project

So now, we're ready to rock and roll. We have a features branch called MultiUser1 into which we'll complete our first user story. The only trick: I need a project structure that Lift will like. To do this, there are a number of maven archetypes available as described on liftweb's wiki page Using Maven. I think I'll start out with "basic":

$ mvn archetype:generate \
-DarchetypeGroupId=net.liftweb \
-DarchetypeArtifactId=lift-archetype-basic_2.9.1 \
-DarchetypeVersion=2.4-M4 \
-DarchetypeRepository=http://scala-tools.org/repo-releases \
-DremoteRepositories=http://scala-tools.org/repo-releases \
-DgroupId=org.philgra \
-DartifactId=contactdb \
-Dversion=0.1

$ git add contactdb
$ git commit -m 'creating initial project structure from basic archetype'

The Using Maven page also describes a pom.xml file (the maven build metadata), but I'm not entirely sure why, as the archetype included one of these. Needless to say, lesson learned: ignore that incomplete-looking pom.xml and stick with the one that came with the archetype.

Starting the Project

A quick test that it works, as allegedly we should have a working application:

$ cd contactdb
$ mvn jetty:run

And browse to http://localhost:8080 ... presto! Well, not so "presto"... maven resolved a TON of dependencies and spent the better part of 5 minutes downloading bits (my 3 Mbps "broadband" connection is stellar).

Curiously, I note a number of links on the menu, including "Login" and "Sign up". Perhaps my story has already been written for me!

Stopping jetty, and then checking out a few new additions to my project folder:

$ git status
# On branch features/MultiUser1
# Untracked files:
# (use "git add <file>..." to include in what will be committed)
#
# lift_proto.db.h2.db
# lift_proto.db.trace.db
# target/

So, "target" is the compiled code and it seems strange to track anything there. Furthermore, I don't think I care to keep these .db files in my repository (they are presumably the underlying database...I'll figure out how to get that onto a MySQL server later).

A .gitignore file will come in handy:

$ cat <<EOF > .gitignore

/target/

/*.db

EOF

$ git add .gitignore

$ git commit -m 'ignore target and database files'

Well, that's me done then for the night. The next task will be to work out how, exactly, test-driven development works in Lift/Scala. No friends, we do not write the application first...we write our tests first, and then we fix our broken tests.

Getting Started with Scala / Lift

My first User Story involves allowing multiple users to log into the application at the same time.

The Environment

But, to accomplish this...I first need a working development environment! I'm not going to use an IDE: this will force me to do things the "hard" way which means I'll probably learn more. Plus...there was no IDE in the early days of JDK 1.0.2, so we'll keep it old school.

Well, almost old school. First things first, I'll need a version control system. I've recently picked up on git, having been a user of various tools like Subversion, VCSS, RCS, and PVCS. I like git a lot: branching for individual features is virtually "free" and it's easy to isolate commits to a subset of team members.

Next, a build tool...well...where even to begin. I've used make, ant, maven, and gradle. Were I honest with you, I'd say that I've been woefully unimpressed with the state of the art of build tools. Maybe this is just because the underlying software in Java has become so fragmented as to make dependency mapping an impossible task, I don't know. What I do know is that the Lift/Scala community seem to be using maven...so since I have no strong opinion either way, maven it will be.

So:

$ uname -a
Darwin mieslep-personal-mbp15.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 i386

$ java -version
java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07-334-10M3326)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02-334, mixed mode)

$ mvn --version
Apache Maven 3.0.2 (r1056850; 2011-01-09 00:58:10+0000)
Java version: 1.6.0_24, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.6.8", arch: "x86_64", family: "mac"

$ git --version
git version 1.7.5.4

$ scala -version
Scala code runner version 2.9.1.final -- Copyright 2002-2011, LAMP/EPFL

There you go: I'm currently on OS X 10.6 "Snow Leopard" and have what I assume to be a recent-ish set of utilities.

"Hello, World" ala Scala

Using the example at scala-lang.org:

$ cat <<EOF > HelloWorld.scala
> object HelloWorld {
> def main(args: Array[String]) {
> println("Hello, world!")
> }
> }
> EOF
$ scalac HelloWorld.scala
$ scala HelloWorld
Hello, world!

Things are working Scala-wise anyhow.

"Hello, World" ala Lift

Using the example at liftweb.net:

mvn archetype:generate -U \
  -DarchetypeGroupId=net.liftweb \
  -DarchetypeArtifactId=lift-archetype-blank \
  -DarchetypeVersion=2.0 \
  -DarchetypeRepository=http://scala-tools.org/repo-releases \
  -DgroupId=demo.helloworld \
  -DartifactId=helloworld \
  -Dversion=1.0-SNAPSHOT

(taking the defaults when prompted)

cd helloworld
mvn jetty:run

And then, http://localhost:8080 returns the nice and basic example as given. It's just that easy! Wow, I'll be done in no time! Though...one might suspect that there's a lot going on under the covers that I'll soon be pulling back :)

Sunday, 11 September 2011

My Customer

My "customer" for this project is a non-profit who really, really needs a new database application. Their current application is a single-user custom database built in PowerBuilder. What's more, the developer of this application had no real idea about usability and interface design.

I can't really single him out for blame though, without also "blaming" the customer for not really being able to articulate in technical terms what they wanted when they were first working to build this application. It's not their fault for not being able to speak to technology, any more than it isn't this developer's fault for not having the benefit of the intervening 15 years of interface design.

So, I get the benefit of standing on a whole bunch of people's shoulders to build what I would expect is going to be a very basic interface to a very basic contact database. It's very easy to throw stones, but quite another thing entirely to build a glass house.

I could probably cheat this by falling back to technologies with which I am familiar: I am sure, for example, I could knock out this application in a couple of days using Oracle APEX. I know Oracle databases like the back of my hand, and while APEX gives a very crude-looking result that's all the more I am after.

However, I want to learn a new language and framework, and being well-impressed with what I see in Scala I'm going to give it a lash. While I've done some functional programming in the past I am far more familiar with the imperative style. So, apologies in advance to those who might be looking to this as a source of "how do I get started programming in Scala/Lift". I am unlikely, at first attempt, to devise some brilliant coding patterns.

So, where to begin. I'll start with a summary of my observations after spending much of last Monday in their offices reviewing their IT systems as a whole. This, their sole contact database, came up repeatedly. As I asked how they do various things involving the database, it became quite clear that the application is very difficult to use.

For example, to generate a mail shot there are about 40 different clicks involving three different applications (this database, Excel, and Word). To log a call, there are 15 different clicks involved, including logging in and out of the database.

This last point is probably the biggest value add: only one user can be logged into the database at a time. This means that only one person can log a call, or query the database, or do anything in the database at once. There's not that many staff (maybe a dozen) but when the majority of your staff are interacting with customers and trying to log those interactions into the database ... well ... the database becomes a problem.

So, there is my first story:

As a user, I want to be able to use the database at the same time as other users.

Of course, I don't even have an application, let alone a database or users. So...I'll break this story down into the following tasks:

Create a login page
Create a "add user" request function
Allow users to edit their details, including password, and log in/out of the system

The acceptance of this story will be:

Log into the application as "user 1"
On a different machine, log into the application as "user 2"
"User 1" should still be logged into the application

The completion of this story will, of course, expose a whole new problem...namely that there is no application (or "database" as they call it) to provide any real function. But, we'll have solved the biggest pain point: multiple users will be able to log into the application simultaneously.

The Setup: Waterfall vs Agile

The Setup

Consider the following scenario:

A not-for-profit company wants to keep track of their contacts in a centralized location. They want to keep a record of donations from each contact, as well as a history of calls from each contact. They can not afford to purchase specialized software but have a consultant who has volunteered his services.

Within the scenario is the nugget of an idea (a “contact database”) and a vague notion of the requirements (“record donations and calls”). There are also presumably many unstated requirements (e.g. viewing this data in reports). How can they get from this abstract concept to a working and tested application? Broadly, there are two mainstream methodologies of software development: Waterfall and Agile.

Waterfall

In the Waterfall methodology, the Consultant would sit down with representatives from the company (the “Users”) and build out a complete requirements matrix, perhaps even including User Interface wire diagrams and workflows. He would then proceed to plan out the complete development and testing effort, estimating various work units and arriving at a reasonable delivery date.

The problems with this methodology are numerous and documented. There are two fundamental flaws: 1) it pre-supposes that Users actually know everything they might possibly want up-front, and 2) it pre-supposes that the Consultant has an accurate estimating model. Neither of these conditions is likely to be true: Users will always have new ideas (it is hard to staunch the flow of creative juices once they get flowing), and Consultants will always mis-estimate (either over or under) the effort required to deliver.

Even if a Consultant had an exact estimating model, User Change Requests would quickly destroy any semblance of a plan. Further, if one were to assume a perfect estimate and zero tolerance for Change Requests, once the application were delivered and in use there would undoubtedly be a backlog of desired enhancements and changes to the application, many of which might be very desirable/valuable.

Agile

The Agile methodology works differently; it is a Lean methodology applied to software. In Lean, the expenditure of effort on anything other than adding value is wasted effort and should be stopped as quickly as possible.

Rather than pretend that Users and Consultants are perfect, it embraces the fact that they are not. Instead, Agile focuses on the delivery of maximum value as quickly as possible. Furthermore, what a user might perceive as “high value” one day could change the next morning when they’re taking a shower and discovered a completely new way of working.
The problem with Agile is that deadlines lose meaning: when does “done” ever happen? Large enterprises in particular struggle with this, because there are many moving parts: System A has new functionality that requires System B to be updated, but if both these changes aren’t in place and working by Date X, N millions of Dollars/Euros/Pounds/Seashells will be LOST!

Agile has many problems, they are just not as well-known because it is the "flavor of the day" and, quite frankly, isn't as mature as Waterfall. There is certainly ample documentation demonstrating that stand-alone applications are developed more efficiently using Agile than using Waterfall, but the enterprise world is littered with the abandoned corpses of "new development methodologies" gone wrong.

Why Stick With Waterfall?
For lack of a proven-to-be-better delivery model, enterprise shops end up choosing Waterfall over Agile: Project and Program Managers build in contingency to increase the probability of deadlines being met, giving the executive team some comfort in the knowledge that “we have a deadline, milestones, and estimates, so we will be able to keep our millions Seashells.”

However, this is something of a fallacy: contingency is almost always eaten. “Work expands to the time allotted for the work to be completed.” It is why hour-long meetings always take an hour, even if the meeting topics could be completed in 20 minutes.

Contingency is waste: it is a block of time allocated not for the purpose of creating value but rather to buffer deadlines. (In Lean manufacturing, buffers are waste.) Furthermore, because Waterfall spends so much time up-front focusing on capturing a "complete" set of requirements, it "back-loads" the creation of actual value. More times than not, the contingency gets eaten before the value is delivered ... leading to a considered abbreviation of testing (there's a deadline to meet, after all). Finally, "newly discovered" value opportunities in the middle of a Waterfall project are either highly disruptive ("Change Requests") or deferred.

The Premise

If there is one thing that I've learned in 16 years of writing enterprise software, it is this: Waterfall wastes lots of money and rare is the project that delivers exactly the list of requirements it set out to at the start. I've also yet to see a working alternative to Waterfall in an enterprise shop.

So, I intend to undertake a journey of sorts: I am the volunteer consultant in The Setup.

A partial list of my qualifications are this:

I have been writing software in various forms for more than 25 years, starting with BASIC on a Commodore 64, dabbling with 8088 assembler, Turbo Pascal 1.0, working up to the present day where I am quite accomplished in Oracle PL/SQL.
That said, I am not a "programmer"in the sense that I a) took a single Computer Science class in college, b) spend a large portion of my working day thinking about architecture, design, requirements, and how to deliver working enterprise software. I do write programs, but really it is of a nature of building database-oriented frameworks that allow our Java developers to do their job more effectively.
I am not an expert in Agile. My current employer (Quantum Retail) uses Agile (or a a bastardized version thereof...we work with retailers who would much rather we behave in a Waterfall manner). I've been a victim...er...developer of an enterprise XP (eXtreme Programming) regime. BUT, I most definitely am a student/fan of Lean.
I know Java reasonably well...I started out on v1.0.2 (the first version) and have dabbled with it ever since. However, were you to ask any of my colleagues about my style of programming, they might describe it as "arcane."
Spring? Hibernate? AJAX? Heard of them, dabbled with them, would do very well if I could build something with these technologies from scratch. I understand the problem these are trying to solve, though I would suggest that they sometimes create too much complexity in their implementations while trying to cater for every possibility in the quest to simplify development.
.NET? I'd probably be a fan if I worked with it more. My main qualm with the world of Java is that there are too many splinters; Microsoft provides a very solid, integrated, and complete development platform. If you're happy to drink their flavor of kool-aid you don't need to look elsewhere.
User Interface design and development? Apart from "Hello, GUI World" and a few other very simple learning exercises, I know very little about how to do this.

So, my language of choice? I've been reading about Scala and a related development framework called Lift. After looking into Ruby on Rails and reminded myself of the loose-type nightmare that is Javascript, I've decided that Rails is not the framework for me. My instinct is that type specificity is easier to test.

What do I hope to learn from this? Well, at this point I'd like to learn Scala and Lift. I'd also like to explore Agile development from a Consultant/Developer perspective (at the coalface, as it were), without all of the bother and pressure that comes from having a customer who is actually going to pay you money for a job.