28 Apr 2009

We've just finished upgrading our sites to Grails 1.1 and I thought it would be worth sharing some of the fun we had with the process.

First and foremost was a call site caching bug in Groovy 1.6.0 that caused chaos with our unit and integration tests. Basically any time we mocked a class then demanded behaviour on a static method or any time we used the metaClass to override a static method (among other things that's pretty much any unit test that goes anywhere near a domain class) the behaviour wouldn't get torn down again. This caused symptoms such as stubbed dynamic finders leaking from unit tests and breaking integration tests that would pass when re-run in isolation. We got as far as refactoring a whole bunch of test code that mocked static methods before Burt Beckwith pointed out to me that simply replacing $GRAILS_HOME/lib/groovy-all-1.6.0.jar with the newer version from the Groovy 1.6.1 or 1.6.2 release would fix the problem.

Unfortunately the same bug has another effect that was quite catastrophic for us. Any Hibernate proxy of an instance of a domain class involved in an inheritance hierarchy will throw ClassCastException when you try to access a subclass property or do an instanceof check. I blogged already about the instanceof issue but it's actually wider ranging than I realised at the time. It can cause very unpredictable behaviour because complex relationships between objects and the way they are loaded by a controller before a page is rendered can cause this error to pop up under obscure conditions that are very hard to nail down with test coverage. To solve the problem we've had to use eager fetching in places where associations are typed as the base class of an inheritance hierarchy and to use explicit checks for proxies such as:

if (o instanceof HibernateProxy) {
o = GrailsHibernateUtil.unwrapProxy(o)
}

Hopefully this bug will be fixed in Grails 1.1.1 and we will be able to use the default lazy loading behaviour in all relationships.

Other problems we encountered included:

  • The onLoad Hibernate event handler on domain classes now seems to run at a slightly different time in the object life cycle: before any eager fetched collections are loaded. We had one domain class that was doing some initialisation (calculating the percentage of votes each answer to a poll had received and storing it in a transient property) that no longer worked as the persistent collection it was trying to iterate over was always empty. This was simple to refactor out and in fairness was pretty horrible anyway.

  • Config.groovy and related external config files are no longer loaded when running unit tests so any unit test that excercises code doing ConfigurationHolder.config.some.value.or.other blew up. We solved this by simply adding the following to _Events.groovy to load up config prior to the unit tests running:
    eventTestPhaseStart = { phase ->
    if (phase == 'unit') {
    ConfigSlurper slurper = new ConfigSlurper(GrailsUtil.environment)
    def configClass = getClass().classLoader.loadClass("Config")
    def config = slurper.parse(configClass)
    ConfigurationHelper.initConfig config // this step loads the external environment config file(s)
    ConfigurationHolder.config = config
    }
    }

  • Logger configuration has completely changed (for the better) in Grails 1.1

  • Some of our tests were explicitly setting the level of particular loggers to 'OFF' because the test is deliberately causing an error condition and we didn't want the console polluted with a huge stack trace. This no longer works as the logging abstraction has changed to SLF4J which does not allow you to directly set the level on its Logger class. Having better things to do than unwrapping logging abstractions I just turned logging off altogether in the test Grails environment.
    log4j = {
    root {
    off()
    }
    }

  • It looks like binding errors don't get reported on nullable properties when using DomainClass.properties = params in a controller. I've raised a bug.

  • There's been a minor change to the <g:select> tag. Previously the disabled attribute followed the HTML convention (disabled="disabled") but now the attribute value needs to be a boolean. I guess this actually makes more sense than the goofy HTML convention but I really expect most of the attributes on those sorts of tags to be pure pass-through.

  • The mockDomain method used by unit tests seems to be a little inconsistent when dealing with inheritance heirarchies. If you do mockDomain(BaseClass) then try to use subClassInstance.subClassProperty it fails with a MissingPropertyException. I've raised a bug.

  • We had customised Package.groovy to deploy our app at the root context. It seems this is now directly supported as an option in Grails 1.1 by setting grails.app.context = '/' in Config.groovy. I think this option was already available in Grails 1.0.3 but our patch pre-dated us upgrading to 1.0.3. One of the things I was trying to achieve while upgrading was to ensure we were patching Grails and any plugins as little as possible.

  • I'd upgraded our selenium plugin (not the one in the Grails plugin repository) a while back to cope with Grails 1.1 and upgraded the selenium server it contained at the same time. It turns out the new selenium server will time out attempting to click any anchor that uses the javascript: protocol in its href. Luckily this was only being done in 2 places in our app and was easily fixed by using href="#" instead.

One other thing we decided to do was to replace our existing suite of webtests with functional tests. Webtest has never been popular with the developers on our team and we had made a number of modifications to version 0.4 of the plugin in order to support issues like config loading and running the app at the context root of the server. These modifications caused some headaches when trying to upgrade the plugin so we decided to drop it and port the tests over to the functional test plugin. Generally I'd still rather be writing Selenium tests than using either webtest or functional tests but the functional test syntax is a little nicer and less "pseudo-XML" than webtest's.

5 comments:

Post a Comment
  1. Whaou, that is NOT what you would call a trivial upgrade :/ Thanks for all the explanations, it may help.

  2. It is things like that which make me think about leaving my current project the way it is (1.0.3) and migrate it to a regular Spring MVC, Hibernate, Maven, GWT app some time in the future. I am so sick of this tiny little issues ALL OVER THE PLACE.

    I just dont want to deal with this buggy stuff anymore. Using Grails feels like walking on eggs, you have to be careful all the time (did I use x uncorrectly or is it a bug? did others encounter the same issue -> check mailing list? Is there a bug already-> check JIRA) all the time for stuff that should just work.

  3. All I can say is sorry for all the difficulties you had in upgrading. Grails 1.1.1 (out next week) addresses most of the hassles you have had including those related to proxies, the static method Groovy bug, and the unit testing hierarchy issues.

  4. Well, it has been difficult but in my perverse way I actually enjoyed it and learned a lot! I think the upgrade will prove to be worth the effort. For a start we're already seeing an improvement in page response time, which I'm guessing is due to the enhancements to GSP rendering. There is also a small mountain of tech debt around things like form binding, failed save handling, locking, enum persistence, etc. that Grails 1.1 gives us the tools to tackle. I'm looking forward to getting stuck into some of that stuff now that the app is up and running on 1.1.

    We'll upgrade to 1.1.1 as soon as it's out barring any blocking issues. I expect that will be a vastly simpler process.

    I can sympathise with Daniel K's comments but I'd honestly still rather be using Grails than any other platform. As I see it we've been bitten by; 1) adopting Grails early hence having code written against Grails 0.6 at the core of our app, 2) having a massive application so the sheer weight of code is a problem unto itself and 3) by the fact we have an inheritance hierarchy at the core of our domain which unluckily was the thing that caused us the most pain.

  5. Oh boy here we go again. I just tried Grails 1.1.1 - 110/531 unit tests fail. That's just unit tests, I haven't even tried running integration tests.