When Maven meets Rational Team Concert…

Regularly, the Jumpstart team members are invited to assist customers who either decided to adopt Rational Team Concert (RTC) and need to adapt their Maven build and deployment process to RTC or, as an “open source shop” wants to challenge RTC on this subject.

So, I would like to share with you some recommendations which helped our customers  in dealing with Maven when it meets RTC

Maven-RTC integration: first impression

Generally, the customer first impression, when we show the Maven/RTC integration, is pretty good.

Actually, RTC-Maven integration is straight forward (cf: Jazz-Wiki: A Jazz-based Maven build); once the user has chosen to create a Build Definition using the Maven Build template, he needs to specify:

  1. in which folder is stored the pom.xml (Project location) and
  2. which Maven goals to reach during the build. That’s it!

Unfortunately, this is not always so easy… Each customer I have met has had some particularities that I would like to share in his post.

Multimodule Projects

Don’t tell to my Eclipse buddies: I truly believe that the main integration issue we have to face when we import a Maven project into RTC has to do with some Eclipse constraints/limitations; the issues with RTC happen latter, once this one is fixed…

Actually, most of my Maven customers are using Multimodule Projects. This aggregation of projects is generally defined as a hierarchy of folders and sub-folders. So when you have to import such structure into an Eclipse client, because Eclipse doesn’t support the notion of project/sub-projects, you have to create dedicated Eclipse projects to flatten the hierarchy using, for example, the Eclipse notion of linked projects. The Q4e Maven client is using this technique to import Maven projects into Eclipse.
I didn’t have the opportunity to investigate M2Eclipse which seems to be the way the Maven+Eclipse community (and Sonatype) are going. It might worse to study their own approach on importing Maven projects.

Unfortunately, this way of dealing with a Maven Multimodule project has several inconveniences. The main one is on the fact that you have to check-in the whole project: root project and sub-modules. You cannot split the project into smaller pieces and, for example, use the Jazz SCM Component artifacts to manage and version your Maven when it would be so useful.

The recommendation we provide on such situations is to flatten the Maven structure. This transformation requires small changes in the pom.xml files and it really simplifies the end-user life when he has to import and manage such structure in RTC.

Pulling or Pushing, this is a question

The RTC Build Component promotes the Source Code Pushing model. By “Pushing model” I mean that the Build Engine pushes the code to Build Machine before running the build script. With Maven, because you can include some dedicated goals to Pull the Source Code(Maven SCM API), it is required that the build script pulls the code into the Build Machine.

I will not argue which model is better (Pushing or Pulling); it could be a very interesting subject for another post. I will just recommend to use the first one -Pushing model- with RTC because it is perfectly integrated to RTC and the build master gets for free lots of interesting behaviors, that a Pulling model will have to handle by itself, like: Snapshot creation, list of delivered change sets, list of delivered Work Items, …

Actually, the easy part of this issue is to remove from the pom.xml files the code calling the Maven SCM APIs.

Sometimes, the complex part is to convince the customer that the Pushing model is another good way of building. Actually, once you have shown him how easy it is to use Personal Builds on such context, then the customer understands, appreciates and starts to be addicted to this model…

The Jazz project is presently developing the Maven SCM plugin for Jazz SCM to support faster migration. You can follow up this enhancement  here: Maven SCM plug-in extension for RTC (Enhancement 90042). An initial implementation of a Maven SCM plugin for Jazz SCM is available in RTC 3.0 M7a (which also works with RTC 2.x). Information on using this plugin is available in the UsingMavenScmPlugin topic.

Build Toolkits API

The third point we have to solve when we want to integrate a Maven project into RTC is the insertion of the Build Toolkits API into a pom.xml file.

The Build Toolkit API are a set a ANT tasks used by a build script to communicate with the Jazz Team Server: publishing build results and contributions, enabling progress monitoring, working with Jazz source control and controlling the build life cycle.

If you want to make your build script “Jazz aware” you will need to (1) import the maven-antrun-plugin used to call ANT scripts from a Maven script and (2) insert Build Toolkit ANT calls appropriately in the pom.xml files. The following wiki page provides a good example to help you setup the Maven ANT plug-in: Invoking Jazz build toolkit Ant tasks in a Maven build.

Maven repository

On this section, I would like to talk about coupling a Maven repository with your Jazz team Server. A Maven repository is used to hold build artifacts and dependencies of varying types. It is a sort of Asset Manager. Most of the time my customers have built all their deployment process based on the assets stored in this repository.

The question which often shows up when a customer is using its own Maven repository: where should we store our built deliverable knowing we can also store them in the Jazz Team Server?

Usually, I recommend that customers store in the Maven repository any deliverable you want to be able to consume later on as a prerequisite of another pom.xml script. Now, in the context of continuous testing and continuous integration process you might have to reconsider this point. Actually, continuous testing and continuous integration are heavy deliverable generators. So, it might make sense to split the build in 2 phases: (1) Maven build, (2) Maven repository deployment.

In any cases, (1) it is really important to identify which deliverable should really be stored in the Maven repo and which one could be recreated each time you need to run a build and (2) don’t forget to publish in the build result the link to retrieve the deliverable you have stored into the Maven repository (Build Toolkit <artifactLinkPublisher> ANT task).

This last point concludes my post.

The following FAQ is an excellent place to retrieve all the entry points on Jazz.net covering this subject: Does Jazz Team Build support Maven?

Feel free to comment and share  your own feedback on this particular subject…

-Philippe

Advertisements

9 thoughts on “When Maven meets Rational Team Concert…

  1. Thanks for the write up – there are a lot of good tips here.

    One I don’t like is the suggestion on how to handle multi-module maven projects. Is changing the project structure really the most efficient way to solve this problem?

    I don’t see why your choice of SCM software should affect your project layout. In particular since Subversion, Git, and Mercurial handle any layout you want. Also, at least the first two have eclipse plugins which coexist with the m2eclipse plugin.

    • Thanks Mike for your feedback.
      Neither am I like this suggestion that I consider more like a workaround.

      The eclipse plugin for Maven I have tested required to use a dedicated loader to install a Maven project. One of the roles of this loader was to identify a tree structure and flat it creating on the fly new eclipse projects pointing to the real sub-folders.

      In RTC you cannot specify that you want to use a specific loader for a particular file system folder when you load a folder from a repository workspace to your eclipse workspace.

      I didn’t have time to push my investigations farther.So, if you have any feedback on another one which could avoid such workaround, please don’t hesitate to share the info on this thread…

      thanks again.

      • I don’t think that it uses a custom loader. It checks out the code from the SCM once, from the root pom onwards (downwards, really) and then creates linked projects to the nested projects. Eclipse has supported this sinsce 3.4 (RAD/RSA 7.5) onwards.

        I do not recommend flattening the project structures. That breaks the maven release plugin (which currently does not work with Jazz SCM anyway, but I’m working on it).

  2. I really don’t think that the comments about CT and CI are correct.

    RTC/RAM are not capable or optimised for being used as a Maven Repository Manager; that job is better left to specific tool for the job, eg Apache Archiva.

    And we need to be clear as to how these work, and *which* repository we’re talking about.

    Maven has it’s own local repository (typically ~/.m2/repository) where it saves all of the artifacts that it needs to run (jars, poms etc). It is also used to store the output of a build when the install goal is used (ie mvn install).

    Then we have remote repositories. This is where Maven Repository Managers come in. We have release repositories and snapshot repositories, as defined in the distribution management section of a pom.xml file.

    When we perform a ‘mvn deploy’ if it’s a snapshot build, it gets installed/uploaded/copied into the snapshot repository. If it’s a release build (generally, run under the maven release plugin, but not always) then it gets installed/uploaded/copied into the release repository. Note: as the deploy goal also executes the install goal, the build artefact will also be installed into the local repository.

    Continuous Integration typically monitors a trunk/branch (in svn terms) and continuously builds a snapshot, not a release.

    Snapshot artefacts should not be placed in a formal repository system as they are not baselined/labelled/taged – that is what a release is for.

    So, using CI is not a problem with maven or snapshots, as the same artefact is constantly rebuilt/replaced in the remote repos.

    Personally, on a CI build server, I only ever use ”mvn clean install site’, though I have seen some use ‘mvn clean deploy site’.

    In my current project, I use RAM to store the output of a release build only. I have tied the antrun plugin to the deploy phase, and use the RAM ant tasks to upload the asset into the RAM server for permanent storage, approavals and other workflows, etc.

    I would not use RTC to store the built artefacts, ever. You shouldn’t check in built binaries, which, from what I’ve understood above, is what it being suggested, it’s a bad SCM practice.

    -Chris

    • Thanks a lot Chris for your very interesting inputs…

      The RTC Build component provides a Build result artifact which is NOT immutable contrarily to the RTC SCM artifacts like a component baseline.

      I understand and agree that you should not store any built binary in your SCM because you can blow out SCM.

      The Build component in RTC doesn’t behave this way:
      – Your build script can attach to the Build Result any file created during the build without using any external asset manager.
      – You can also specify in the Build Definition the amount of Build Results you want to prune by indicating the successful build results to keep and the failed build results to keep.
      – Obviously, you can also indicate the specific build results you want to keep. For example the Built Result that you use as your reference for a release.

      In this case, only these Build results and the files attached to these build results will be stored and kept in RTC…

      So I agree, it might not worth a good asset manager. Nevertheless it is a pretty good start for a medium size development team.

      • Ah. You’re talking more about the build engine (jazz is such a large beast! – I still have much to learn).

        I was more approaching the issue as using maven as the build tool, and only really looking at Jazz/RTC as the SCM provider.

        In that way, I’m not really using the build engine, as the builds would be occuring locally, outside of the RTC build engine (eg maven from the command line).

        As a build/release engineeer/dev/scm/whatever else is needed, I really prefer the lifecycle advantages that maven provides over ant. As a build and release engineer, I live, eat and breathe by the maven release plugin. Which is why I am more interested in the SCM interactions from the maven echo system.

        Thanks for the response,

        -Chris

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s