Earlier this week, Jetbrains launched version 6.0 of their IntelliJ IDEA Java development tool. One new feature I'm particularly excited about is the ability to chat and collaborate with other developers from inside the IDE.  See the IDETalk section of their collaboration feature description -- the feature allows you to see the presence of co-workers, send code-pointers, and view one another's source files. It's all built on XMPP using our Smack library.

I'm happy to announce that we've partnered with Jetbrains to launch a free hosted service that allows IntelliJ users to take advantage of the feature: intellijoin.org. Just choose the "register new Jabber account" option inside IntelliJ and select intellijoin.org as the server (the default). You'll get a new account on the Intellijoin Wildfire server which will allow you to collaborate with anyone on the federated XMPP network (including Gmail contacts). Alternatively, you can install Wildfire for your own organization, which will also work great with IntelliJ.

This IntelliJ IDEA release is a good example of "contextual collaboration" (adding collaboration directly into an application where it makes sense). The IDETalk feature is already pretty useful and I'm excited to see how it will evolve. I expect to see many other business applications adopt contextual real-time collaboration over the next couple of years. I can only hope that others will be as visionary as Jetbrains in using an open standard (XMPP) to make it happen. The advantages to doing so are pretty clear: faster time to market with great tools like Smack and Wildfire, a huge existing install base, and interoperability with other applications that use the open standard.

Sometimes it can feel tough to get any actual work done at the office in between meetings, the deluge of email, and people interrupting to ask questions. The work done by software engineers requires concentration over long periods of time. Any engineer knows what it's like to be "in the zone" -- where your surroundings melt away and you're able to crank on code highly efficiently. The bad news is that "the zone" is a delicate state of mind. A co-worker's annoying cell-phone ringtone, an Outlook email alert, or someone stopping by your desk will snap you right out of productivity. Joel (from Joel on software) estimates that it takes about 15 minutes to achieve full concentration after an interruption. Factor in a typical engineer's salary, and the dollars of lost productivity add up quickly. In fact, some estimates peg the cost of interruptions in the workplace at 588 billion dollars per year.

Tips for Surviving Interruptions

• Realize you're part of the problem. Most people say "I hate being interrupted" but will turn to their co-worker anytime they have a quick question. By being respectful of others' time and minimizing the interruptions that you initiate, you'll help foster a culture in your workplace that will allow you to work more efficiently. Other ways to be a good citizen in the office include setting your cellphone to vibrate, listening to music with headphones instead of speakers (seems obvious?), and not laughing out loud at jokes only you can see.

• Create an office environment that's quiet and that minimizes visual distractions. At Jive, we use system furniture with tall glass partitions. This keeps the office feeling light and airy, but helps cut down on noise. If your office is noisy and you can't do anything about it, try headphones.
   
  [588 billion dollars per year|http://www.flickr.com/photos/50884898@N00/248520050/|Photo Sharing]

• Use the best communication medium. Interrupting someone in person or by phone is the most intrusive. It forces the other person to focus their full attention on you. Everybody hates email and it just doesn't work for questions that need a quick answer. So, use instant messaging (IM). It's fast but also much less intrusive compared to other options. This is non-intuitive to some, who figure that instant messaging will only increase interruptions. We've found that IM may slightly increase the raw number of interruptions, but answering an IM is much better than having to answer a question in person since you can usually do so with only partial attention and at your own speed. IM's are also typically short and to the point (unlike email). IM is especially effective when combined with good presence management (more on that below).

• Optimize your time. Do you really need to attend all those meetings? A good rule of thumb -- if the meeting can be held without you, you shouldn't be there. At Jive, we try to schedule all meetings that include engineers for the mornings (close to lunch time) so that afternoons are left open for focused work. We also make meetings as short as possible and have very few regularly scheduled meetings (instead only meeting when necessary). It's also helpful to block out time to get work done. For example, schedule an hour or more in your calendar to work on a tough problem, then politely communicate to your co-workers that you're heads-down.

• Use on-line presence information. Communicating to co-workers when it's ok and not ok to be interrupted is critical. We've integrated our IM presence with our phone system using Asterisk-IM, so that we can see whenever somebody is on the phone. We've  also built a culture where people are reluctant to interrupt when they see someone's presence set to "Do Not Disturb" (DND). Although most IM clients will automatically toggle between "available" and "away" based on activity on your computer, be diligent about setting more complete presence information such as "Running errands, back at 2:30 PM".

The tips above include both technology and non-technology fixes to interruptions. Both are required to really mitigate the problem and there's no silver bullet. However, I'm a big believer that presence and instant messaging can make a big impact. What if your computer noticed you working in your IDE (programming environment) and automatically helped protect against interruptions by setting your presence and supressing non-urgent requests until you were finished? We're working on lots of interesting ideas like these.

How did I manage to actually get this blog post written? Headphones and the DND presence setting in Spark, of course!

This is my first week back in the office after attending my first Burning Man Festival.  My wife and I found it to be a powerful and inspiring experience. It's jaw-dropping to witness the amount of boundless engineering and creativity that people pour massive amounts of energy, time and money bringing to life for a week's time. The desert rockets from barren to the third most populous Nevada city, instantaneously. This year, 40,000 people came with no money or agenda to gather in the ancient lake bed.

It was interesting that when stripped of routine daily responsibility, how people assumed natural roles, like leaders, followers, chefs, or handimen. Even more impactful was how perfectly everything worked with 40,000 people self-managing themselves. Sorta like a real-life tag clouds. The experience did reinforce for me that communities can work amazingly well when given an open, clear space, full authority to manage themselves and when they have a powerful way to reward each other.

On another note, it was surprising to me to read the article that Dave sent to me about the Google guys using Burning Man as a recruiting event. At Burning Man everyone abandons their work ego. No one wants to talk or think about their career. Everyone is equal. For example, I learned later that one guy I drank coffee with had won three Academy Awards including work on Star Wars but when we talked he just shared which art he liked so far. I think it was the massive sculpture of the man on his hands and knees with gasoline pouring from his eyes into a pool of flame. That is, if I remember correctly.

Probably the most requested feature for Wildfire and Spark is the ability to chat with users on the public proprietary IM networks: AIM, ICQ, MSN, and Yahoo (Google Talk already works great with Wildfire and Spark since it can federate through the open XMPP protocol).

I'm happy to announce that we've been working with Daniel Henninger on a new Open Source gateway plugin for Wildfire. Daniel has brought his experience working on the Python gateway components to create a very easy to use gateway system. At the moment, there's support for AIM, ICQ, MSN, and Yahoo, with IRC support coming soon. The code is still in the very early stages, but there's already some things that make these gateways different than what's been done before:

• Installation and setup is trivial since it's a Wildfire plugin. Existing gateways for XMPP servers have to be installed as external components, which means installing dependencies, config file edits, etc. Also, working as a Wildfire plugin gives the gateways internal access to the server, which allows nice features like dynamic changes to users' rosters.

• Web-based administration of the plugin allows each network to be enabled or disabled along with features and permissions, and to view and edit gateway registrations.

• Tight integration with Spark: we're building extremely easy to use Gateway support into Spark. Of course, any other client with gateway support will work as well.

The current plan is to have beta releases available in the next several weeks. We'll also provide continued updates on development progress in the forums.

[forums|http://www.flickr.com/photos/50884898@N00/203286219/] [forums|http://www.flickr.com/photos/50884898@N00/203286220/]

Recently, a debate has erupted on the Lucene (Open Source search library) mailing list about whether to make the next version of the library require Java 1.5 instead of Java 1.4. Predictably, opinions fall into two camps. I've done my best to summarize the arguments on both sides:

• Move forward: "Java 1.5 has been out for about two years and making the next version of Lucene require that release is a reasonable way to move forward. Backwards compatability can't be preserved forever, and there are some nifty features to use in the newer versions of Java."

• Backwards compatability: "There are still a ton of users on Java 1.4. Switching to 1.5 provides only incremental engineering advantages and could leave existing users in a lurch. It would be better to wait at least another year until even more users are on Java 1.5."

I've watched the passionate emails on both sides with some interest, as we went through similar discussions in the not-too-distant past. Our Jive Forums product has been locked to Java 1.4 for a long time because we have a lot of large, important customers that just can't upgrade their infrastructures very quickly. So, not much of a debate on which version of Java to use for that product, unfortunately. However, when we were going through the process of making Wildfire Open Source nearly two years ago, there was a choice -- use Java 1.5 and get all the cool stuff, or appeal to more existing users and work on more platforms by supporting 1.4? Back then, it was a pretty tough choice since Java 1.5 had just been released.

Progress at the expense of backwards compatability is obviously one of the core issues in software development, and I can't hope to fully address the problem in this blog entry. However, I can say that we were very happy that we picked 1.5 for Wildfire. Generics and features like the new concurrency package certainly helped our development process. But, there was something else about moving to the new platform that I wasn't expecting. If I had to distill it down to one word, using Java 1.5 simply felt "fresh". It got our engineers fired up and the community of users that developed around the project seemed only too happy to be using the latest and greatest. We also embraced a lot of other cutting edge technologies at the same time like XMPP. I truly believe that if we had gone down the "safer" path with 1.4, Wildfire just wouldn't have developed the same level of energy that it has today.

Maybe this is the path that all software products take as they "cross the chasm" -- start with what's new and risky (which appeals to all of us that love technology) and then gradually slow down into being more mainstream and then eventually obsolescence.  I guess we'll see what happens with Wildfire when Java 1.6 comes out later this year and we're faced with the same choice all over again. Will things be entirely different now that the Wildfire project is so much more mature?

Finally, what will the Lucene maintainers decide? We'll find out in the near future, but I cast my vote for the move to Java 1.5.

[http://www.flickr.com/photos/50884898@N00/162086950/]Today, Wildfire won a ServerWatch product award in the Real-Time Communication Server category while Jabber took top honors in the Open Source Community category; check out the article for full details. Two things about the Wildfire award get me excited.

First, it's typical for Open Source projects to do well in reader's choice awards. Large Open Source communities are willing to take the time to cast votes for their project (if they know about the contest); it's hard for proprietary vendors to inspire that same passion. But that's not what happened in this case. Although we knew Wildfire was nominated for the award, we didn't know the details of the voting process and didn't make any mobilization efforts. So, the voting came directly from ServerWatch readers. On behalf of the entire Wildfire community: thanks ServerWatch! Even better, not only did Wildfire win, but we "ran away with the real-time communication category, capturing more votes than Microsoft Live Communications Server 2005, IBM Lotus SameTime, and Antepo OPN XT combined."

Second...notice who came in second? We think Wildfire is emerging as a viable alternative to Microsoft's LCS. We're disrupting the market using Open Source, open standards, and innovative features. But while this award shows we're heading in the right direction, we have a lot of work left to do. LCS is a clear leader and most organizations choose Microsoft products because they're a  safe and familiar choice. We've only just started to get the word out about Wildfire and Spark. To the Wildfire community: thanks for your help so far and keep it up!

The Spark Skinning service has been officially launched. It's an easy way to customize the look and features of the Spark IM client (and have those customizations automatically applied upon new releases). While I hope that you'll check out and use the service, this entry is for those that are interested in the behind the scenes story.

As originally scoped, Spark Skinning was going to be a fairly simple two week project. So, how did it take us two months to launch?

1. Feature Creep. We're typically pretty good at spec'ing out and scheduling releases at Jive. We develop detailed overviews for each feature in our wiki (so that everybody can edit the documents) and create mockups of how the features will look and function. Since this was "only a two week project", we thought we could get away with a less formal process -- a couple of quick meetings around a whiteboard. You can probably guess the outcome. Because we hadn't made the right investment in planning, the first version we built was missing some basic features. For example, the service didn't have the ability to lock the client to a particular server or to specify connection proxy settings. The color schemes also only applied to the login screen instead of the whole client. Yes, sounds pretty dumb, but that's what happens without planning. Lesson learned: Even "small" projects should go through a proper planning process.

2. Platform Fun. Spark is a cross-platform client, so the build process has to create installers for Windows, Linux/Unix, and Mac OS X. It turns out that in order to build an OS X installer the way we want to, the installer has to be created on OS X (Windows and Linux/Unix installers are built using pure Java so can happen on any platform). This was our first OS X box in a production environment and we choose to use a Mac Mini (pictured in its rack below). After setting up and deploying, everything was working well except that the service crashed once every two hours. After two weeks of debugging, we finally ran across a blog entry that explained a workaround to the problem.Lesson learned: New and unfamiliar production environments will always account for unseen problems and delays.
   

3. Integration. Spark Skinning is the first product we've launched that's purchased online via a credit-card. Getting online purchases setup involved a lot of non-engineering work. We had to set up an account with an online card processing service (Verisign) and then hook that up to our merchant account. Our operations team needed the right transaction information to correlate purchases in our accounting system. We had to setup the server so that we could process the transactions securely. Etc, etc. etc. Lesson learned: Make sure that all stakeholders (from development through operations) are involved in the schedule development.

Of course, we can apply rose-colored glasses to these three problems. Ultimately, the Spark Skinning feature set is richer and more compelling than originally planned, we know a lot more about OSX server hosting, and we've done a lot of work on ecommerce that can be applied to our other products. I guess I'll take the lessons learned and be happy with the result.

In Wildfire2.6, we've introduced support for the Publish-Subscribe extension to XMPP (Jabber). A loose analogy is that it's like RSS on steroids, but for instant messaging. A slightly more technical take: it's a comprehensive system for publishing and consuming topic-based events.

Like RSS, pubsub offers a simple way to get notifications. But far beyond RSS, pubsub has rich publishing and permissions systems. As a more radical example than standard news syndication, a company could use pubsub to power a file sharing service; certain users would be allowed to publish files, while others could read them (with optional moderation). Users would be notified in real-time when files are added or modified, and could even filter notifications using keywords. Other IM twists to the pubsub protocol allow you to choose to only receive events when you're online, use your buddy list for permissions management, etc. The reason we're excited about pubsub is two-fold:

1. It's much more comprehensive than the existing mainstream event protocols like RSS and Atom, which means you can do much cooler stuff. Of course, RSS and Atom should be thought of as complementary rather than competing technologies since they're for a different medium.

2. If you believe like Jive and Google that an XMPP instant messaging client will be on every user's desktop, that means pubsub is a viable<span style="font-weight: bold"># platform</span># for building all sorts of services.

At this point, pubsub is just an interesting technology that remains to be proven. However, we'll be building some innovative services on top of it and I'm sure others will be too.

If you're interested in learning more about pubsub and how it can be applied, read our article on the topic.