Blog Posts From Jive Talks Tagged With developers

Feature M&M's

greg@jivesoftware.com — Tue, 07 Aug 2007 14:32:24 GMT

Once a month the entire company gets together (beer and snacks provided) for a brainstorming session that we call Blue Sky. This month we broke into several groups and brainstormed small but tasty features for Clearspace. Hmm... what else is small and tasty? M&M's of course. Hence, the idea of feature M&M's was born and one of them quickly made it into the product. Check it out in Clearspace 1.4 by visiting someone's profile and you'll see a list of other people who have similar profiles. Why brainstorm this topic? It's often the little features that make a product special. On the other hand, it can be pretty difficult to get enough attention for the individual features to make it into the roadmap. Each of the ideas has been written onto a piece of paper (in the appropriate shape of course) and then put into a special bowl.

Whenever an engineer has a little time they can grab an M&M (no cherry picking), then attach it to the M&M board and implement it. Here is a shot of the M&M board just after installation:

The Release Train

matt@jivesoftware.com — Wed, 01 Aug 2007 21:18:48 GMT

Back in April, I blogged about how we were adopting a release train model for our Open Source projects. Since then, we've rolled out the same process to our commercial products Clearspace and Jive Forums. The release train is a fairly fundamental departure from how we've done releases in the past, so we wanted to provide more details about exactly how it works.

Why did we make a switch in how we build our software? There were many motivating factors, but the general theme is "move as fast as possible with high quality". For end users of our products, the key thing to know is that there will be a new release every three weeks. Each version contains bug fixes and new features and we're committed to maintaining high quality for every release (no more rushed bug fixes a week after a release). The graphic below illustrates how this process works:

Each release (from top to bottom of the graphic) takes a total of nine weeks: three weeks of planning, three weeks of development, and three weeks of QA. All three processes run in parallel, which leads to the three week release cycle.

Answers to common questions:

Q: Do we expect customers to upgrade every three weeks?

A: No, that's unreasonable in most environments. We've made it as easy to do upgrades as possible, and we hope you'll upgrade at least once per quarter to take advantage of all the great changes. When you do upgrade, the release train process will help ensure you're on high quality code.

Q: How will version numbers work?

A: Each release will get a minor version number: 1.5, 1.6., 1.7, etc. Major version numbers will change approximately once per year.

Q: How will you develop major new features that take more than three weeks?

A: Good question. No model is perfect and we're already working on new features that will take more than one train cycle to fully finish. In those cases, we're breaking the projects into milestones and using code branches as necessary.

Other Release Train Fun

The release train has had a deeper cultural impact than just being a way that we engineer our software. The marketing team now times a lot of their work on the train, and even our major happy hours are now on the three week cycle. Late afternoon of every third Friday, we gather the company for a demo of the new features and then adjourn for partying.

Time will tell how well this new process works, but we're excited about it and the results so far are promising.

Wiki syntax challenges

bruce@jivesoftware.com — Fri, 19 Jan 2007 16:26:04 GMT

Out of all the things that I've had a hand working on for Clearspace (and that's a lot) the one that I've found to be the most challenging and frustrating to implement would have to be the new wiki syntax support. As many of you are aware wiki syntax is a very useful (and fast) way to markup text to create things like lists, tables, links and styles without having to type html. We looked at a variety of existing wiki markup syntaxes and decided on a syntax that was as common as possible. It's very fast to use once you've know the syntax - and potentially a deal breaker to new users who don't want or have the time to learn the syntax no matter how simple it may be. Thus in Clearspace we decided it would be best to allow documents to be created in one of two ways - using a graphical editor (IE or Firefox) or a plain text editor with preview functionality.

[existing wiki markup syntaxes|http://jivesoftware.com/blog/wp-content/uploads/2007/01/create.png]

Since users will have the ability to switch between the two editors we needed a way to convert the html content generated from the GUI editor into the wiki syntax supported by the plain text editor and vice versa. It was this requirement that proved to be the source of many frustrating hours stepping through code trying to isolate yet another bug.

Not surprisingly we couldn't find an open source wiki syntax implementation that met all of the requirements so we had to roll our own. Our existing rendering solution found in Jive Forums proved not to be up to the task so we had to design one from scratch to meet the new requirements. Before that though we had to evaluate and test a variety of html parsers to facilitate the conversion from the html syntax generated from the GUI editor back into the wiki syntax. We settled on a flexible and extremely fast parser from the open source Sitemesh project which we modified slightly to suit the requirements. Spending the time to research and choose the right tools proved to be a life saver - it allowed us to handle the "wikification" of the html in a very clean manner.

Once the tools were chosen we created the API for the new render system and started writing the implementation. As you can see from the following code the general use case is fairly straightforward for generating html from the stored wikified text:

RenderManager rManager = jiveContext.getRenderManager();
String htmlText = rManager.render(doc, RenderType.DOCUMENT_BODY, wikifiedText);

Going the other way from html to wiki text requires a bit more work but is also fairly straightforward:

RenderStrategy strategy = new RenderStrategy(
RenderTarget.TARGET_PLAIN_TEXT, RenderStrategy.RENDER_ALL);
RenderManager rManager = jiveContext.getRenderManager();
String wikifiedText = rManager.render(doc,
RenderType.DOCUMENT_BODY, strategy, htmlText);

Getting it all right though required a lot of time and a large number of test cases to work out the bugs. Without these test cases I'm sure some of the more subtle bugs that cropped up would never have been caught. I'm very proud of the outcome of all this work - it's going to be a great feature as well as the basis of quite a bit of new functionality in future releases of Clearspace.

Plugin Support, Finally

matt@jivesoftware.com — Tue, 19 Dec 2006 17:21:52 GMT

Now that we've added support for plugins to Clearspace and the next version of Jive Forums, it's hard to believe it took us so long to add such a useful and powerful feature. The basic idea behind plugins is to dynamically extend the functionality of the application without having to modify the core source code. For example, our professional services team will use plugins to build custom functionality for clients that is easily portable between different product releases. Plugins will also be used to deliver features that are only useful to a portion of our customers, like Salesforce integration.

Plugins have been very successful in Wildfire and our friends from Atlassian has tons of cool plugins in their products. We borrowed a lot of ideas from Wildfire plugins (see the developer guide) in order to create the system in Clearspace and Jive Forums. Of course there are some unique twists as well. For example, if you install a plugin on one node of a Clearspace cluster, the plugin will get automatically pushed to all the other nodes. Developers will also appreciate the fact that plugins can easily override any page or action in the application, which makes some very deep customizations quite easy. Plugins are installed and managed from the admin console:

[developer guide|http://jivesoftware.com/blog/wp-content/uploads/2006/12/pluginadmin.png|pluginadmin.png]

For the launch of Clearspace, we wanted a plugin that would be a great example of what the framework allows. So, we created the WebDAV plugin. WebDAV is a useful protocol for publishing and accessing documents. For example, you can access content through WebDAV using Windows Explorer or OS X (just like a shared drive):

[WebDAV|http://jivesoftware.com/blog/wp-content/uploads/2006/12/clearspace-web-view.png|clearspace-web-view.png] [WebDAV|http://jivesoftware.com/blog/wp-content/uploads/2006/12/clearspace-webdav-view.png|clearspace-webdav-view.png]

For now, the WebDAVplugin provides read-only access to the latest version of documents. In a future version, we'll add write support as well. I'm really looking forward to all the interesting plugins that our engineering team and community members put together!

Tips for Surviving Interruptions

matt@jivesoftware.com — Wed, 20 Sep 2006 19:25:34 GMT

Sometimes it can feel tough to get any actual work done at the office in between meetings, the deluge of email, and people interrupting to ask questions. The work done by software engineers requires concentration over long periods of time. Any engineer knows what it's like to be "in the zone" -- where your surroundings melt away and you're able to crank on code highly efficiently. The bad news is that "the zone" is a delicate state of mind. A co-worker's annoying cell-phone ringtone, an Outlook email alert, or someone stopping by your desk will snap you right out of productivity. Joel (from Joel on software) estimates that it takes about 15 minutes to achieve full concentration after an interruption. Factor in a typical engineer's salary, and the dollars of lost productivity add up quickly. In fact, some estimates peg the cost of interruptions in the workplace at 588 billion dollars per year.

Tips for Surviving Interruptions

Realize you're part of the problem. Most people say "I hate being interrupted" but will turn to their co-worker anytime they have a quick question. By being respectful of others' time and minimizing the interruptions that you initiate, you'll help foster a culture in your workplace that will allow you to work more efficiently. Other ways to be a good citizen in the office include setting your cellphone to vibrate, listening to music with headphones instead of speakers (seems obvious?), and not laughing out loud at jokes only you can see.
Create an office environment that's quiet and that minimizes visual distractions. At Jive, we use system furniture with tall glass partitions. This keeps the office feeling light and airy, but helps cut down on noise. If your office is noisy and you can't do anything about it, try headphones.

[588 billion dollars per year|http://www.flickr.com/photos/50884898@N00/248520050/|Photo Sharing]
Use the best communication medium. Interrupting someone in person or by phone is the most intrusive. It forces the other person to focus their full attention on you. Everybody hates email and it just doesn't work for questions that need a quick answer. So, use instant messaging (IM). It's fast but also much less intrusive compared to other options. This is non-intuitive to some, who figure that instant messaging will only increase interruptions. We've found that IM may slightly increase the raw number of interruptions, but answering an IM is much better than having to answer a question in person since you can usually do so with only partial attention and at your own speed. IM's are also typically short and to the point (unlike email). IM is especially effective when combined with good presence management (more on that below).
Optimize your time. Do you really need to attend all those meetings? A good rule of thumb -- if the meeting can be held without you, you shouldn't be there. At Jive, we try to schedule all meetings that include engineers for the mornings (close to lunch time) so that afternoons are left open for focused work. We also make meetings as short as possible and have very few regularly scheduled meetings (instead only meeting when necessary). It's also helpful to block out time to get work done. For example, schedule an hour or more in your calendar to work on a tough problem, then politely communicate to your co-workers that you're heads-down.
Use on-line presence information. Communicating to co-workers when it's ok and not ok to be interrupted is critical. We've integrated our IM presence with our phone system using Asterisk-IM, so that we can see whenever somebody is on the phone. We've also built a culture where people are reluctant to interrupt when they see someone's presence set to "Do Not Disturb" (DND). Although most IM clients will automatically toggle between "available" and "away" based on activity on your computer, be diligent about setting more complete presence information such as "Running errands, back at 2:30 PM".

The tips above include both technology and non-technology fixes to interruptions. Both are required to really mitigate the problem and there's no silver bullet. However, I'm a big believer that presence and instant messaging can make a big impact. What if your computer noticed you working in your IDE (programming environment) and automatically helped protect against interruptions by setting your presence and supressing non-urgent requests until you were finished? We're working on lots of interesting ideas like these.

How did I manage to actually get this blog post written? Headphones and the DND presence setting in Spark, of course!

The Playa Community

communities@jivesoftware.com — Fri, 08 Sep 2006 12:54:12 GMT

This is my first week back in the office after attending my first Burning Man Festival. My wife and I found it to be a powerful and inspiring experience. It's jaw-dropping to witness the amount of boundless engineering and creativity that people pour massive amounts of energy, time and money bringing to life for a week's time. The desert rockets from barren to the third most populous Nevada city, instantaneously. This year, 40,000 people came with no money or agenda to gather in the ancient lake bed.

It was interesting that when stripped of routine daily responsibility, how people assumed natural roles, like leaders, followers, chefs, or handimen. Even more impactful was how perfectly everything worked with 40,000 people self-managing themselves. Sorta like a real-life tag clouds. The experience did reinforce for me that communities can work amazingly well when given an open, clear space, full authority to manage themselves and when they have a powerful way to reward each other.

On another note, it was surprising to me to read the article that Dave sent to me about the Google guys using Burning Man as a recruiting event. At Burning Man everyone abandons their work ego. No one wants to talk or think about their career. Everyone is equal. For example, I learned later that one guy I drank coffee with had won three Academy Awards including work on Star Wars but when we talked he just shared which art he liked so far. I think it was the massive sculpture of the man on his hands and knees with gasoline pouring from his eyes into a pool of flame. That is, if I remember correctly.

Vanderzand It

matt@jivesoftware.com — Sun, 06 Aug 2006 22:27:00 GMT

Over the past couple months, anytime we want to make a GUI prettier and more functional, you'll hear someone say "Vanderzand it!". The phrase was coined in honor of Jive's first dedicated web designer/developer (with the unique last name of Vanderzanden). You'll find the latest Vanderzanding below -- a "before" picture of the Wildfire 3.0 setup tool and an "after" shot of the upcoming Wildfire 3.1 setup tool.

A good-looking user interface has intrinsic value. It forms the first impression of an application -- before they determine the depth of features or stability, users form an opinion based on looks. The UI also plays a long-term role since users want to keep coming back to apps that are both beautiful and functional.

Look out for a lot more Vanderzanding in Wildfire in the near future.

How We Manage Our Website

communities@jivesoftware.com — Fri, 17 Mar 2006 13:57:50 GMT

We recently relaunched our website and with that we changed how we manage content on the site. Before, the website code (Java classes, JSP pages, HTML, images, CSS, etc) was checked in to source control and usually one developer (sadly, me) would have to make all kinds of changes. It was a really inefficient process and not sustainable the bigger we grew. We needed a system where non-engineers could edit the pages, see their changes easily and deploy the changes to the site without my help.

First, we use an open source template framework called Sitemesh. This allows our guys to write very basic HTML and not worry about things like the header, footer, sidebar, etc. Sitemesh takes the output of a JSP page (or anything that serves up HTML), parses it then merges it with a template. You can have any number of templates in the system, all mapped to patterns or various other things. This is a pretty simplified description of Sitemesh -- overall, it's a very powerful and extremely simple framework. As I mentioned, it'll handle anything that produces HTML so it's compatible with PHP, Perl, ASP, etc. We use it in our products to make templating easy and to keep the JSP pages clean.

Next, I created a Subversion hook to do automatic deployments of our content when a check-in is received. (I got the idea for this from an Ars Technica article.) Subversion is our source control and a hook is a way to execute a script based on an event like a check-in, update, etc. Local to the jivesoftware.com machine is a checkout of our site code. From there, the hook script calls an Ant target to deploy the latest JSP/HTML code. I only deploy JSP or HTML in this step and not Java classes or config files. Doing updates to the core source is less frequent and ususally requires a server restart.

Finally, we have a checkout of the site project on our internal file server (Linux + Samba). I also use that checkout as the web root for an internal staging site. Since changes are picked up automatically, our guys can edit the files on the file server then reload the page immediately in their browser. After they make their changes, code is checked in and automatically deployed.