Return to Jive Software

Currently Being Moderated

More Details on the Jive 3.0 Platform

Posted by Bill Lynch on Apr 3, 2009 6:00:49 PM

We got a lot of good feedback from customers when we announced the 3.0 builds and platform changes. As a result, we've added back support for the MySQL and SQLServer databases. I'd like to take a minute to address some of the feedback and explain some of the background for this move.

 

As background, we've changed the way we package and ship our software. I just published an in-depth look at the benefits of this move - a good read if you're looking to understand more.

 

The package we've built is available on Linux and Solaris and runs on the following databases: MySQL, Postgres, Oracle, and MS SQLServer. The application itself is available as an installable package and bundles an application server, Java runtime, the app itself, and a set of optimized preconfigurations (example: tuned JVM settings).

 

Why make this move? A number of reasons. We talk to customers often and the consistent number one priorities are performance, scalability, stability, and the support experience. We get a lot of great feature and improvement ideas but those top priorities are the air customers breathe. Based on that, we try to move the needle in these areas significantly for each release. For example, for the 2.5 release we managed a 40% performance improvement overall. In 3.0 the numbers aren't quite as dramatic but we've added new features and performance has not suffered. As we looked up and down our architecture it became clear that some new thinking was needed.

 

The application has evolved past the point where it can be distributed as a standalone WAR file (however, to be clear - a WAR does still exist in the bundle). The context in which it runs is vitally important, especially for things like performance but also for features. We need the ability to depend on the environment we run in and there is a lot of variance in server-side Java deployments.

 

On the support side, we've seen too many misconfigurations in the environment. For example, sometimes we see issues where a customer is unable to change the parameters of the VM either to allocate more heap memory or to change the GC settings - these kinds of things are vitally important to the Jive app. Another big challenge is just access to the right log/debug files - none of this is consistent among appservers. Also, oftentimes the best information is a Java heap dump and we've seen the ability to get that information be a real challenge. In the new package, we've distributed a script that grabs every relevant log file and heap dump and zips it up in a way that can easily be sent to our support team. We feel this will dramatically improve the time-to-resolve issues.

 

What doesn't change? It's important to note that development, customizations, plugins, themes, etc don't change in at all. For example, how you deploy a plugin is exactly the same. Clustering also works in the same way. This is not an appliance, nor is it a black box. (Note, an appliance is something we'd consider as an option for future deployments - let us know if you have feedback on this approach).

 

We believe there are a lot of benefits to this approach. Ultimately, this foundation will allow us to develop some very unique features while delivering continued huge performance and scalability gains. We definitely recognize some customers will have problems migrating to this platform and we're more than willing to make it work for you - just let us know.

 

(One final note: within a week we will announce when the extra database support will be available.)

 

If you have questions, comment on this post or feel free to drop me a line at bill@jivesoftware.com.

3,158 Views Tags: platform, 3.0, sbs


Apr 6, 2009 8:15 AM dlevancho dlevancho    says:
The package we've built is available on Linux and Solaris and runs on the following databases: MySQL, Postgres, Oracle, and MS SQLServer. The application itself is available as an installable package and bundles an application server, Java runtime, the app itself, and a set of optimized preconfigurations (example: tuned JVM settings).

Nice, Very Nice, so you start Supporting mysql and MSsql back?,

but I dont udnerstand  if you support msSQL should not you also support win OS?

 

On the support side, we've seen too many misconfigurations in the environment. For example, sometimes we see issues where a customer is unable to change the parameters of the VM either to allocate more heap memory or to change the GC settings - these kinds of things are vitally important to the Jive app. Another big challenge is just access to the right log/debug files - none of this is consistent among appservers. Also, oftentimes the best information is a Java heap dump and we've seen the ability to get that information be a real challenge. In the new package, we've distributed a script that grabs every relevant log file and heap dump and zips it up in a way that can easily be sent to our support team. We feel this will dramatically improve the time-to-resolve issues.

this is totally wrong.

 

 

 

customer is unable to change the parameters of the VM....

Java heap dump and we've seen the ability to get that information be a real challenge....

do you really beleive that IT departments do not know how to adjust heap size?  I am 100% sure, companies that use Clearspace have smart anought IT to be able to change JVM parameters(common man, what kind of statistics are you basing this from? ). or where to locate log files? especialy if jive  (besides server logs) managers its own additinal layer of logs and logfiles.

Apr 6, 2009 8:36 AM Bill Lynch Bill Lynch    says in response to dlevancho:

Nice, Very Nice, so you start Supporting mysql and MSsql back?,

but I dont udnerstand  if you support msSQL should not you also support win OS?

 

It's a good question - they're actually different issues. It doesn't matter what OS the database runs - as long as it speaks SQL (and is one of the 4 supported), then we're good. The actual work we would do on Windows to build and create a package is different than Linux which is why it's not automatically supported when we support SQLServer.

 

this is totally wrong.

 

How so?

 

do you really beleive that IT departments do not know how to adjust heap size?  I am 100% sure, companies that use Clearspace have smart anought IT to be able to change JVM parameters(common man, what kind of statistics are you basing this from? ). or where to locate log files? especialy if jive  (besides server logs) managers its own additinal layer of logs and logfiles.

 

Not saying they don't know how to adjust GC settings. We've worked with plenty of customers who can do this but also plenty who can not because of their policies - yes, sounds crazy I know. For example, I've seen customer be installed in a big shared environment. When we're one of 8 applications the right analysis needs to be done before changing GC settings because all applications are affected. At best, it means a long time before the changes can be deployed, at worst, they can't be deployed.

 

 

 

Also - we don't believe that it should be the job of our customers to tune our application. When you're talking about GC settings, heap size, and a bunch of defaults we are the ones who know the best default settings. Can you still tweak those settings? Sure, and we make that possible in the build. However, what makes an application scale or perform in an environment is a set of things working together. For example, I'd rather provide guidance but ultimately have our customers responsible for backup, the database, the user directory, the OS, the physical hardware, the network gear, the firewall, security policies, etc. Those are the things out of our control and the *right* things for our customers to address.







Apr 6, 2009 8:46 AM Joe Cotellese Joe Cotellese    says in response to Bill Lynch:

Nice, Very Nice, so you start Supporting mysql and MSsql back?,

but I dont udnerstand  if you support msSQL should not you also support win OS?

It's a good question - they're actually different issues. It doesn't matter what OS the database runs - as long as it speaks SQL (and is one of the 4 supported), then we're good. The actual work we would do on Windows to build and create a package is different than Linux which is why it's not automatically supported when we support SQLServer.


Ah, is that a hint that MS Windows support might come back too .

 

I appreciate the fact that you've put back MSSQL and don't like to belabor the point but for some organizations linux and solaris are just not viable options. When your customers committed to your product the implied commitment back was that you would not pull the rug out from underneath them.

 

I understand you are on the front lines and are bearing the brunt of the communities frustration. Given your change in stance on db support it's clear to me that you do listen to your customers. Thanks.

 

(we will still continue to beat you up on Windows Server support though...)

Apr 6, 2009 8:57 AM Bill Lynch Bill Lynch    says in response to Joe Cotellese:

Joe - no problem, happy to get the feedback. I definitely understand that this puts some in a lurch but I believe there are huge benefits to this overall move and we'll be able to deliver some more compelling features.

Apr 7, 2009 10:14 AM tschramel tschramel    says:

I agree that the decision to continue support for MySQL and SQL Server is a step in the right direction. However, the primary concern for us continues to be the lack of support for the Windows OS. Our IT department manages hundreds of servers -- all of them either Windows or AIX. Introducing a new OS is no small jump. Were this an enterprise critical application, we would certainly find a way to make that leap. However, as a recently-introduced, non-critical application, the cost to switch vendors is considerably less than the cost to introduce a new OS.

 

Also, I have to take issue with the continuing insinuations that only non-Windows environments can offer performance, scalability and stability. I've been running very large Java-based portal and WCM systems -- much more complex than Clearspace -- on W2K3 servers with WAS for years without OS-related issues. If these were truly the issues, how could anyone choose Tomcat over the likes of WAS, JBoss or WebLogic? Let's be honest: there's obviously another agenda here.

 

Finally, I don't see limiting choice as a customer-focused move, no matter how much easier it makes support. If that were the case, Detroit would be selling only one type of car and we'd all be driving a Cobalt.

Apr 7, 2009 10:51 AM Bill Lynch Bill Lynch    says in response to tschramel:

Tony,

 

First thing's first: I want to be clear that in no way are we implying that Windows or other application servers are bad platforms - they're not. There's a lot of good stuff there and obviously many companies have built complex and scalable applications on them.

 

For the Jive application, it's evolved to a point beyond where a single web application (WAR) is appropriate. This move is less about not supporting application servers and is more about Jive needing a consistent environment and context in which it runs. Our application already does more than just serve web pages - it has a rich web services layer, a plugin framework, processes large amounts of emails, etc. In the future we'll do more with instant messaging, video processing, file storage, more rich plugin API, etc - all of which demand we expand the scope of the application. There's just too much variance in the types of environments out there for us to reasonably cover all the bases.

 

Our goal is to keep providing an amazing support experience as well as an application that's easy to install and maintain. These changes help us do that while at the same time adding a lot of top-requeseted features.

Apr 8, 2009 2:06 PM LG LG    says in response to Bill Lynch:

Hi  Bill,

 

what does "our application already does more than just serve web pages" mean for further SBS versions? One could expect that you will soon use JBoss instead of Tomcat. Will you add native C++ code or can one expect that it will remain a Java application?

 

I'd still like to see is a Java EE "war" or "ear" deployment option for those who do not want a fine tuned application which runs only on a few operating systems. So also MS-, Suse- or BSD-only companies may take a look at Jive SBS.

 

LG

 

PS: Detroit would be selling only one type of car and we'd all be driving a Cobalt and we'd need to drive on streets built and certified by RedH*t or S*n.

Apr 8, 2009 2:20 PM Bill Lynch Bill Lynch    says in response to LG:

We will not be adding any C++ code. Most likely we'll start to bundle other applications (for example - OpenFire), or other more complex integrations that demand some native components. No concrete plans at the moment but the changes for 3.0 set a foundation for that.

 

for those who do not want a fine tuned application

 

What customer would want a non-tuned application? Also - SuSE-based companies CAN run SBS (it's already supported). And, I've never seen a request or desire to run it on BSD.

Apr 8, 2009 2:54 PM dlevancho dlevancho    says in response to Bill Lynch:

I am still trying to figure out what is the real reason for all these IMHO uneccessary requirement changes,
Clearspace is a very powerfull platform, one of the kind, as far as I know, and I know dev team does great job, and it has very solid  codebase to work with, but all these new platform modifications, requirements , and limitations introduced in V 3 just dont make sence to me, this is not a microsoft office that end user just installs on his machine and thats it , most of the clients of clearspace are companies that have a very solid IT, and actualy would want to tweak instalation themselves,.I dont understand why would you not want clients to tweak cough** clearspace cough**  (SBS)  themselves?

 

 

The actual work we would do on Windows to build and create a package is different than Linux which is why it's not automatically supported when we support SQLServer.

I am, as we speak building clearspace on windows and deploying it on Linux (integration) server so,

I could just as easily deploy same build on windows . is not it a whole ideaof Java/j2ee, write once run anywhere?

if its was possible in 2.5x how come its not in 3.x .to run on Win?

 

 

 


We've worked with plenty of customers who can do this but also plenty who can not because of their policies - yes

IMHO, Companies that have that kind of policies that does not allow them to adjust jvm params or to tweak clearspace, most likey are going to have even more hard time fitting into  this tight system requirements .

 

anyways I guess it will get adjusted mor eand more as time goes by, BTW, good job on performance improvements  in SBS

Apr 8, 2009 3:21 PM Bill Lynch Bill Lynch    says in response to dlevancho:

dlevancho - couple of things. Hopefully you've read over this blog post, my comments, and the linked document as to the reasons we're doing this. Ping me over email if you'd like to jump on a phone call to talk it over more.

 

Next - if a company desires to tweak the settings of the JVM they can do so. We just think that most customers will benefit from the out-of-the-box settings and will avoid a lot of trouble down the road. Also - it's more than just the JVM that makes the app work correctly. I'd rather our customers focus on things in their control - network, physical hardware, firewalls, load balancers, web servers, database backup, CDNs, etc rather than be on the hook to tweak the specific performance settings of our application.

 

In terms of development, not much actually has changed. Yes, the target installed OSes are different but the app is still a Java application, and the customization options are mostly Java or web-based. We've kept all the same conventions that have made developers successful with the application.

Apr 15, 2009 10:55 AM corris corris    says in response to Bill Lynch:

We run it on FreeBSD 7.1 with tomcat 6 and jdk 1.6, and it works just fine....  does that mean we can't upgrade to SBS?

Apr 15, 2009 11:06 AM Bill Lynch Bill Lynch    says in response to corris:

corris - what database are you on? Likely you CAN upgrade without issue.

Apr 15, 2009 11:34 AM corris corris    says in response to Bill Lynch:

MySQL ( also on FreeBSD, but that shouldn't matter as much as the application ), moving forward, I hope that Jive considers MySQL a first-class citizen.

Apr 15, 2009 12:10 PM Bill Lynch Bill Lynch    says in response to corris:

Yep, MySQL is supported (we're going to announce exact dates shortly). We definitely consider it a first-class citizen.