Recent News

OOPSLA 2004 Workshop: Component and Middleware Performance.

Deadline September 2, 2004 - Conference October 24-28, 2004, Vancouver, canada

The goal of the workshop is to help advance the current practice of gathering performance characteristics of component systems and middleware implementations through benchmarking. Positions are solicited especially from people with previous or impending benchmarking experience. More information.

July 16th 2004: 2nd ObjectWeb Benchmarking Workshop

A workshop on benchmarking was held in Prague, Czech Republic on july 7th 2004.
The slides of the presentations given during the workshop are available.

June 15th 2004: RUBBoS Servlets v1.2 now available.

Version 1.2 of the Java Servlets implementation of RUBBoS, the bulletin board benchmark, is now available on the JMOB website. Thanks to Niraj Tolia this version fixes the folowing bugs:

Fixed connection pooling
Use of the executeUpdate() method instead of executeQuery() to execute updates
Fixed bug in ModerateComments that tried to load servlet/classname instead of rubbos/servlet/classname
Added two indices missing from the comments and old_comments table
Fixed a couple of grammatical errors in the client

Download RUBBoS Servlets version.

April 14th 2004: Evaluation of JGroups performance and scalability.

An evaluation of JGroups performance and scalability in the context of clustered J2EE application servers is available. JGroups configuration used by popular software such as the Tomcat JSP server or JBoss J2EE server are evaluated with different network technologies, protocol stacks and cluster sizes. Have a look at the results.

JMOB: Java Middleware Open Benchmarking

The main objectives of JMOB are:

to make available open-source middleware benchmarks such as RUBiS,
to share benchmarks and results on ObjectWeb middleware, and in particular on application servers (in a first step on JOnAS), including clustered configurations;
to exchange experience on client load generation, and to contribute a distributed client load generator,
to investigate impact of hardware configurations on performances, and in particular of new architectures (in a first step IA64).

Discussions around the JMOB project are supported by the JMOB mailing list.

Hosted Projects

RUBiS (Rice University Bidding System) is an auction site prototype modeled after eBay.com. RUBiS is currently used to evaluate design patterns, application servers and communication layers scalability.

Reports and articles about RUBiS are available on the RUBiS Web site Visit also the RUBiS's ObjectWeb Forge Web site for the latest code release.
CLIF (CLIF is a Load Injection Framework) is a starting development activity aiming at providing a generic load injection platform for benchmarking middleware.

More information can be found on the CLIF Web site.
Stock-Online is the CSIRO application used to stress middleware performance and scalability. It is a simulation of a simple on-line stockbroking system. It enables subscribers to buy and sell stock, inquire about the up-to-date prices of particular stocks, and get a holding statement detailing the stocks they currently own. It was designed to be implemented in multiple technologies to provide a rapid means for evaluating and comparing different distributed infrastructures and products. The workload is intended to be representative of a "typical" web based application (80% read transactions), and stress the middle-tier rather than the database.

The open-source version is implemented according to the Enterprise Java Beans (EJB) 1.1 specification, and has been used to evaluate in excess of six J2EE 1.2 Application Servers as part of the CSIRO Middleware Technology Evaluation (MTE) project. It uses a 3-tier architecture consisting of a Java (or J2EE) client, EJB middle-tier, and Oracle database. Two different EJB design approaches are provided: Stateless Session Beans with JDBC, and Stateless Session Beans with Container Managed Persistence (CMP) Entity Beans.

Visit the Stock-Online's ObjectWeb Forge Web site to checkout the source code.
RUBBoS is a bulletin board benchmark modeled after an online news forum. As RUBiS, RUBBoS was designed to to evaluate design patterns, application servers and communication layers scalability. We provide a PHP version of RUBBoS.

More information can be found on the RUBBoS Web page.
TPC-W is a standard benchmark that models an online bookstore. We just provide a Java servlets implementation based on the version from the University of Wisconsin that works with MySQL as database.

Download this implementation here.

Benchmark Results

JVM benchmarks: IBM, BEA and Sun JVM are evaluated using various benchmarks such as Volano, RUBiS or SPECjbb2000. Available results.
CORBA benchmarks: benchmarking suites for CORBA and results are available from Charles University Distributed System Group's web page.