Performance Regression Test for Apache Derby

Please note that this is *NOT* a benchmark! The purpose is to see the relative performance over time.

Figures are updated on a nightly basis, with the newest development branch of Apache Derby.

The current baseline is based on repeated runs of the Apache Derby "10.2.2.0" release.

Test environment:

• Solaris 10 x86, 2x2400MHz AMD Opteron, 2GB RAM.
• Java Virtual Machine version 6 (Sun)
• Derby is using two separate disks, one for data files and one for log
• 10 minutes runtime (plus 30 seconds warmup and 5 seconds cooldown)
• Derby is checked out and built every day at 22.00 CET/CEST
• Derby is configured with sane=false (not debug mode)

Latest Derby revision: 907791 Changes

Log of performance events

NOTE:

• The server machines used for running these tests were upgraded from running Solaris 10 MU2 32 bit version to running Solaris 10 MU3 64 bit version on February 6th 2007.
• The Java version used for running the test was upgraded to JDK6 (from JDK 5 update 7) on February 9th 2007.
• The server machines were upgraded to Solaris 10 MU5 on May 24 2008.
• The internal tests are currently being replaced by tests from the Apache Derby repository. The results may be more unstable than usual until this process has been completed.
• One of the disks on the server machine behaved strangely after a power failure on January 2 2009 and caused slow-down for update-intensive tests. The issue was resolved on January 7 2009.

TPC-B like load

Data volume is 1 branch per client, 10 tellers per client, 100.000 accounts. Each client is given its own branch to avoid lock conflicts.

Summary last 30 days, 16 clients (% of current baseline):

Results: Last 30 days | Last year | Forever

Single Record Operations

This test operates on a simple table with two integers (the first is primary key), and a character(100). Select and update are performed on a database with 100.000 records for 10 minutes, and insert and delete are allowed to insert or delete a maximum of 100.000 records.

Primary key operations

Summary last 30 days, 16 clients (% of current baseline):

• Insert (Last 30 days) (Last year) (Forever)
• Select (Last 30 days) (Last year) (Forever)
• Update (Last 30 days) (Last year) (Forever)
• Delete (Last 30 days) (Last year) (Forever)

Non-primary indexed column

All operations are based on the content of the second column (integer) which is indexed.

Summary last 30 days, 16 clients (% of current baseline):

• Insert (Last 30 days) (Last year) (Forever)
• Select (Last 30 days) (Last year) (Forever)
• Update (Last 30 days) (Last year) (Forever)
• Delete (Last 30 days) (Last year) (Forever)

Non-primary non-indexed column

All operations are based on the content of the second column (integer), which does not have an index.

Summary last 30 days, 16 clients (% of current baseline):

• Select (Last 30 days) (Last year) (Forever)
• Update (Last 30 days) (Last year) (Forever)
• Delete (Last 30 days) (Last year) (Forever)

CLOB operations

All operations select a row based on its primary key column (integer). The row also contains a CLOB(100) column which is fetched and read.

Summary last 30 days, 16 clients (% of current baseline):

(Last 30 days) (Last year) (Forever)

JOIN on Primary Key

Summary last 30 days, 16 clients (% of current baseline):

(Last 30 days) (Last year) (Forever)