The minimal requirements of a system running openBIS are:
- Operating System: Linux / MacOS X
unzipneed to be installed and in the
PATHof the openBIS user.)
- Java Runtime Environment: recent versions of Oracle JRE 8
- PostgreSQL 9.6 or newer
We find Linux to be the easiest choice to get an openBIS server running quickly.
For recommended memory settings, see Recommended Java and Postgres memory settings.
An SMTP server needs to be accessible and configured if you want to obtain notifications.
The server distribution is a
tar file named
.tar.gz. It contains:
console.properties:configuration file for a console/non-gui installation
extract.sh:helper script for installation
jul.config:Log configuration for the openBIS install process
openBIS-installer.jarJava archive containing openBIS
run-console.shInstallation script for console/non-gui installation
run-ui.shInstallation script for gui installation
- Create a service user account, i.e. an unprivileged, regular user account. Do not run openBIS as root!
- Gunzip the distribution on the server machine into some temporary folder.
Run either the console/non-gui installation script or the gui installation script:
In the non-gui version you have to edit the
NOTE: Please be aware that the directory where openbis is installed should not already exist. Users should specify the directory where they want to install openBIS (in the console.properties or in the graphical installer) and this directory will be created by the installation procedure. If the directory already exists, the installation will fail.
After the successful installation you should have a look at the configuration file called s
ervice.properties. It is located in
This file is a an Extended Properties File. Here is an example which can be used as a template:
All properties starting with
database. specify the settings for the openBIS database. They are all mandatory.
Type of database. Currently only
Part of JDBC URL denoting the host of the database server. If openBIS Application Server and database server are running on the same machine this property should be an empty string.
Part of the name of the database. The full name reads
ID of the user on database server with admin rights, like creation of tables. Should be an empty string if default admin user should be used. In case of PostgreSQL the default admin user is assumed to be
Password for admin user. Usual an empty string.
ID of the user owning the data. This should generally be openbis. The openbis role and password need to be created. In case of an empty string it is the same user who started up openBIS Application Server.
Password of the owner.
The credentials for the database user with the privilege to create a new database depends on the installation and configuration of the PostgreSQL database.
The server is started as follows:
On startup the openBIS server creates the database on PostgreSQL and checks the connection with the remote authentication services (if they are configured). Log files can be found in
<installation folder>/servers/openBIS-server/jetty/logs. Also the following command shows the log:
The first user logged in into the system will have full administrator rights (role
The server is stopped as follows:
Generic openBIS currently supports three authentication systems: a self-contained system based on a UNIX-like passwd file, LDAP, and the Crowd system (see http://www.atlassian.com/software/crowd). Beside this there are also so called stacked authentication methods available. Stacked authentication methods use multiple authentication systems in the order indicated by the name. The first authentication system being able to provide an entry for a particular user id will be used. If you need full control over what authentication systems are used in what order, you can define your own stacked authentication service in the Spring application context file:
The default authentication configuration
In the template service properties, we set
authentication-service = file-ldap-crowd-caching-authentication-service, which means that file-based authentication, LDAP and Crowd are used for authentication, in this order. As LDAP and Crowd are not configured in the template service properties, this effectively corresponds to
file-authentication-service, however when LDAP and / or Crowd are configured, they are picked up on server start and are used to authenticate users when they are not found in the local
passwd file. Furthermore, as it is a caching authentication service, it will cache authentication results from LDAP and / or Crowd in
file <server folder>/jetty/etc/passwd_cache. See section Authentication Cache below for details on this caching.
The file based authentication system
This authentication schema uses the file
<server folder>/jetty/etc/passwd to determine whether a login to the system is successful or not.
<server folder>/jetty/bin/passwd.sh can be used to maintain this file. This script supports the options:
A new user can be added with
If no password is provided with the
-p option, the system will ask for a password of the new user on the console. Please note that providing a password on the command line can be a security risk, because the password can be found in the shell history, and, for a short time, in the
ps table. Thus
-p is not recommended in normal operation.
The password of a user can be tested with
The system will ask for the current password on the console and then print whether the user was authenticated successfully or not.
An account can be changed with
An account can be removed with
The details of an account can be queried with
All accounts can be listed with
The password file contains each user in a separate line. The fields of each line are separated by colon and contain (in this order): User Id, Email Address, First Name, Last Name and Password Hash. The Password Hash field represents the salted SHA1 hash of the user's password in BASE64 encoding.
The interface to LDAP
To work with an LDAP server, you need to provide the server URL with (example) and set the
authentication-service = ldap-authentication-service
and the details of an LDAP account who is allowed to make queries on the LDAP server with (example)
Note: A space-separated list of URLs can be provided if distinguished name and password are valid for all specified LDAP servers.
The interface to Crowd
authentication-service = crowd-authentication-service in
passwd file has no effect. Instead, the following properties need to be configured via the following properties.
The URL (without port information):
The Port of the URL:
The name of the application account in Crowd:
The password of the application account in Crowd:
If Crowd is used as an authentication service, the IP of the openBIS server and the name (of the openBIS application) has to be registered with the Crowd server.
If configuring a caching authentication service like
file-ldap-crowd-caching-authentication-service, authentication results from remote authentication services like LDAP and / or Crowd are cached locally in the openBIS Application Server. The advantage is a faster login time on repeated logins when one or more remote authentication services are slow. The disadvantage is that changes to data in the remote authentication system (like a changed password or email address) are becoming known to openBIS only with a delay. In order to minimize this effect, the authentication caching performs "re-validation" of authentication requests asynchronously. That means it doesn't block the user from logging in because it is performed in different thread than the login.
There are two service properties which give you control over the working of the authentication cache:
authentication.cache.timelets you set for how long (after putting it into the cache) a cache entry (read: "user name and password") will be kept if the user does not have a successful login to openBIS in this period of time (as successful logins will trigger re-validation and thus renewal of the cache entry). The default is 28h, which means that users logging into the system every day will never experience a delay from slow remote authentication systems. A non-positive value will disable caching.
authentication.cache.time-no-revalidationlets you set for how long (after putting it into the cache) a cache entry will not be re-validated if the login was successful. This allows you to reduce the load that openBIS creates on the remote authentication servers for successful logins of the same user. The default is 1h. Setting it to 0 will always trigger re-validation, setting it to
authentication.cache.timewill not perform re-validation at all and thus expire every cache entry after that time.
An administrator with shell access to the openBIS Application Server can see and change the current cache entries in the file
. The format is the same as for the file-based authentication system (see section The file based authentication system above), but has an additional field Cached At added to the end of each line. Cached At is the time (in milli-seconds since start of the Unix epoch, which is midnight Universal Time Coordinated, 1 January 1970) when the entry was cached. Removing a line from this file will remove the corresponding cache entry. The authentication cash survives openBIS Application Server restarts because of this persisted file. If you need to clear the cache altogether, it is sufficient to remove the
passwd_cache file at any time. No server restart is needed to make changes to this file take effect.
You can switch off authentication caching by either setting
authentication.cache.time = -1, or by choosing an authentication service which does not have
caching in its name.
In order to allow anonymous login a certain user known by openBIS (not necessarily by the authentication system) has to be specified. This is done by the property
user-for-anonymous-login. The value is the user ID. The display settings and the authorization settings of this user are used for the anonymous login.
Anonymous login is possible with URL parameter
anonymous set to
true or by property
default-anonymous-login in web configuration properties (see Web Client Customization).
Single Sign On Authentication
Currently only Shibboleth SSO is supported. For more details see Single Sign On Authentication.
openBIS authorization is described here: Authorization.
Login Page - Banners
To add banners to the main OpenBIS change
loginHeader.html page. It is stored in the same directory as
index.html. Note that if the height of
loginHeader.html is too big, the content may overlap with the rest of the OpenBIS login page.
Example of the
For announcements you have to edit the
index.html file. Here is an example showing the tail:
Note: the current work around with
br tags between the lines ensures that the login box is still centered.
To reconfigure some parts of the openBIS Web Client and Data Set Upload Client, prepare the configuration file and add the path to the value of
web-client-configuration-file property in openBIS
Web client customizations
- Enable the trashcan. When the trashcan is enabled, deleting entities only marks them as deleted but not deletes them physically (it is also called "logical deletion"). When clicking on the trashcan icon in the Web GUI, the user can see all of his deletion operations and can revert them individually. Only an admin can empty the trashcan and thus delete the entities physically. Only with enabled trashcan is it possible to delete complete hierarchies (e.g. an experiment with samples and datasets attached).
- Default view mode (
SIMPLE/NORMAL) that should be used if user doesn't have it specified in a URL.
- Replacement texts for 'Experiment' and 'Sample' by
- Anonymous login by default.
- Sample, material, experiment and data set
detail viewscan be customized by:
- hiding the sections (e.g. attachments)
data set detail viewcan be customized by:
File Viewfrom the list of available reports in
- Technology specific properties with property
technologieswhich is a comma-separated list of technologies. For each technology properties are defined where the property names start with technology name followed by a dot character.
Data Set Upload Client Customizations
It is possible to restrict the set of data set types available to the user in the data set uploader. This is useful when there are some data set types that a user would never upload; for example, if there are data set types that are used only internally exist only to support third-party software.
The restriction is specified in the web-client.properties file using either a whitelist or a blacklist. If both are specified, the whitelist is used. To specify a whitelist, use the key
creatable-data-set-types-whitelist; for a blacklist, use the key
creatable-data-set-types-blacklist. The value for the property should be a comma-separated list of regular-expression patterns to match. In the case of the whitelist, data set types that match the specified patterns are shown to the user, whereas for the blacklist, the data set types that match the specified patterns are those that are not shown to the user.
- Specifying a whitelist
Assume we have the following data set types in our system:
PROCESSED-DATA, MICROSCOPE-IMAGE, IMAGE-SCREENING, ANALYSIS, ANALYSIS-FEATURES, THUMBNAIL1, THUMBNAIL-BIG
In this case, the follwing data set types will be available to the user:
MICROSCOPE-IMAGE, IMAGE-SCREENING, ANALYSIS, THUMBNAIL1
- Specifying a blacklist
Assume we have the following data set types in our system:
PROCESSED-DATA, MICROSCOPE-IMAGE, IMAGE-SCREENING, ANALYSIS, ANALYSIS-FEATURES, THUMBNAIL1, THUMBNAIL-BIG
In this case, the follwing data set types will be available to the user:
PROCESSED-DATA, ANALYSIS-FEATURES, THUMBNAIL-BIG
Full web-client.properties Example
Configuring File Servlet
This service is specially tailored for web applications requiring to upload files to the system without using the DataSet concept, it was meant to be used for small images and rich text editors like CKEditor.
|Property Key||Default Value||Description|
|file-server.maximum-file-size-in-MB||10||Max size of files.|
|file-server.repository-path||../../../data/file-server||Path where files will be stored, ideally should be a folder on the same NAS you are storing the DataSets.|
|file-server.download-check||true||Checks that the user is log in into the system to be able to download files.|
Configuring DSS Data Sources
It is quite common that openBIS AS is using a database filled by DSS. Depending on the DSS (specified by the DSS code) and the technology different databases have to be used.
Configuration is best done by AS core plugins of type
dss-data-sources. The name of the plugin is just the DSS code. The following properties of
plugin.properties are recognized:
|technology||Normally the technology/module folder of the core plugin specifies the technology/module for which this data source has to be configured. If this is not the case this property allows to specify the technology/module independently.|
|database-driver||Fully qualified class name of the data base driver, e.g. |
|database-url||URL of the database, e.g.|
|username||Optional user name needed to access database.|
|password||Optional password needed to access database.|
|validation-query||Optional SQL script to be executed to validate database connections.|
|database-max-idle-connections||The maximum number of connections that can remain idle in the pool. A negative value indicates that there is no limit. Default: -1|
|database-max-active-connections||The maximum number of active connections that can be allocated at the same time. A negative value indicates that there is no limit. Default: -1|
|database-max-wait-for-connection||The maximum number of seconds that the pool will wait for a connection to be returned before throwing an exception. A value less than or equal to zero means the pool is set to wait indefinitely. Default: -1|
The interval (in ms) between two regular log entries of currently active database connections if more than one connection is active. Default: Disabled
The number of active connections that will trigger a NOTIFY log and will switch on detailed connection logging. Default: Disabled
database-url can be omitted if a
etc/dss-datasource-mapping is defined. For more see Sharing Databases.
Changing the Capability-Role map
openBIS uses a map of capabilities to roles to decide what role is needed to perform a given action. The defaults can be overridden by creating a file
etc/capabilities. One line in this file has one of the following formats:
which sets a new (minimal) role for the given capability. There is a special role
INSTANCE_DISABLED which allows to completely disable a capability for an instance. Note: to set multiple roles for single capability use multiple lines in the file.
This is the default map:
Delete datasets (this capability IS NOT enough to delete datasets with deletion_disallow flag set to true in their type - see FORCE_DELETE_DATASET)
|Delete datasets (this capability IS enough to delete datasets with deletion_disallow flag set to true in their type - see DELETE_DATASET)|
Move dataset from data store into archive
Copy back dataset from archive to data store
|Prevent data sets from being archived|
|Release locked data sets|
Registration / update of experiments, samples and materials in one go
|The user will become space admin of the freshly created space|
Permanently delete experiments, samples and datasets in the trashcan (this capability IS NOT enough to delete datasets with deletion_disallow flag set to true in their type - see FORCE_PURGE)
|Permanently delete experiments, samples and datasets in the trashcan (this capability IS enough to delete datasets with deletion_disallow flag set to true in their type - see PURGE)|
Get back experiments, samples and datasets from the trashcan
|Re-assign a sample to a new experiment (called in 'register experiment', 'update experiment', 'update sample'')|
|Re-assign a sample to a new space (called in 'update sample')|
|All search or list operations being performed on behalf of another user. Supposed to be used by a service user for server-to-server communication tasks.|
Older versions of openBIS used to allow changing entity relationships to regular
SPACE_USER. If you want to get this behavior back, put these lines into
Capability Role Map for V3 API
|Method of IApplicationServerApi||Default Roles||Capability|
|confirmDeletions, forceDeletion == false||PROJECT_ADMIN, SPACE_ETL_SERVER||CONFIRM_DELETION|
|confirmDeletions, forceDeletion == true||disabled||CONFIRM_DELETION_FORCED|
|createRoleAssignments, instance role||INSTANCE_ADMIN||CREATE_INSTANCE_ROLE|
|createRoleAssignments, space role||SPACE_ADMIN||CREATE_SPACE_ROLE|
|createRoleAssignments, project role||PROJECT_ADMIN||CREATE_PROJECT_ROLE|
|createVocabularyTerms, official terms||PROJECT_POWER_USER, SPACE_ETL_SERVER||CREATE_OFFICIAL_VOCABULARY_TERM|
|createVocabularyTerms, unofficial terms||PROJECT_USER, SPACE_ETL_SERVER||CREATE_UNOFFICIAL_VOCABULARY_TERM|
|deleteProjects||SPACE_POWER_USER, PROJECT_ADMIN, SPACE_ETL_SERVER||DELETE_PROJECT|
|deleteRoleAssignments, instance role||INSTANCE_ADMIN||DELETE_INSTANCE_ROLE|
|deleteRoleAssignments, space role||SPACE_ADMIN||DELETE_SPACE_ROLE|
|deleteRoleAssignments, project role||PROJECT_ADMIN||DELETE_PROJECT_ROLE|
|updateDataSets, properties||PROJECT_POWER_USER, SPACE_ETL_SERVER||UPDATE_DATASET_PROPERTY|
|updateExperiments, attachments||PROJECT_USER, SPACE_ETL_SERVER||UPDATE_EXPERIMENT_ATTACHMENT|
|updateExperiments, properties||PROJECT_USER, SPACE_ETL_SERVER||UPDATE_EXPERIMENT_PROPERTY|
|updateMaterials, properties||INSTANCE_ADMIN, INSTANCE_ETL_SERVER||UPDATE_MATERIAL_PROPERTY|
|updatePersons, set home space||SPACE_ADMIN||UPDATE_HOME_SPACE|
|updateProjects||SPACE_POWER_USER, PROJECT_ADMIN, SPACE_ETL_SERVER||UPDATE_PROJECT|
|updateProjects, attachments||SPACE_POWER_USER, PROJECT_ADMIN, SPACE_ETL_SERVER||UPDATE_PROJECT_ATTACHMENT|
|updateSamples, attachments||PROJECT_USER, SPACE_ETL_SERVER||UPDATE_SAMPLE_ATTACHMENT|
|updateSamples, properties||PROJECT_USER, SPACE_ETL_SERVER||UPDATE_SAMPLE_PROPERTY|
|updateVocabularyTerms, official terms||PROJECT_POWER_USER, SPACE_ETL_SERVER||UPDATE_OFFICIAL_VOCABULARY_TERM|
|updateVocabularyTerms, unofficial terms||PROJECT_USER, SPACE_ETL_SERVER||UPDATE_UNOFFICIAL_VOCABULARY_TERM|
Querying Project Database
In some customized versions of openBIS an additional project-specific database is storing data from registered data sets. This database can be queried via SQL Select statements in openBIS Web application. In order to protect modification of this database by malicious SQL code openBIS application server should access this database as a user which is member of a read-only group. The name of this read-only group is project specific.
It is possible to configure openBIS to query multiple project-specific databases.
Create Read-Only User in PostgreSQL
A new user (aka role) is created by
This new user is added to the read-only group by the following command:
The name of the read-only group can be obtained by having a look into the list of all groups:
Note that by default openBIS creates a user
openbis_readonly which has read-only permissions to all database objects. You can use this user to access the openBIS meta database through the openBIS query interface.
To enable querying functionality for additional databases in openBIS Web application a core plugin of type query-databases has to be created. The following
plugin.properties have to be specified:
Label of the database. It will be used in the Web application in drop down lists for adding / editing customized queries.
JDBC Driver of the database, e.g.
JDBC URL to the database containing full database name, e.g.
Above-mentioned defined read-only user.
Password of the read-only user.
In order to configure authorization two additional properties can be configured:
To which data-space this database belongs to (optional, i.e. a query database can be configured not to belong to one data space by leaving this configuration value empty).
What role is required to be allowed to create / edit queries on this database (optional, default:
The given parameters data-space and creator-minimal-role are used by openBIS to enforce proper authorization.
For example, if
is configured, then for the query database configured with key
- only a
SPACE_ADMINon data space
INSTANCE_ADMINare allowed to create / edit queries,
- only a user who has the
OBSERVERrole in data space
CISDis allowed to execute a query.
For query databases that do not belong to a space but that have a column with any of the magic column names, the query result is filtered on a per-row basis according to what the user executing the query is allowed to see. In detail this means: if the user executing the query is not an instance admin, filter out all rows which belong to a data space where the user doesn't have a least the observer role. The relationship between a row and a data space is established by means of the experiment / sample / data set whose
permId is given by one of the magical column names.
For sensitive data where authorization needs to be enforced, there are two setups possible:
- Configure a query database that does not belong to a data space and set the creator-minimal-role to
INSTANCE_ADMIN. Any instance admin can be trusted to understand authorization issues and ensure that only queries are added for this query database that contain a proper reference to an experiment / sample / data set. This way, it can be ensured that only properly filtered query results are returned to the user running the query.
- Configure a query database that does belong to a specific data space and set the creator-minimal-role to
POWER_USER. The datastore server (or whatever server maintains the query database) ensures that only information related to the configured data space is added to the query database. Thus whatever query the power user writes for this database, it will only reveal information from this data space. As only users with
OBSERVERrole on this data space are allowed to execute the query, authorization is enforced properly without the need of filtering query results.
Master data import/export
The master data of openBIS comprises all entity/property types, property assignments, vocabularies etc. needed for your customized installation to work. The system offers a way to export/import master data via Jython scripts. More information on how to do create such scripts and run them manually see the advanced guide Jython Master Data Scripts.
A master data script can be run automatically by start up of the AS if it is defined in an AS core plugin. The script path should be
<installation directory>/servers/core-plugins/<module name>/<version number>/as/initialize-master-data.py. For more details about the folder structure of core plugins see Core Plugins. If there are several core plugins with master data scripts the scripts will be executed in alphabetical order of the module names. For example, the master data script of module
screening-optional will be executed after the master data script of module
screening has been executed.
Execution of master data script can be suppressed by disabling
initialize-master-data core plugin. For more details see Core Plugins.
Limit of open files
When putting a lot of files in a drop box you might run into the problem of '
too many open files error'. Please consider changing the ulimit value (for RHEL6 edit
/etc/security/limits.conf ) to a higher value.
Runtime changes to logging
<installation directory>/servers/openBIS-server/jetty/bin/configure.sh can be used to change the logging behavior of openBIS application server while the server is running.
The script is used like this: configure.sh [command] [argument]
The table below describes the possible commands and their arguments.
Turns on / off detailed service call logging.
When this feature is enabled, openBIS will log about start and end of every service call it executes to file <installation directory>/servers/openBIS-server/jetty/log/openbis_service_calls.txt
Turns on / off logging of long running invocations.
When this feature is enabled, openBIS will periodically create a report of all service calls that have been in execution more than 15 seconds to file <installation directory>/servers/openBIS-server/jetty/log/openbis_long_running_threads.txt.
Turns on / off logging about database connection pool activity.
When this feature is enabled, information about every borrow and return to database connection pool is logged to openBIS main log in file <installation directory>/servers/openBIS-server/jetty/log/openbis_log.txt
|log-db-connections||no argument / minimum connection age (in milliseconds)||5000|
When this command is executed without an argument, information about every database connection that has been borrowed from the connection pool is written into openBIS main log in file <installation directory>/servers/openBIS-server/jetty/log/openbis_log.txt
If the "minimum connection age" argument is specified, only connections that have been out of the pool longer than the specified time are logged. The minimum connection age value is given in milliseconds.
Turns on / off logging of stacktraces.
When this feature is enabled AND debug-db-connections is enabled, the full stack trace of the borrowing thread will be recorded with the connection pool activity logs.
Turns on / off database connection pool logging to separate file.
When this feature is disabled, the database connection pool activity logging is done only to openBIS main log. When this feature is enabled, the activity logging is done ALSO to file <installation directory>/servers/openBIS-server/jetty/log/openbis_db_connections.txt.
Deleted Entity History
Logging the history of deleted entities can be enabled / disabled in service.properties using setting
entity-history.enabled = [true | false]
The default value is false (meaning, entity history is disabled).
Deleted entity history can be queried with script show-history.sh, which is located in $OPENBIS_INSTALL_DIR/bin
Samples with datasets and no experiments
In the openBIS UI users could detach samples with container data sets from the experiment. This bug was fix on version S176 released on 14 of march of 2014.
The following SQL script lists all samples with data sets but no experiments:
If the last query shows no output the system is fine, if not, it can be repaired with the following update query.