Child pages
  • openBIS FAQ
Skip to end of metadata
Go to start of metadata

General

What is the difference between attachments and datasets?

Attachments offer much simpler functionality than datasets, but in some cases can be easier to use and upload.
Attachments should be used only to store small amounts of unstructured data (technical details: they are stored directly in the database in contrast to datasets, which are stored in the file system).
Datasets are "first class objects" in openBIS and are much more sophisticated than attachments - they can be annotated, archived, deleted in batches, visualized in special ways etc.
A nice feature of attachments is that they offer the concept of "versions" - an older version of attachment can be retrieved.
Only attachments can be linked to projects directly.

What is the difference between Master Data and Meta Data?

All experiment types, sample types, vocabularies, property types and their assignments and all scripts are master data.  

It is easiest to describe with examples:
Let's assume I have a sample type called 'Flowcell'. This is master data. If I enter an existing flowcell called '121127_SN792_0186_AD1F9VACXX' to openBIS I have added Meta Data.

Where can I get news on openBIS?

If you want to get regular information and news on openBIS, e.g. about new development, you should subscribe to the openBIS mailing list (mailing list archive). The list is in general "low-volume".

Administration

How much time do I need to plan for introducing openBIS in my lab?

This depends on the complexity of your lab and on your prior experience with databases and information systems. openBIS is a full-blown information system and will require you to learn some concepts before it becomes useful to you. If you have prior experience with databases and installing software on Unix systems, you should plan about a week to setup the system and understand the most important concepts. Then you can start integrating your lab's data sources. We recommend you start with a simple data source to get used to writing dropbox scripts. Your first dropbox script will likely take you something like two to three days to write. If you don't have prior experience with databases and basic Unix skills, you will have to learn basics about these topics on the way. Unfortunately we can't give you any estimate for that.

What hardware do I need to run openBIS?

This depends on the number of users and data sources you have and the size of your data. A test instance of openBIS can run already on a mini server with 2 CPU cores, 4GB RAM and a 100GB harddisk. A regular openBIS server for a high-throughput lab will usually have specs like 4 CPU cores, 16GB RAM, a fast 300GB harddisk for the database and access to one or more NAS servers in the two-digit TB range. For high data volumes and a larger user community, mid-size servers may be required. In this range we cannot give general recommendations as the needs are too diverse. To give you an idea, we list the specs of a larger high-thoughput lab: 12 CPU cores Intel Xeon X5680 3.33GHz, 148GB RAM, 500GB SSD, 4TB RAID6 array (DAS), dynamically growing share on an EMC NAS system. Many admins prefer virtual machines, as theirs specs can be upgraded easily when needed.

Independent of the size of your server, do not forget that you need a data backup that covers the database and the data store!

What software prerequisites do I need to have installed for openBIS?

We recommend you use Linux to run an openBIS server. There is not much experience with running openBIS on Windows, even though it should work in principle. Note: For testing we use Redhat Enterprise Linux 6.

All you need to have installed other than the operating system itself is:

  • Recent versions of Oracle Java 7 or Java 8 (recommended)
  • PostgreSQL 9.2, 9.3 or 9.4 or 9.5 (recommended)

Problems with SSL

openBIS is shipped with a self-signed SSL certificate which is not accepted by web browsers without an explicit confirmation.
It causes that non advanced users can have problems with using the system.

If you access openBIS through the API you have to add "-Djavax.net.ssl.trustStore=/path_to_openbis/servers/datastore_server/etc/openBIS.keystore" to the JVM to accept openBIS self-signed SSL certificate.

Switching off SSL in openBIS versions newer than sprint S207 or 16.05.0

If you are running the instance in a test mode or do not intend to arrange a valid certificate, you can switch off SSL in a following way:

  • Change the AS jetty configuration:
    1. rename the https.ini and ssl.ini files in servers/openBIS-server/jetty/start.d/ to https_orig.ini and ssl_orig.ini
    2. create a new file called http.ini with this content

# ---------------------------------------
# Module: http
--module=http
### HTTP Connector Configuration
## HTTP port to listen on
jetty.port=8080
## HTTP idle timeout in milliseconds
http.timeout=30000
## HTTP Socket.soLingerTime in seconds. (-1 to disable)
# http.soLingerTime=-1
## Parameters to control the number and priority of acceptors and selectors
# http.selectors=1
# http.acceptors=1
# http.selectorPriorityDelta=0
# http.acceptorPriorityDelta=0

 

change DSS configuration file in servers/datastore_server/etc/service.properties:

  • in host-address replace https with http
  • set use-ssl to false
  • change the port in server-url to 8080
    Your file will look like this:

# host name of the machine on which the datastore server is running
host-address = http://my-openbis-domain.com
...

# Port
port = 8081
use-ssl = false
...
# The URL of the openBIS server
server-url = ${host-address}:8080
  • restart AS and DSS (e.g. by calling alldown.sh and allup.sh)

Switching off SSL in openBIS versions older than sprint S207 or 16.05.0

If you are running the instance in a test mode or do not intend to arrange a valid certificate, you can switch off SSL in a following way:

  • change AS jetty configuration file servers/openBIS-server/jetty/etc/jetty.xmlbe replacing

              <New class="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
    	             <Set name="Port">8443</Set>
    	                 <Set name="maxIdleTime">30000</Set>
                       <Set name="Acceptors">2</Set>
                       <Set name="AcceptQueueSize">100</Set>
    	                 <Set name="Keystore">
    	                     <SystemProperty name="jetty.home" default="." />/etc/openBIS.keystore
    	                 </Set>
                      <Set name="Password">changeit</Set>
                      <Set name="KeyPassword">changeit</Set>
              </New>
    

    with

              <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
                 <Set name="Port">8080</Set>
                 <Set name="maxIdleTime">30000</Set>
                 <Set name="Acceptors">2</Set>
                 <Set name="AcceptQueueSize">100</Set>
              </New>
    
  • change DSS configuration file in servers/datastore_server/etc/service.properties:
    • in host-address replace https with http
    • set use-ssl to false
    • change the port in server-url to 8080
      Your file will look like this:

      # host name of the machine on which the datastore server is running
      host-address = http://my-openbis-domain.com
      ...
      use-ssl = false
      ...
      # The URL of the openBIS server
      server-url = ${host-address}:8080
      
  • restart AS and DSS (e.g. by calling alldown.sh and allup.sh)

Note that now openBIS is running on port 8080 and because of that the URL with which you can access it in the web browser has changed.

Be careful, if you do not use SSL users passwords are transmitted as plain text over the wire.

 

Postgres performance 

On heavily used openBIS instances the performance of the Postgres DB can decrease. Even a full autovacuum does not help. Therefore we currently recommend to do a full dump/load cycle of the DB to get back the full performance.

 

Running openBIS locally for demonstration purposes or testing

If you want to run  openBIS locally use the general address for your computer, which is localhost or 127.0.0.1 as a configuration in the service.properties files of openBIS. This might prevent problems when the openBIS Application Server needs to communicate with the openBIS Datastore Server. Additionally you might run into problems of Cross-Site-Scripting Errors. If you still have problems also check the /etc/hosts file of your computer to spot any misconfiguration.

Tips and Tricks

How do I find out the version of openBIS in the Web Browser?

Login to openBIS. On the right hand-side, click on your username, and then click on the "About" button: this will open up a pop-up window with the openBIS version.

How do I find out the version of openBIS on the shell?

On version S141 or later, call:

In order versions, have a look at <OPENBIS_HOME>/jetty/webapps/openbis/WEB-INF/classes/BUILD*:
General openBIS:

Screening openBIS:

How to force openBIS to re-create its search index?

In rare cases it can occur that the full-text search index gets out-of-sync with the database. In this case the re-creation of the fulltext search indices can be forced by:

You can check in the file openBIS-server/jetty/logs/openbis_log.txt whether the indexing is successful: look for lines containing OPERATION.DefaultFullTextIndexer, and in particular for lines like:

After you've got the last of the four lines, you're set and your full-text search index is up-to-date again.

Apache 'Proxy Error' pops up

Currently we have not a full solution. Only an Apache setting which lowers the number of those errors:

Add to /etc/httpd/conf/httpd.conf:

Timeout 5400
ProxyTimeout 5400

And restart Apache: /sbin/service httpd restart

The problem is described here as well: http://stackoverflow.com/questions/169453/bad-gateway-502-error-with-apache-mod-proxy-and-tomcat

 

Using iptables instead of Apache

Instead of using Apache to serve openBIS on port 443 you can also use iptables for this. This might solve several issues:

 

Limit of open files

When putting a lot of files in a drop box you might run into the problem of  'too many open files error'. Please consider changing the ulimit value (for RHEL6 set nofile in /etc/security/limits.conf ) to a higher value.

The default value of RHEL, which is 1024, is known to be sufficient for most usage scenarios.

 

Relation between the Sharding Structure of the DSS store and the Data Set IDs:

Get the shard folder structure from a given Data Set ID:

 

The corresponding calculator which uses the the MD5sum to generate the folder structure:

ch.systemsx.cisd.openbis.dss.generic.shared.utils.DatasetLocationUtil