Introduction

OpenBIS is an Open Biology Information System, designed to facilitate robust data management for a wide variety of experiment types and research subjects. It allows tracking, annotating, and sharing of data throughout distributed research projects in the biological sciences, including, but not limited to Phenomics, Transcriptomics, Genomics, Proteomics and Metabolomics, and provides facilities for subject-specific concerns, such as HCS/Imaging, as well as Next Generation Sequencing.

This document introduces new users to the basics of working with openBIS. The first section provides an example of how openBIS can be practically used in a lab. The following section, #Authorization, Roles and Privileges describes how to define users with different roles in openBIS. The section  #Overview on How to Set Up a New openBIS Instance provides a general overview on the steps you need to follow when setting up a completely new instance of openBIS. The following four sections, #Storing Information in openBIS, #Registering Samples, Experiments and Data Sets, #Managing Controlled Vocabularies, and #Curating Metadata, explain how to add and maintain data in openBIS. The sections #Registering Master Data and #Curating Master Data   describe the aspects of setting up and maintaining an openBIS installation. The sections #Browsing in openBIS, #Searching, #Working with Tables, and #Downloading of data using CIFEX explain how to find, summarize, and export information stored in openBIS. The remaining sections describe advanced and/or project-specific features.

Terms Used in the Documentation

Throughout this documentation, certain terms will be used with the following technical meanings:

• A sample refers to any object that has been observed, measured, or compared to another. It must be uniquely identifiable, which means that any two samples must be distinguishable from one another. Please note that different use cases may use the term "sample" with slightly different meanings, dependent upon the context and utility.
  For example, in one experiment a sample may refer to a biological specimen taken from an organism, while in another it could refer to a protein extract analyzed in a mass spectrometer. In a third experiment, the term "sample" could refer to an individual well in a multi-titer plate containing cells of different phenotypes.  Each of these cases refers to a different sample type in openBIS.

• An experiment is an experimental or theoretical investigation involving samples. The simplest experiment would involve just one sample, but, in general, most would include many. A typical example of an experiment would be a High Content Screening campaign, where the samples are the plates being screened.
  Different types of experiments may make use of data queries that are unique to their fields. In order to accommodate these differences, openBIS provides a variety of experiment types, such as HCS_CAMPAIGN for the example above.

• A data set is the computer's representation of a series of sample measurements, or the results of computational processing derived from those measurements.  As with samples and experiments, data sets also have specific data set types to better handle searching and analysis needs, such as PROTEOMICS_MASSSPEC_QUANTIFICATION for HCS_FEATURE. Additionally, data sets have a file type, which represents the formatting used to store the data, such as TSV for tab-separated values, or XML for files in the extensible markup language format.

Additionally, here are a few helpful ideas to keep in mind about samples, experiments and data sets as they are used in openBIS.

• A sample is something that is measured. A data set is a measurement or the analysis of a measurement. An experiment is the process of measuring and analyzing things.

• The sample should identify what has been measured, e.g., the biological entity

• Samples are assigned to experiments before they are measured

• When the sample is measured, the computer representation of the measured data is called a data set. The data set is assigned to a sample, but the data set is not the sample itself.

Typographic Conventions

In this documentation, we use the following typographic conventions.

• Italics are used to introduce new terms
• Monospace text is used to refer to user interface elements such as menus and buttons

User Scenario: openBIS - A Flexible Annotation Service

Ian Smith leads a mass spectrometry laboratory. He plans to document all of the data which has been generated in his lab, and intends to publish this data in a well-organized manner. His goal is to allow all lab members to search for particular experiments, samples or datasets generated in the lab. New lab members should have access to all data and metadata in a clearly represented information system where they can easily browse for data generated in a particular lab project.

For collaborative purposes this system should be a medium with which to share data. It is mandatory that all laboratory members upload all relevant datasets with the associated metadata in a consistent way.

All samples should contain a description of the origin of the sample and a record of who prepared it. Protocols for sample preparation and MS instrument type should also be stored. Result data should be visible as well, either in table format or graphical representations.

Brian is a postdoctoral researcher in Ian's lab and uploads all of the data from his recently accepted article in Nature. In order to share this information with the lab, he should follow the following procedure.

Registering a new experiment

First, he creates a new experiment by selecting New -> Experiment
The experiment code is defined manually as NEURAMINIDASE_JURKAT
The project is chosen from a drop down menu of existing Projects: CS_PROTEOME

Adding experiment metadata

Metadata, such as the organism the samples were derived from, the instrument which was used to analyze the samples, and the person who ran the samples (operator) is added by choosing terms from the lab's internal controlled vocabularies or by typing the relevant name.

Registering samples

The samples associated with the experiment are uploaded as shown below by using a .tsv file (SAMPLE-N_GLYCOPROTEIN.tsv) prepared in advance.

The uploaded file looks like this:

Browsing experiments, samples

Within the Experiment Browser (select Browse -> Experiments), all lab members can view the newly added experiment.
By clicking on the link of the Experiment name (NEURAMINIDASE_JURKAT) a new window with a detailed view of the experiment will open:

Adding attachment metadata

In the next step, Brian attaches a word file to the experiment. This file contains a sample preparation protocol and an image displaying the sample analysis workflow.

Within the Experiment detailed view in the right hand window, he first chooses the Attachments tab (1). Second, he clicks on the Add Attachment button (2). Then, a new window opens where he can upload the relevant files, including the description and title of the attachment (3).

Viewing attachments

By clicking on the respective attachments, the methods section and the graphical workflow diagram can be displayed:

Registering datasets

Using the dataset uploader application integrated with openBIS, Brian uploads two protein identification lists from neuraminidase treated and untreated samples. He also uploads a jpeg image displaying the proteins identified based on molecular function in the same step.

Brian selects Import -> Data Sets to start the dataset uploader. In the next step, he chooses the files to be uploaded and the respective experiment to which the datasets belong. In the last step, he uploads all three files by clicking on the button in the upper right corner of the window.

Viewing datasets

The newly added datasets can be seen in the experiment detailed view by choosing the Data Sets tab in the right hand window. By clicking on the dataset code, a new tab with the dataset information opens up.

Sharing and downloading data using CIFEX

To share datasets, the collaborator highlights the datasets of interest and clicks on Export Data in the bottom right corner of the window. This opens up a new window where the CIFEX user name and password must be entered in order to continue. The file name and a descriptive comment regarding the datasets can also be added.

Authorization, Roles and Privileges

OpenBIS allows users to operate under five basic Roles: Observer, User, Power User, Space Admin, and Instance Admin. Furthermore, these privileges can be extended across multiple spaces, if necessary, or custom-tailored so that individual users and groups can have different roles in different spaces. If there are for example 3 groups, A, B and C using the same openBIS instance, each group has its own space. Some users have access only to one space, while some users have access to several spaces: U_B1 can access Space A and Space B; U_A2 can access Space A, Space B and Space C.

Privileges associated with Roles

• Observer
  An observer has read-only access to openBIS.
• User
  A user has additional permissions, such as registering and editing abilities.
• Power User
  A power user has some deleting, editing and processing abilities.
• Space Admin
  A space admin is allowed to manage roles and projects within a given data space.
• Instance Admin
  An Instance Admin has full access to a given openBIS installation.

Adding Users

• You need Admin privileges to add users. You first need to authenticate the user
• To add new users to the system, select Admin -> Authorization -> Users.
• A list of all users will be displayed. Click the Add Person button in the bottom right corner to continue.
• Enter the user ID of the person you wish to add and save.

Working with User Groups

• To modify User Groups, you will need Admin privileges.
• Select Admin -> Authorization -> User Groups.
• To add a new user group, click the Add Group button in the bottom right corner of the window.
• To add users to an existing group, click on the group name to open a tab for the Group. Add group members by clicking Add Person in the bottom right corner and selecting existing users from the drop down menu that appears.

Assigning Roles to Users

• Assigning user roles requires Admin privileges.
• To assign new roles, select Admin -> Authorization -> Roles.
• Click on the Assign Role button at the bottom of the Role Assignment Browser.
• Select a role and the person from a drop-down list.
• To change a user's role, select his or her name from the list and click the Assign Role button.
• Select a specific role from the drop down menu. Then select the Person checkbox and click the Save button to apply the change.

Assigning Roles to User Groups

• Assigning and changing group roles requires Admin privileges.
• To assign new roles, select Admin -> Authorization -> Roles.
• To assign or change a group's role, you do not need to select any name from the list. Just click the Assign Role button.
• Choose a group name and a role, making sure to check the User Group checkbox first.
• Click the Save button to apply your changes.

Removing Roles and Groups from a User

• To remove a role or group assignment, select Admin -> Authorization -> Roles to see the Role Assignment browser
• Click on the name of the user or group that you wish to modify
• Click the Release Assignment button to remove the selected role from the user or group. 
  
  

Overview on How to Set Up a New openBIS Instance

When you set up a completely new instance of openBIS you first need to consider how you want to organize your data. The first thing you will have to do is define a Space (e.g. you could use the name of your lab). Then you will have to create samples under this space. Each sample needs to be assigned to a sample type and various properties can be associated to each sample type. The samples can then be organized in experiments and the experiments can be grouped into projects, but these steps are not strictly necessary. If you decide to use experiments, you will need to define experiment types and you can associate properties to each experiment type. In addition, you can also define materials (with material types) and data sets (with data set types) and define properties for each type.

The following sections will explain each step in detail.

Add and Maintain Data and Metadata

Storing Information in openBIS

The openBIS software provides a number of ways to capture and track your research data. In openBIS, there are several kinds of information that are distinguished. The core entities are

• Experiment - the process of investigating a question.
• Sample - Something that is measured.
• Data set - Data that results from a measurement or analysis.
• Material - A complex attribute that might be common to many entities (e.g., an siRNA that appears in many samples).

For each of these core entities, it is possible to capture additional relevant information known as metadata. For example, there may be a project in which several experiments are done, each measuring some property of a sample at different temperatures. In this case, there would be an experiment with a property called temperature. But the same lab may perform different kinds of experiments in which other variables change and temperature is held constant or otherwise irrelevant. For this reason, openBIS defines experiment type, sample type, data set type, and material type. A type defines the metadata relevant to the experiment, sample, data set, or material.

Another concept central to openBIS is something called spaces. Spaces are used specifically to define access permissions. For example, a lab may be doing work for two clients, GeneCo and Widget Therapeutics, that are competitors of each other. In this case, researchers on the GeneCo project should not see the data on the Widget Therapeutics projects. This restriction can be set up using spaces.

Another concept is the project, which is used to group related experiments, as in the example above. The figure below shows the hierarchy used in openBis.

Typical day-to-day usage of openBIS revolves around creating, viewing, and editing experiments, samples, materials, and data sets. The requirements for performing these tasks are described in this section. The section #Registering Master Data describes how to set up types, spaces, and projects, which are typically created and modified less frequently.

Authorization

The following privileges are required to create the entities mentioned above:

Registering Samples, Experiments and Data Sets

Registering Samples - One-by-one method

• Before registering a new sample, you need to create a Sample Type (unless already defined) by clicking on Admin -> Types -> Sample Types.
• Add new samples to openBIS by selecting New -> Sample.
• Select a Sample Type.
• Complete all mandatory fields (marked with an asterisk).
• The code can be entered either manually or openBIS can automatically generate a code by clicking the button to the right of the Code field (Figure 1).
• Once the first sample has been registered, additional samples will be auto-filled using the data from the first sample (Figure 2).

• Additionally, you can assign the new sample to a pre-existing experiment by clicking and editing the Experiment field.
• Another method of adding a new sample is by clicking the Add Sample button in the Sample Browser view, which can be seen by selecting Browse -> Samples.

Registering Samples - Batch Method

• Before registering new samples, you need to create a Sample Type (unless already defined) by clicking on Admin -> Types -> Sample Types.
• Samples can be added several at a time by selecting Import -> Samples.
• Select a Sample Type from a list of available types.
• To use Batch Import, you must upload a file containing the samples that need to be registered.
• To help you prepare your sample data file, a file template containing instructions and mandatory column headers can be downloaded from the Batch Import Window.
• When new properties are added to the sample type, the template file will automatically be updated.

Registering Experiments - One-by-One Method

• Before registering new experiments, you need to create an Experiment Type (unless already defined) by clicking on Admin -> Types -> Experiment Types.
• Add new experiments by selecting New -> Experiment.
• Select an Experiment Type.
• Make sure to fill out all mandatory fields, which are marked with an asterisk (Figure 1).
• Manually enter your experiment Code, or generate this automatically by clicking the button to the right of the Code field, as described above for the Sample registration.
• After your first experiment is registered, additional experiments will be auto-filled with data from the first experiment (Figure 2).

• Alternatively, you can add a new experiment by clicking the Add Experiment button in the bottom right corner of the experiment browser view. You can find this view by selecting Browse -> Experiments.

Registering Experiments - Batch Method

• Before registering new experiments, you need to create an Experiment Type (unless already defined) by clicking on Admin -> Types -> Experiment Types.
• To add more than one experiment at a time, select Import -> Experiments
• Select an Experiment Type.
• By uploading a properly formatted data file, you can register all relevant experiments at once.
• A file template can be downloaded from the Import Experiments Page.
• The only mandatory column in an experiment batch import file is "identifier," which should contain the space, the project, and the experiment code in the following format: /space-code/project-code/experiment-code.
• Additionally, you can specify experiment properties by adding additional columns.
• To see all available properties, download the template file from the Import -> Experiments page.

Registering an Experiment and Associated Samples in one step

• Begin registering the experiment by selecting New -> Experiment.
• Select the correct Experiment Type from the drop-down menu.
• Fill in all mandatory fields, marked with an asterisk.
• In The "Samples" field select "register from a file and attach".
• To help properly format your data file, a file template can be downloaded.
• Once your data file has been created, upload it using the Browse button next to the File field.
• Upon successful completion of upload, all of your samples will be entered into the database, with all fields completed except for Code, which can be entered either manually or automatically using the button to the right of the Code field.

Registering Data Sets - Using a WebStart Client program

• The Data Set Uploader application provides a GUI for simultaneously uploading many data sets associated with properties (metadata). To start the Data Set Uploader, select Import -> Data Sets

• Data Set List Management
  • In the upper left and right part of the UI, there are buttons to manage the list of data sets to be uploaded. The "+" button adds a data set to the list of those to upload, and the "-" button removes the selected data set from the list. The Upload All button sends data sets to the server for registration.

• List

  • Below the buttons, the specified data sets are listed. Information about each data set is presented in the following columns:

    Column	Description
    Owner	The experiment or sample that owns the data set
    Type	The type of the data set
    Metadata	A summary of the properties of a data set
    Path	The file or folder to upload
    Status/Action	The display of this column changes depending on the context. If the specification of the data set contains errors, an error is shown. If the data set is ready to upload, an upload button is shown. During upload the status of the upload is shown.

• Data Set Information

  • The appearance of this section depends on the data set types that have been configured in openBIS. The first part is always the same and contains the following fields:

    Field	Description
    File	The file or folder to upload
    Owner	The experiment or sample that owns the data (specified using the "Experiment" / "Sample" radio buttons)
    Data Set Type	The type of the data set

  • In the bottom part, a form with fields for specifying the properties of a data set is shown. What is displayed here depends upon the properties that have been assigned to the selected data set type.

• Editing and Uploading Data Sets
  • Upon starting, one data set to upload has already been defined.
  • Fields that are in italics and have an "*" in the name are required and thus must be specified.
  • Fields that are highlighted in red have not been specified correctly.
  • Hovering the mouse over the field will show an error message explaining the problem.
  • Once a data set has been fully specified (i.e., all required fields have been filled and all input passing validation), it can be uploaded by either pressing the Upload button in the corresponding table row, or by pressing the Upload All button.

Uploading Data Sets using CIFEX

• To upload data sets in openBIS select Import -> Data Sets (via CIFEX)
• Data sets can be linked to a sample or directly to an experiment. By default, openBIS will assume that a selected data set is linked to a sample, but this can can be modified by clicking the Connected with Sample checkbox. If the data set you are uploading needs to be linked to an experiment, the box must be unchecked.
• The sample field drop-down menu will be populated with existing samples from which the appropriate one can be chosen.

• Fill out all mandatory fields (marked with an asterisk) and click the Upload Data button to continue.
• After clicking on Upload Data, the CIFEX (CISD File EXchanger) application will open automatically, and you will need to log in.

• Once you are logged in, click the Browse button to locate the file you wish to upload, and click Upload to finish.
• When the data has been successfully uploaded, you will be able to view it in the openBIS Data Sets window by selecting Browse -> Experiments and then choosing your experiment from the list.

Registering Samples and Materials - General Batch Import

General Batch Import makes it possible to simultaneously import samples and materials. There are many situations in which this function is useful. Some examples include the following:

1. Importing a library of assays provided by a third party into openBIS
2. Importing data stored outside of openBIS (e.g., in another database or in MS Excel)

General Batch Import supports the same data format as the Import Materials and Import Samples. Templates for these formats can be obtained from Import -> Samples and Import -> Materials, respectively, as shown in the Sample Registration session above.

When using General Batch Import, the metadata for the samples and materials must be specified in a particular way on a specific sheet of an Excel file. If the format is that for one particular sample type or material type, the sheet should have the name sample-[Sample Type] or material-[Material Type]. If a sheet contains data for multiple sample or material types (the multiple option in the Batch Import form), then the sheet should have the name sample or material.

How to refer to samples in the imported Excel file:

Samples without container, using column "Identifier":

• Define both space and code of the sample. Example: /MY_SPACE/MY_CODE
• Define only code of the sample; example: MY_CODE. Space of the sample is defined to be the DEFAULT_SPACE of the import file. If the default space is not defined, then space is defined to be the home space of the user doing the import.

Samples with container, using only column "Identifier":

• Define space, code and subcode of the sample. Example: /MY_SPACE/CONTAINER_CODE:COMPONENT_CODE. Note: using this syntax means that the component is in same space than the container.

Samples with container, using columns "Identifier" and "CURRENT_CONTAINER":

• Define space and code of the component sample in the "Identifier" column". Define space and code of the container sample in the "CURRENT_CONTAINER" column.
• Define only code of the component sample in the "Identifier" column" Define space and code of the container sample in the "CURRENT_CONTAINER" column. Space of the component sample is defined to be the same than its container's.
• Define only code of the component sample in the "Identifier" column" Define only code of the container sample in the "CURRENT_CONTAINER" column. Space of both container and component sample is defined to be the DEFAULT_SPACE of the import file. If the default space is not defined, then the space is defined to be the home space of the user doing the import.

Note: General batch import does not support automatic generation of sample codes, even if the "Generate Codes Automatically" attribute of the sample type is set.

Managing Controlled Vocabularies

Browsing vocabularies

• To browse controlled vocabularies, select Admin -> Vocabularies.

Listing vocabulary terms

• By selecting any code in the Vocabulary Browser window you can get a complete list of controlled vocabulary for that item either by clicking on the Code, or by selecting the Show Details button at the bottom right of the page.

Adding, updating, or replacing Controlled Vocabulary terms

• To add, update or replace current items in your controlled vocabulary, navigate to the vocabulary browser by selecting Admin -> Vocabularies.
• To edit a particular term, click on the hyperlinked Code item and select Edit Term at the bottom of the page.
• To delete/replace a term, click on the hyperlinked Code item and select Delete/Replace Term at the bottom of the page.
• To add new terms click the Add Terms button at the bottom right side of the page.

• To update terms select the term you would like to modify and click on Update Terms at the bottom of the page. This option also allows you to add several new terms in batch. To do this, generate a TSV file containing all current terms, add the new terms to this file and then upload it.

Ad-hoc Vocabulary Terms

In some cases, all the relevant vocabulary terms are not known a priori. In these cases, managing vocabulary terms in the traditional way becomes an impediment to productivity. For these situations, openBIS supports ad-hoc vocabulary terms. These are vocabulary terms that can defined by any user with SPACE_USER privileges (as opposed to the normal vocabulary terms, which can only be added by users with SPACE_POWER_USER privileges). Ad-hoc vocabulary terms can be created as needed, such as when uploading a data set, and can then be used as otherwise normal vocabulary terms. They are, however, visually marked as being ad-hoc by italics.

Defining Ad-hoc Vocabulary Terms

New ad-hoc vocabulary terms can be defined as needed when creating or editing an existing entity.

A new term can be typed directly into the drop-down list. Clicking on the + icon opens a dialog where the various properties of the vocabulary term can then be defined.

Ad-hoc vocabularies are shown in light gray and italics.

Approving Ad-hoc Vocabulary Terms

Users with SPACE_POWER_USER privileges can accept ad-hoc terms into the official list of vocabulary terms from the vocabulary browser (Admin -> Vocabularies).

This process is called "approval". To approve a term, select it in the vocabulary browser and click the Approve button. Multiple terms may be selected using the command key on the Mac or the control key in Windows or Linux.

Configuration

By default, ad-hoc terms are not supported. To activate ad-hoc terms, an administrator must follow the instructions here: Ad Hoc Vocabulary Terms#Configuration.

Curating Metadata

Managing Attachment Metadata

• Any kind of document can be attached to the Project, Experiment, or Sample entities during registration. This includes, but is not limited to common file types such as .doc, .xls, .pdf, .tsv, and any other files that you may need.

• You can add attachments while you are creating new Projects, Experiments or Samples.
  • New -> Project
  • New -> Experiment
  • New -> Sample

• When you are creating one of the above types, click the Add Attachment link to add attachments.

• You can search for a particular attachment by typing the name or part of the name in the search window at the top right corner of the openBis interface.

Editing

Most entities in openBIS can be easily edited through the interface. To make changes, click on the edit button, which can be found in the top right or bottom right corner of the screen, depending on the view that is being used. Clicking the edit button will cause a new window to appear which displays all editable items.

User roles play an important part in editing, and different entities can only be edited by certain types of users.

Editing of properties

Properties of types Real, Integer, or Varchar can be edited directly in their tables by double clicking individual cells and entering the new value

Batch Updating of Metadata

• Properties of experiments, samples, data sets or materials of one type can be updated in batches by uploading a properly formatted file. This file can also be used to update the parent/child relationships of samples and data sets.
• To batch update experiment properties, select Import -> Experiments Updates and choose an experiment type.
• To batch update sample properties, select Import -> Sample Updates and choose a sample type.
• To batch update data set properties, select Import -> Data Set Metadata Update and choose a data set type.
• To batch update material properties, select Import -> Material Updates and choose a material type.
• Once you have selected your desired entity type, download the template file provided on the Batch Update page.
• Edit the file as required and upload it.

Deleting

OpenBIS allows users to easily delete any entities in the database. Simply click on the delete button located at the bottom right corner of the respective window. A new window will appear, and you will be required to confirm the deletion and provide a reason for the deletion.

Deletion is a privileged operation and only certain users can delete certain items:

Trashcan

If the trashcan is turned on, samples, experiments, data sets, and materials are not immediately removed from the database and the file system when they are deleted in openBIS. Instead, they are moved to the trashcan, which should be familiar to users of most desktop operating systems. One thing, however, that differs between the openBIS trashcan and common operating system trashcans, is that openBIS places restrictions on who can empty the trashcan.

Power users (SPACE_POWER_USER access rights) can delete items and undelete items they deleted, but cannot empty the trashcan. Only different kinds of administrators may empty the trashcan.

What happens when the trashcan is emptied depends upon the user's privileges. Users with SPACE_ADMIN privileges may empty those items in the spaces they administer. Users with INSTANCE_ADMIN privileges can delete all items from the trashcan.

Currently, trashcan deletion is an optional feature of openBIS and must be explicitly turned on. By default, deletion is immediate. To activate trashcan deletion, an administrator must add the following line to the web-client.properties file:

enable-trash = true

Administer and Configure a Server

This section describes steps to configure and administer an openBIS server. All the functions described here require SPACE_ADMIN or INSTANCE_ADMIN privileges to execute.

Registering Master Data

Due to its highly flexible nature, openBIS requires some configuration to specify how it captures information about the core entities (experiments, samples, data sets, and materials), and who is allowed access to different types of information. We call this configuration information master data.

The master data is made up of spaces, projects, and types. Spaces are used to specify access rights while projects are used to group together experiments. The types, experiment types, sample types, data set types, and material types, are used to describe different kinds of experiments, samples, data sets, and materials and to specify what metadata is captured for each of them.

Registering Projects

• To add a new project to openBIS, select New -> Project (Figure 1).
• Alternatively, you can add a new project in the project browser window by clicking the Add Project button at the bottom right corner of the page (Figure 2).
• Make sure to fill out all mandatory fields (indicated with an asterisk).
• The Data space will be set to your home space by default.
  
  
  

Registering Experiment Types, Sample Types, Data Set Types and Material Types

• In order to organize metadata in a flexible way, openBIS has introduced the concept of Experiment Types, Sample Types, Data Set Types and Material Types. At this level, you will assign metadata by using property types.

• A sample type could be DNA, a protein or a multi-titer plate. Each sample type would in turn have specific metadata, such as information about a cell line, or a chemical compound used with a multi-titer plate.

Examples
For a multi-titer plate sample type, you might assign the property types cell line and chemical compound. You could then assign property type values HeLa for the cell line and Gleevec for the compound.

• To add a new experiment type, select Admin -> Types -> Experiment Types
• To add a new sample type, select Admin -> Types -> Sample Types
• To add a new data set type, select Admin -> Types -> Data Set Types
• To add a new material type, select Admin -> Types -> Material Types
• Click the Add button at the bottom right. A new windows will appear where the code and the description of the respective type can be entered.

Curating Master Data

To create and manage Types is recommended to use the new default interface shown at Unified Management of Types and Property Types.

Is also possible to access the legacy master data  interface enabling it for a certain user. To do that, with that user logged in, select at the user settings Enable Legacy Metadata UI, logout and login again to make the changes effective.

Properties and Property Types

Using the properties, you can add any kind of information or descriptions to the Experiment, Sample, Data set and Materials entities.

• To register property types and assign them to entities, you need to have Admin privileges.
• To browse for property types, select Admin -> Metadata -> Browse Property Types.
• To get a list of property types and their assignments, select Admin -> Metadata -> Browse Assignments.
• To add new property types, select Admin -> Metadata -> New Property Type:

• All fields must be completed.
• Codes must be entered manually. The code will always be displayed in capital letters. Spaces are not allowed in Codes.
• The label specifies how the property will be displayed in openBIS.

• Choose the correct data type depending on the kind of data that will be stored in the property type:

  Data Type	Description
  Boolean	Value can be true or false, Yes or No
  Controlled Vocabulary	Choose from the vocabulary available in openBIS or create a new controlled vocabulary for your own purposes
  Hyperlink	Link to another place on the internet
  Integer	Integer number (…, -2, -1, 0, 1, 2, 3, …)
  Material	Choose from one of the materials available in the openBIS system
  Multiline Varchar	Text (variable characters) with newline characters preserved. No length restriction
  Real	Real number such as 2.4871 or -5.0
  Time Stamp	Data and Time
  XML	XML document (variable character). No length restriction. Optional validation against a schema and transformation before rendering.

• Once you have added a property in openBIS, you should assign it to an Experiment Type, Sample Type, Data Set Type, or Material Type.

• Here are some examples for the respective entity types for particular research domains:

  research domain	experiment type	sample type	data set type	material type
  HCS	siRNA screen	Plate	Image Analysis	Gene
  proteomics	MS Injection	Affinity purified biological sample	mzXML	Protein
  Next Gen Sequ	High-Throughput-Sequencing	Flow Cell	FASTQ	Gene

• After assigning the property to an entity type, you may provide a value for it when creating or modifying experiments, samples, data sets, and materials of that entity type.

Assigning new Property Types

It is always possible to augment or reduce the properties that are captured for an entity.

• To assign a new property type to an experiment, select Admin -> Metadata -> Assign To Experiment Type
• To assign a new property type to a sample, data set, or material, proceed as above for the selected entity type

Defining a property type by a controlled vocabulary

• To define a property type, select Admin -> Metadata -> New Property Type.
• Choose Controlled Vocabulary as the data type, and select the desired vocabulary item from the drop-down menu.

Unified Management of Types and Property Types (Available from release 13.04.1)

Under the Admin -> Types panel, for Experiment Types, Sample Types, Data Set Types and Material Types is possible to list entity types.

For example for Sample Types as shown on the image below. From this tab is possible to add, edit and delete a Sample Type, this actions are available using the buttons at the bottom left corner.

When adding or editing a Sample Type a new tab will be presented as shown on the next screenshot, such tab contains all information related to the Type. This is also available for Experiment Types, Data Set Types and Material Types.

From this screen is possible to not only modify the Sample Type fields but also the assigned property types that are shown on a grid. This grid present the options to add, edit and delete property types at the bottom left corner.

When adding a property type to a type a new pop up is presented. From this pop up is possible to assign an existing a property type and also to create a new one that will assigned. This two options are shown on the next two screenshots.

A very important aspect of the unified editing is that changes are not made until the save button is pressed. Until this happens new Property Types are not created or assigned and no fields are updated.

When the save button is pressed a message of feedback should appear and the edit tab should close as shown on the image below.

Find, Summarize, Export

Browsing in openBIS

OpenBIS provides tools for browsing numerous different entities, including projects, experiments, samples and materials. To make this browsing faster and more intuitive, hyperlinks are provided between related entities.

For example, viewing an experiment in the openBIS interface will allow you to see a list of all samples contained within the experiment. By clicking one of these samples, you will be able to see the sample view, including the associated data sets and the samples from which those data sets are derived. You can then click on the name of a data set to see further information about it.

Browsing last-seen experiments, samples, data sets and materials

• Click on the Home icon in the left top corner of the window. This will list the last several browsed items, as well as the time they were last accessed.
• Immediately after logging in to the system, this list will only display the last 20 items viewed.
• During an active session, the list will accommodate a larger number of items. The number of items listed can be configured to suit your preferences.
• After logging in the last 20 items (number is configurable) which were browsed will be listed next to the date. During one session additional sites visited will be listed. However, after logging out only the last 20 items will be saved.

Browsing experiments

• Select Browse -> Experiments. A list of all current projects will be displayed in the left-hand window. By clicking on a project, you will be shown a list of all related experiments in the right-hand window. By default, all experiments will be displayed for the selected project.

Browsing samples

• Select Browse -> Samples. By default, additional user input will be required before a list is generated. Choose a sample type and a data space to view specific information, or choose "all" to see all available samples.

Browsing projects

• In the openBIS user interface, select Browse -> Projects. This will list current projects, brief descriptions, and other relevant details.

Searching

The search box at the top right of your openBIS screen allows you to search openBIS from any view. Simply type in your keyword or phase and press the Search button.

Performing a full text search

• When you perform a search, by default openBIS will search all content across all entities. This includes all properties and attachments throughout the database. The results are displayed in a table containing information about the entity kind and entity type, as well as the matching text and the identifier.

• Properties are searched based on their value, and attachments are searched by comparing keywords to their filenames.

Filtering your search results

• Select the drop-down menu to the left of the search box to filter your search results by entity kind.  You can choose to view only experiments, samples, data sets or materials.

Sample searching

• To enter sample-search mode, choose Browse -> Sample Search

• Sample searching can be refined and made more efficient by providing keywords and data filters such as:
  • Sample Code
  • Sample Type
  • Space Code
  • Any property type in openBIS assigned to Sample

Data set searching

• To enter data-set-search mode, choose Browse -> Data Set Search

• Searching for data sets can be refined and made more efficient by providing keywords and data filters such as:
  • Data Set Code/ or Type
  • File Type
  • Any property type in openBIS assigned to Data Set

• Advanced data set searches can be refined to include any number of criteria. As a typical example, a user can search for data sets where type, project, and experiment all have the same defined value.

Working with Tables

Editing Data in Table Cells

• In tables for experiments, samples, datasets, or materials, most cells can be directly edited by double-clicking on the cell. To ensure responsiveness of the application, the UI is not refreshed immediately. Rather, only those values that are known to have changed are refreshed. Values that could have been affected by an edit are shown in light gray. Clicking the refresh button will force an update of all values from the sever.

Sorting

• Sorting causes all rows to be re-ordered based on the contents of the column chosen by the user.
• Rows are sorted in ascending or descending order by clicking on the column header. To reverse the sort order, click the column header a second time.
• An arrow will appear next to the column header text indicating which column is being used to sort the data. The upward arrow indicates an ascending sort order (A to Z, 1 to 9), while a downward arrow indicates a descending sort order (Z to A, 9 to 1)

Filtering

• To activate filtering, click the Filter button at the bottom of the table window.

• To display a list of standard filters, click on the Settings button at the bottom of the table window. A new window titled "Table Settings" will open, showing the columns that can be filtered. Check the columns that you would like to filter.

• By clicking to the right of a filtered column, a list of up to 50 items will be displayed showing distinct values from the data in that column.  If there are more than 50 possible values, you may need to enter your choice manually.
• Filtering allows you to limit the number of results you see. It does not modify your data. By clicking on the Refresh button at the bottom of the table window, all of your data will become visible again, exactly as it had been before being filtered.

• Combined Filters for Non-Numerical Values
  Users can filter the column data by using the following regular expressions:

  Character	Example	Meaning	Notes
  !	!ECOLI!	without ECOLI	Data will only be displayed if it does not contain the string "ECOLI"
  &	ECOLI & BSUBTILIS	ECOLI and BSUBTILIS	Data will only be displayed if it contains both strings
  SPACE	ECOLI BSUBTILIS	ECOLI _or _BSUBTILIS	Data will be displayed if it contains either of the strings
  ^	*^*FK	FKBP1A, FKBP1B, FKBP2, FKBP3, .... (starts with)	Data must begin with the selected string
  $	R1$	CR1, EGFR1, FGFR1, GPR1, TNFR1, .... (ends with)	Data must end with the selected string
  ^...$	^C1$	C1 (exact match)	Data must exactly match the string

• Numerical Filters
  The following numerical comparison operations are supported:
  (<, >, <=, >=, =, !=) with a number immediately after those characters (e.g. <10)

• Combined Numerical Filters
  You can filter the content of a column by combining a numerical value contained in the column with the following regex characters:

  Character	Example	Meaning	Notes
  &	>3 & <15	>3 and <15	Both conditions must be met
  SPACE	>99 <50	>90 or <50	Either condition or both conditions must be met
  !=	!=0	without 0	Will match every value except the specified one

• Date Filters
  The following comparison operations on dates and timestamps are supported:
  (<, >, <=, >=, =, !=) with a date immediately after those characters (e.g. <2011-07-23). The day and the month of specified date can be dropped (e.g. >=2009 or =2010-11). Date filters can be combined with an ampersand to specify a range. Here are some examples:

  Example	Meaning
  >2010-06-07 & <2010-07	Every day in June 2010 after the 7th
  >2009 & <2011	Every day in 2010
  =2010	Every day in 2010
  >=2010 & <=2011-06	Every day in 2010 and in the first half of 2011

Customizing Table Views

• Table views can be customized in a number of ways to create personal settings.  Users can choose to hide columns, show them, or change their sequence by clicking the Settings button at the bottom of the table window.
• A new window titled "Table Settings" will open, displaying a list of available columns. By checking or unchecking individual columns, they can be shown or hidden in the table view.  By highlighting a column and choosing "Move Up" or "Move Down", the sequence of the columns can be edited. For added convenience, columns can also be rearranged by drag & drop.
• Column widths can be adjusted by holding down the SHIFT key while clicking on the column header.
• Users can also visualize a table as a graph if an appropriate DSS plug-in has been installed.  Choose between the table view and the graphical view by clicking the pull-down menu in the upper right corner of the table window.

Custom Columns and Filters

• Administrators and Power Users are able to create custom table columns and filters using Python expressions.
• Custom columns and filters can be public or private. Those specified Private can only be seen or viewed by their creator, while those labeled Public can be seen and used by all openBIS users.
• Columns and filters are Python expressions that evaluate to a boolean value. The rule is that a filter expression that evaluates to true for a given table row will be kept. Otherwise, it will be filtered out.
• To add a new custom column or filter, click the Settings button at the bottom of the table window. A new window titled "Table Settings" will open, allowing new filters to be created. Existing custom filters can be chosen by clicking their respective tabs at the top of the window.
• By clicking Add Column or Add Filter, a new window appears, allowing you to provide a name, description, and an expression. In this window, you can also choose whether the custom filter or column will be private or public by checking the appropriate box.

Exporting Table Content

• Table content can be exported by clicking on the Export button at the bottom of the table window. The exported text file can be opened with Microsoft Excel or compatible data file viewers. Your browser settings may affect the way Exported files are handled.

Downloading of data using CIFEX

• To download selected data using CIFEX, first select the data you wish to download using the Browse -> Data Set Search tool.
• To change search criteria, click the appropriate button in the top right corner of the window to add or remove search criteria.
• Once you have found the data you require, click the Export Data button in the bottom right corner of the view.

• You will be redirected to CIFEX (the CISD File EXchanger) and you will then need to provide you password to continue.
• Choose a name for your output file and press OK to begin the export process.
  
  

• Once the exported data is ready to be downloaded, you will receive an email containing a link to your new CIFEX file.

Advanced Features

Uploading Data using Dropboxes

Dropboxes are directories located either on a local file system or on a file server that are monitored for write activity by the data store server (DSS). If the DSS finds new data in a dropbox, it will execute the ‘Extract Transform Load’ (ETL) routine configured for this dropbox. The ETL routine will perform all steps that are necessary to make the new dataset known to openBIS, link it with the appropriate entities (e.g. samples), extract metadata required for searching and transform it to a pre-defined format that may be needed to facilitate querying or integration with other tools.

This feature is described in detail in the Administators Guide and Dropbox Guide.

Uploading Data using a remote API

The remote API is a set of Java classes that provides a streaming-based programming model to upload any set of files which constitute a dataset. Compared to dropboxes, it allows for a tighter integration of the application using it, can provide immediate feedback (as it runs synchronously) and supports upload over any WAN that allows for HTTPS traffic without additional network configuration. As the use of Java as a programming environment can be limiting in some cases, openBIS also contains a command line tool for data upload that uses the remote API internally. This tool has been used to integrate the openBIS upload with e.g. LabView- or script-based data providers.

Javadoc for the API is available at http://svnsis.ethz.ch/doc/openbis/13.04.0/overview-summary.html. If you are interested in using the API, contact your openBIS administrator for more information and examples.

Querying Data

Custom Queries

• OpenBIS gives users the ability to perform sophisticated custom database queries in addition to those in the default setup. Privileged users can create, store, and execute these queries with the core database, or with any other relational database server that utilizes Structured Query Language (SQL). There are two ways to run these queries:
  
  • Any user in an observer role can run predefined SQL queries, and has the ability to provide search arguments in the case of parameterized queries.
  • Any user proficient with SQL SELECT statements make custom or arbitrary SQL requests.

Private and Public Queries

• OpenBIS allows users to query multiple databases for web-based applications. Additionally, these queries can be made private or public.
  • Private queries are only visible to their creators.
  • Public queries can be seen and executed by all users of the system.

Results from custom queries are shown in a tabular view and can be interactively filtered and exported as Excel-formatted documents.

Project-Specific features

Special Queries - Protein Viewer for PhosphoNetX SyBIT project

1. Select Queries -> All Proteins of an Experiment
2. Choose the desired experiment
3. Select the different values for the given values (False Discovery Rate & Aggregate Function).
   
4. OpenBIS will display a list of proteins, including protein name, accession numbers, and abundance values from the appropriate biological samples. A typical result may say "0% plasma, replicate 1".
   
5. Click on the protein name hyperlink to see a detailed view of an identified protein, including a list of samples in which it has been identified.

Data Upload - YeastX SyBIT project

1. Put all data files into one directory.
2. Specify which file belongs to which sample in the additional index.tsv file, which must be located in the same directory.
3. Copy this data directory into the openBIS system's "incoming" folder, typically located at //nas-biol-imsb-1/share-5-$/openbis/incoming.

Creating the index.tsv file

1. Download the template file from the Data -> Upload page
2. Open the template file in Excel. When you see the "Text Import Wizard" interface, simply click the "Finish" button and continue.
3. The downloaded file should have the appropriate structure, which looks like this
   • your-email@imsb.biol.ethz.ch
   • file_name sample experiment project space conversion datasetcomments
4. Modify the file in the following ways:
5. The first line should contain your email address preceded by the # sign
6. It is important that this email be correct, since it will receive notification of any mistakes in the index.tsv file
7. Fill in the given fields keeping in mind the descriptions below:

• The example index.txv file should look something like this:

  # your-email@imsb.biol.ethz.ch
  file_name			sample	        experiment	project		space		conversion	datasetcomments
  glc1.mzXML			s32	        				IMSB_ZAMBONI	fiaML		my first dataset
  P1_-4f_-Glc_all_2ul_1.eicML	glucose 1					IMSB_ZAMBONI			this is quite interesting
  glc2.mzXML			glucose 2	My experiment	bacteria	IMSB_ZAMBONI	eicML		double-check if the result is really correct

For more details, please click here.

Creating a Unified Report from more than one Dataset - AGRON-OMICS project

1. Select Browse -> Experiments
2. Choose the experiment that you would like to display
3. Select all data sets from the Data Set window and then select Compute -> Queries

1. This will trigger a data store task which fetches all of the selected data sets from the data store.
2. Additionally, this will concatenate all tabular files of the fetched datasets by creating a unified report. This report can be exported as an Excel-formatted spreadsheet.