Data Store Share Use Cases
Segmented Storage and Shares
The store of a Data Store Server (DSS) can be distributed among several storage devices. This segmentation of the store allows to add new storage when needed.
Each segment of the store is called a share. Shares are identified by numbers. They correspond to directories in the store directory specified by property
service.properties of DSS. In case that the store and the share are on different storage devices the share directory will be a symbolic link to the storage. This assumes that all storage is mount on the machine running DSS.
The incoming directory of a drop-box has to be on a storage device which also contains a share. On start-up DSS finds for each incoming directory the corresponding share. It scans all sub-folders of the store and picks the first one to whom it can move a file from incoming directory. During data set registration all data sets will be moved into the share found by this scan. Such shares are called incoming shares. All other shares are called extension shares.
Initially distributed storage isn't needed because there will be not much data. This situation is characterized as follows:
- There are one or more drop boxes. Their incoming directories are all on the same disk (say
- The store is also on
- There is one share (share 1) which is a sub-folder of the store (not a symbolic link). This sub-folder is created automatically. This is the incoming share. All data sets will be stored in share 1.
Use Case: Filling up disk space
This is a very typical use case where a segmented store is needed:
- Data sets are gradually filling up disk space.
- Before the disk is full additional disk space has to be installed.
This use case can be handled as follows:
- Add the new storage and mount it (say at
- Create in the store folder a symbolic link (named '2') to the new share.
- Add to
Each time disk space become low the first two steps (with incremented share number) should be done.