Page tree
Skip to end of metadata
Go to start of metadata

Solaris 10

Creating a new Zone

bash-3.00# zonecfg -z zone1
imsb-lims: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:zone1> create
zonecfg:zone1> set zonepath=/home/zones/zone1
zonecfg:zone1> set autoboot=false
zonecfg:zone1> add net
zonecfg:zone1:net> set address=
zonecfg:zone1:net> set physical=aggr2
zonecfg:zone1:net> end
zonecfg:zone1> add fs
zonecfg:zone1:fs> set dir=/data
zonecfg:zone1:fs> set special=/data/openbis
zonecfg:zone1:fs> set type=lofs
zonecfg:zone1:fs> add options [nodevices]
zonecfg:zone1:fs> end
zonecfg:zone1> exit

bash-3.00# zoneadm -z zone1 verify
bash-3.00# zoneadm -z zone1 install

bash-3.00# zoneadm -z zone1 boot

bash-3.00# zlogin -e \; -C zone1          # The -C option to zlogin specifies the target zone. The -e option to zlogin specifies the escape character.
Ie: to escape - do ;.

After login, complete the normal Solaris install. (Answer the questions)

To create a read/write sudirectory in a read-only filesystem:

Example: In order that the zones can write to /opt/oracle/9.2.0/dbs

In zone

mkdir /oracledbs

mount -F lofs /oracledbs /opt/oracle/9.2.0/dbs

Put the following entry in /etc/vfstab:

/oracledbs      -       /opt/oracle/9.2.0/dbs   lofs    -       yes     -

In order that a zone can write to an existing directory.
Example: being able to change config files in /opt/oracle/9.2.0/Apache

In global zone

mv /opt/oracle/9.2.0/Apache /opt/oracle/9.2.0/xApache

mkdir /opt/oracle/9.2.0/Apache	# make a mountpoint only

In zone

mkdir /oraApache

mount -F lofs /oraApache /opt/oracle/9.2.0/Apache

cp -rp /opt/oracle/9.2.0/xApache /opt/oracle/9.2.0/Apache

Another problem. /home in the zone is a mount point, so you can't write directly

So we create yet another mount point

In the zone.

mkdir /realhomes

In /etc/vfstab:

/realhomes      -       /home                   lofs    -       yes     -

Seeing the global zone name from a zone

One quick method we use very successfully is to create a readonly lofs to /etc/nodename.
We add the following to all our zonecfgs

add fs
set dir=/etc/GLOBAL
set special=/etc/nodename
set type=lofs
add options [ro, nodevices]

so when your in a ngz you can cat /etc/GLOBAL to get the global host name.

Creating ZFS filesystems

The devices are hardware mirrored, so there isn't a huge advantage here, but Balatocan has 2 zfs filesystems created as:

zpool create -m /home homePool c3t0d0s7
zpool create -m /data dataPool c3t2d0s0

zfs create dataPool/openbis
zfs set mountpoint=/export/openbis dataPool/openbis
zfs set quota=4000G dataPool/openbis
zfs set sharenfs=on dataPool/openbis
zfs set dataPool/openbis

Gromit has:

zpool create -m /home homePool c5t0d0s7

These persist across reboots.

Network configuration files

/etc/hostname.<interfaceName> - defines hostname/IP address/netmask/up or down
/etc/hosts - defines hostname
/etc/inet/netmasks - netmasks
/etc/inet/ntp.conf - ntp

Link aggregation

The interfaces need to be unplumbed:

/etc # ifconfig e1000g3 unplumb 
/etc # ifconfig e1000g2 unplumb

Create the aggregation:
/etc # dladm create-aggr -d e1000g2 -d e1000g3 2
To make persistant, edit /etc/hostname.<interface>:

Jumbo frames

Changed sometime between snv_111 and snv_118.
You no longer enable jumbo frames in /kernel/drv/<driver>.conf, (except for the ixgb driver) you use dladm

dladm set-linkprop -p mtu=9194 <interface>

ixgb driver

In /kernel/drv/ixgb.conf add:

default_mtu = 9000;

Pre snv_118
To configure jumbo frames, you need to edit /kernel/drv/e1000g.conf and change MaxFramesize to 3 MaxFrameSize=3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3;.
Caution: This file may get overwritten by a live upgrade.
In /etc/hostname.aggr? add mtu 16128 and reboot.
Within the ETHZ network, leave the mtu at the default of 1500

On an x4540 with a 10Gbit ethernet card it's:
/kernel/drv/nxge.conf - add:

name = "pciex108e,abcd" parent = "/pci@79,0/pci10de,377@a" unit-address = "0"
accept_jumbo = 0;
name = "pciex108e,abcd" parent = "/pci@79,0/pci10de,377@a" unit-address = "0,1"
accept_jumbo = 1;

No need to put anything in /etc/hostname.nxge[01], they default to 1500 and 9194 respectively

Network configuration for the zones

The boxes that have 2 network interfaces, are configured with the main machine on the CISD server net ( having a valid IP address so we can maintain it. The other interface is given a non-valid IP, but the interface must be up. Therefore, the interface files look like:




Routing in a zone, where the zone does not have sufficient privileges to add a route, needs special consideration.
If a zone is bound to the aggr2 interface, with an IP address of say, the global zone does not know how to route for this network, and therefore nor does the zone.
To get around this problem, the zone is set to not autoboot, then a little start script on the global zone /etc/init.d/zone_route is called to:

  1. add to aggr2
  2. add a default route via
  3. remove from aggr2
  4. boot the zone
  5. In /etc/rc3.d a symlink (S99<zonename>) is created /etc/init.d/zone_route (The legacy way of starting a service)

Firewall Administration

Solaris uses ipfilter by Darren Reed. See
At CISD, firewall configuration files are under RCS control. see man rcs
Rules are in /etc/ipf/ipf.conf
Making changes:
Check out the file:

/etc/ipf # co -l ipf.conf

Edit /etc/ipf/ipf.conf with you favorite editor.
Check the file back in. Example:

/etc/ipf # ci -u -wjohn ipf.conf
RCS/ipf.conf,v  <--  ipf.conf
new revision: 1.9; previous revision: 1.8
enter log message, terminated with single '.' or end of file:
>> Added some firewall rule that blocks ferrets
>> .

At CISD, there is a script called ./ in /etc/ipf on each machine with firewall rules. It does the following for you:

  • # at now "+ 2min" < otherwise you may get stuck without a network connection. ( just does ipf -Fa)
  • Restarts the rules with ipf -Fa -f ipf.conf (where -Fa deletes all the old rules before loading the new ones).
  • Lists all current rules with ipfstat -io and ipnat -l

    To list all currently set firewall and NAT rules, do:

    # ipfstat -io
    # ipnat -l

    Firewall for NFS. There is a workaround here

Stopping/starting a service

Services are controlled with the command svcadm. To find out what a service name is use svcs.
Here's an example of restarting sendmail:

biosa# svcs -a|grep mail
online         13:42:51 svc:/network/smtp:sendmail
biosa# svcadm restart svc:/network/smtp

Finding out what service (PID) binds to a port

The most portable way to do it is

# lsof -i

Works on Linux, Solaris and the BSDs.

pkginfo gives a list of installed packages. Add the -l <pkg> for more infomation

~ # pkginfo
application SUNWzfsgr                        ZFS Administration for Sun Java(TM) Web Console (Root)
application SUNWzfsgu                        ZFS Administration for Sun Java(TM) Web Console (Usr)
system      SUNWzfskr                        ZFS Kernel (Root)
~ # pkginfo -l SUNWzfsgu                                          
   PKGINST:  SUNWzfsgu
      NAME:  ZFS Administration for Sun Java(TM) Web Console (Usr)
  CATEGORY:  application
      ARCH:  i386
   VERSION:  1.0,REV=2007.
   BASEDIR:  /
    VENDOR:  Sun Microsystems, Inc.
      DESC:  This package contains the user (/usr) component of the ZFS administration application for the Sun Java(TM) Web Console
    PSTAMP:  vx86-lx50e-blr0320070312085929
  INSTDATE:  Jun 27 2007 18:49
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:      310 installed pathnames
                   4 shared pathnames
                  31 directories
                3063 blocks used (approx)

installing a new package, eg from Please put downloaded packages in /usr/local/src

/usr/local/src # pkgadd -d  openssl-0.9.8e-sol10-x86-local

The following packages are available:
  1  SMCossl     openssl
                 (x86) 0.9.8e

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [CISDIT:?,??,q]: 

"default" - just hit return
Removing a package

/usr/local/src # pkgrm SUNWzfsgu
## Verifying package <SUNWzfsgu> dependencies in zone <biosa>

The following package is currently installed:
   SUNWzfsgu  ZFS Administration for Sun Java(TM) Web Console (Usr)
              (i386) 1.0,REV=2007.

Do you want to remove this package? [CISDIT:y,n,?,q] n

Performing a Live Upgrade

Sun release a new version of Sun Developer Express edition every 3 months or so. To upgrade without having to take the machine down, you can use Live Update.
You need another partition the same size as the existing root partition. Here, I've used the swap partition.

  1. Download the latest "Solaris Express Developer Edition (sxde)" DVD
  2. Use a swapfile temporarily, by creating a swap file with:

     # mkfile 1g /var/swapfile
     # swap -a /var/swapfile
     # swap -l

    Format the old swap partition with 'format' and change the flag to wm from wu

  1. Remove the swap entry from /etc/vfstab
  2. You'll need to reboot or find another way to remount /tmp and /var/run
  3. I gave the new partitions the name sxde907 as in the first link

I hope this link still works for you:
This one is a more complete/complex explanation titled "Upgrading With Solaris Live Upgrade"

Multi path administration

A Solaris box with 2 FC HBAs will require failover. In the following we configure LUN 68 for failover with mpathadm.

  1. Look at the output from format: only relevant lines shown
bash-3.00# format
Searching for disks...

c3t5006048452A50667d68: configured with capacity of 505.75GB
c4t5006048452A50668d68: configured with capacity of 505.75GB


 1. c3t5006048452A50667d68 <EMC-SYMMETRIX-5771 cyl 552418 alt 2 hd 15 sec 128>

 4. c4t5006048452A50668d68 <EMC-SYMMETRIX-5771 cyl 552418 alt 2 hd 15 sec 128>}}

Specify disk (enter its number): *1*
Disk not labeled.  Label it now? *n*
format> *format*
Ready to format.  Formatting cannot be interrupted
and takes 7673 minutes (estimated). Continue? *yes*
Beginning format. The current time is Thu Sep 27 08:55:37 2007

Verifying media...
pass 0 - pattern = 0xc6dec6de

The EMC Symmetrix needs to be added to /kernel/drv/scsi_vhci.conf
Do a format -> enquiry and you'll get something like:

format> inquiry
Vendor:   EMC     
Product:  SYMMETRIX       
Revision: 5771

Now edit /kernel/drv/scsi_vhci.conf and add:

device-type-scsi-options-list =
"EMC     SYMMETRIX", "symmetric-option";
symmetric-option = 0x1000000;


Now format looks completely different - you'll only see half the number of disks. This is good, they are multipathed

~ # format          
Searching for disks...
Mode sense page(3) reports nsect value as 128, adjusting it to 128

c6t60060480000290100249533033343032d0: configured with capacity of 505.75GB
c6t60060480000290100249533033343345d0: configured with capacity of 1011.51GB

       0. c5t0d0 <DEFAULT cyl 8872 alt 2 hd 255 sec 63>
       1. c6t60060480000290100249533033343032d0 <EMC-SYMMETRIX-5771 cyl 552418 alt 2 hd 15 sec 128>
       2. c6t60060480000290100249533033343230d0 <EMC-SYMMETRIX-5771 cyl 1679 alt 2 hd 255 sec 126>
       3. c6t60060480000290100249533033343345d0 <EMC-SYMMETRIX-5771 cyl 1104838 alt 2 hd 15 sec 128>
Specify disk (enter its number):

The disk that is formatted for Solaris is disk 2.


bash-3.00# luxadm display /dev/rdsk/c3t5006048452A50667d68s2 (s2 is whole disk)

Now run:

~ # mpathadm list lu
                Total Path Count: 2
                Operational Path Count: 2
                Total Path Count: 2
                Operational Path Count: 2
                Total Path Count: 2
                Operational Path Count: 2

We want /dev/rdsk/c6t60060480000290100249533033343230d0s2 in this case.

Now run:

~ # mpathadm show lu /dev/rdsk/c6t60060480000290100249533033343230d0s2
Logical Unit:  /dev/rdsk/c6t60060480000290100249533033343230d0s2
        Vendor:  EMC     
        Product:  SYMMETRIX       
        Revision:  5771
        Name Type:  unknown type
        Name:  60060480000290100249533033343230
        Asymmetric:  no
        Current Load Balance:  round-robin
        Logical Unit Group ID:  NA
        Auto Failback:  on
        Auto Probing:  NA

                Initiator Port Name:  10000000c95d3b14
                Target Port Name:  5006048452a50667
                Override Path:  NA
                Path State:  OK
                Disabled:  no

                Initiator Port Name:  10000000c95d34f0
                Target Port Name:  5006048452a50668
                Override Path:  NA
                Path State:  OK
                Disabled:  no

        Target Ports:
                Name:  5006048452a50667
                Relative ID:  0

                Name:  5006048452a50668
                Relative ID:  0

Now continue following:

Set Font Path

If you get the following or similar error (I had it on my fresh installed VMWare)

Warning: Cannot convert string "-monotype-arial-regular-r-normal--*-140-*-*-p-*-iso8859-1" to type FontStruct
Warning: Unable to load any usable ISO8859-1 font
    Class: XmRendition
    Conversion failed.  Cannot load font.

You have to set the font path. I found the solution in the VMWare Forum:

xset fp+ /usr/openwin/lib/X11/fonts/TTbitmaps

Enable SSL for apache2

SSL is disabled by default.
The following is an example of how to enable it for the Blastwave apache2, but the same is true of the built-in apache2

-bash-3.00# svcprop -p httpd/ssl svc:/network/http:cswapache2
-bash-3.00# svccfg -s svc:/network/http:cswapache2 setprop httpd/ssl=true
-bash-3.00# svcadm refresh svc:/network/http:cswapache2
-bash-3.00# svcprop -p httpd/ssl svc:/network/http:cswapache2

X11 forwarding not working on your Solaris 10 box?

This is because sshd is trying to use the ipv6 localhost (::1) by default which we don't have enabled.
The error message given in /var/adm/messages is:
Aug 23 11:25:09 cisd-egmont sshd[2685]: [ID 800047 auth.error] error: Failed to allocate internet-domain X11 display socket.

A quick fix is to run:
ifconfig lo0 inet6 plumb up
and to ensure the change persists across reboots:
touch /etc/hostname6.lo0

On x86 Solaris, systems with patch 126134-04 exhibit this symptom - systems with 126134-02 do not.

How to set cron's default PATH

Edit /etc/default/cron to look like:


Restart and refresh cron

~# svcadm disable svc:/system/cron:default
~# svcadm refresh svc:/system/cron:default
~# svcadm enable svc:/system/cron:default

The SUPATH is for root's crontab

Sharing a zfs filesystem

Open port 2049/tcp for the clients that need access
On server:

root@cisd-krakatoa # zfs set mountpoint=/export dataPool
root@cisd-krakatoa # zfs create dataPool/imsb-openbis
root@cisd-krakatoa # zfs set quota=4000G dataPool/imsb-openbis
root@cisd-krakatoa # zfs set sharenfs=on dataPool/imsb-openbis
root@cisd-krakatoa # zfs set dataPool/imsb-openbis
root@cisd-krakatoa # zfs get sharenfs

Another example:
root@bs-ssvr01 # zfs set sharenfs='rw=@,rw=@,' dataPool/bsse/bsse-it

On client:

root@cisd-ruapehu # mount -o proto=tcp,public cisd-krakatoa:/export/imsb-openbis /mnt/krakatoa


# echo "/misc           auto_misc       -nosuid,nobrowse" >> /etc/auto_master
# echo "ruapehu -fstype=nfs,rw,proto=tcp        thumper2:/export/imsb-openbis" >> /etc/auto_misc
# svcadm disable autofs
# svcadm enable autofs

Here's the performance:

/mnt/krakatoa/data $ time dd if=/dev/zero of=1G bs=1024k count=1000
1000+0 records in
1000+0 records out

real    0m11.008s
user    0m0.005s
sys     0m2.626s
/mnt/krakatoa/data $ time dd if=/dev/zero of=10G bs=1024k count=10000
10000+0 records in
10000+0 records out

real    1m36.347s
user    0m0.053s
sys     0m27.586s

Rsync setup using rsync from

On the master thumper, add the following to /etc/inetd.conf

rsync   stream  tcp     nowait  root    /opt/csw/bin/rsync      rsyncd --daemon

and the following to /etc/services

rsync           873/tcp                         # Rsync daemon

Then run

~# inetconv 

to put it under control of the SMF

If you get this error:

# inetconv
inetconv: Error /etc/inet/inetd.conf line 44 invalid or inconsistent fields: service-name protocol

Check you have the correct entry in /etc/services

Set up /etc/rsyncd.conf to look like:

log file = /var/log/rsyncd.log

comment = Source of files from master thumper
hosts allow = thumper2
read only = yes

Enable rsyncd using the SMF

# svcadm enable svc:/network/inetd
# svcadm enable svc:/network/rsync/tcp

Check SMF entry

# svcprop -p inetconv/source_line rsync/tcp
rsync\ stream\ tcp\ nowait\ root\ /opt/csw/bin/rsync\ rsyncd\ --daemon

From the slave thumper, do


$RSYNC -av --delete --rsync-path=$RSYNC thumper1::imsb-openbis  /export/imsb-openbis/

Snapshot and save a filesystem

# export DATE=`/bin/date +%Y-%m-%d_%H%M`
# zfs snapshot dataPool/imsb-openbis@$DATE
# zfs list  -t snapshot
NAME                                    USED  AVAIL  REFER  MOUNTPOINT
dataPool/imsb-openbis@2008-01-23_1055      0      -  13.3G  -
# zfs send dataPool/imsb-openbis@2008-01-23_1055 > /misc/ruapehu/snapshot.$DATE

Then on remote host (ruapehu)

# cat snapshot.2008-01-23_1055 | zfs recv dataPool/test3@today # creates a new zfs filesystem at /dataPool/test3

How to find out the memory model of a Solaris system


~# isainfo -b

The output will be 64 or 32, depending on the memory model of the system.

Measure network throughput

# kstat -p 'link:0:aggr1:rbytes64' 1

Add a persistent route.

If you have 2 interfaces on the same network, but want to route packets to a particular host via a particular interface.

       Host A                        Host B
 _____________________         ____________________
| nxge1 | ----> | nge1 |
|                     |       |____________________|             Host C
|                     |                                      _____________________     
| aggr0 | --------------------------------- > | nxge1 |
|_____________________|                                     |_____________________|
~ # route -p add  -interface
add persistent host gateway
~ # route -p add  -interface
add persistent host gateway

View persistent routes

cat /etc/inet/static_routes
# File generated by route(1M) - do not edit. -interface -interface
  • No labels