Below we will discuss
1) How to create NFS filesystem and use it for Database files:
2) Configure DNFS client filesystem and use it for Database files bypassing Kernel NFS system.
1) How to create NFS filesystem client and use it for Database files:
Create a new NFS filesystem
Below I will creating a new NFS file system named /nfs
[oracle@collabn1 ~]$ df -Ph
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_collabn1-lv_root 31G 20G 11G 66% /
tmpfs 1.8G 758M 1.1G 43% /dev/shm
/dev/sda1 477M 66M 382M 15% /boot
stage 465G 280G 186G 61% /media/sf_stage
/dev/asm/shared-162 1.0G 79M 946M 8% /shared
[root@collabn1 ~]# cd /
[root@collabn1 /]# mkdir localfs nfs
Add the below lines in the fstab
[root@collabn1 ~]# vi /etc/fstab
# create an NFS mount on nfs which points back to localfs
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac
Add the below line to the exports file
[root@collabn1 ~]# cat /etc/exports
/nfs *(rw,no_root_squash,sync)
stop/start the nfs service:
[root@collabn1 /]# chkconfig nfs on
[root@collabn1 /]# service nfs stop
Shutting down NFS daemon: [ OK ]
Shutting down NFS mountd: [ OK ]
Shutting down NFS quotas: [ OK ]
Shutting down NFS services: [ OK ]
Shutting down RPC idmapd: [ OK ]
[root@collabn1 /]# service nfs start
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS mountd: [ OK ]
Starting NFS daemon: [ OK ]
Starting RPC idmapd: [ OK ]
[root@collabn1 /]# mount /nfs
[root@collabn1 /]# df -Ph
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_collabn1-lv_root 31G 20G 11G 66% /
tmpfs 1.8G 758M 1.1G 43% /dev/shm
/dev/sda1 477M 66M 382M 15% /boot
stage 465G 280G 186G 61% /media/sf_stage
/dev/asm/shared-162 1.0G 79M 946M 8% /shared
localhost:/localfs 31G 20G 11G 66% /nfs
huray….. /nfs is mounted and available for use.
Create and Manage a tablespace that uses NFS mounted file system file
[root@collabn1 /]# cat /etc/fstab | grep nfs
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac
SHAIKDB>create tablespace nfstbs datafile '/nfs/nfstbs01.dbf' size 100m;
Tablespace created.
SHAIKDB>
SHAIKDB>!ls -lrt /nfs
total 102412
-rw-r----- 1 oracle dba 104865792 Sep 12 19:56 nfstbs01.dbf
2) Configure DNFS client filesystem and use it for Database files bypassing Kernel NFS system.
With Oracle Database 11g release 2 (11.2), instead of using the operating system kernel NFS client, you can configure Oracle Database to access NFS V3 servers directly using an Oracle internal Direct NFS client.
To enable Oracle Database to use Direct NFS, the NFS file systems must be mounted and available over regular NFS mounts before you start installation. Direct NFS manages settings after installation. You should still set the kernel mount options as a backup, but for normal operation, Direct NFS will manage NFS mounts.
WARNING:
The DNFS Guide says to enable the init.ora param filesystemio_options to enable direct I/O support.
Doing this, all DB access to all files will be via DIO.
There is no way to only enable direct IO for certain files and exclude others.
Configure Direct NFS Client (DNFS)
SHAIKDB>select * from v$dnfs_servers;
no rows selected
SHAIKDB>show parameter filesystem
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
filesystemio_options string none
i) Configure oranfstab file
Direct NFS Client can use a new configuration file or the mount tab file (/etc/mtab on Linux) to determine the mount point settings for NFS storage devices.
When the oranfstab file is placed in $ORACLE_HOME/dbs, the entries in the file are specific to a single database. When the oranfstab file is placed in /etc, then it is globally available to all Oracle databases
Direct NFS searches for mount entries in the following order:
1. $ORACLE_HOME/dbs/oranfstab
2. /etc/oranfstab
3. /etc/mtab
Here I will let Oracle use the /etc/mtab instead of using oranfstab file:
Enable the Direct NFS Client ODM Library
[root@collabn1 /]# cat /etc/mtab | grep nfs
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
nfsd /proc/fs/nfsd nfsd rw 0 0
localhost:/localfs /nfs nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,vers=3,timeo=600,noac,addr=127.0.0.1 0 0
cd $ORACLE_HOME/lib
mv libodm11.so libodm11.so_bak
ln -s libnfsodm11.so libodm11.so
[oracle@collabn1 lib]$ ls -lrt libodm11.*
-rw-r--r-- 1 oracle oinstall 7442 Aug 14 2009 libodm11.a
lrwxrwxrwx 1 oracle oinstall 12 Aug 1 17:53 libodm11.so_bak -> libodmd11.so
lrwxrwxrwx 1 oracle oinstall 14 Sep 12 21:12 libodm11.so -> libnfsodm11.so
On 11.2 you can enable it with the following make command
[oracle@collabn1 lib]$ cd $ORACLE_HOME/rdbms/lib
[oracle@collabn1 lib]$ make -f ins_rdbms.mk dnfs_on
rm -f /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libodm11.so; cp /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libnfsodm11.so /u01/app/oracle/product/11.2.0.2/SHAIKPROD/lib/libodm11.so
Re-start the Database instance.
SHAIKDB>shut abort
ORACLE instance shut down.
SHAIKDB>startup
ORACLE instance started.
Total System Global Area 521936896 bytes
Fixed Size 2214936 bytes
Variable Size 494928872 bytes
Database Buffers 20971520 bytes
Redo Buffers 3821568 bytes
Database mounted.
Database opened.
View the alert_log:
[oracle@collabn1 trace]$ tail -f alert_SHAIKDB.log
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0
Sat Sep 12 21:17:46 2015
-
-
--
ALTER DATABASE OPEN
Direct NFS: attempting to mount /localfs on filer localhost defined in mtab
Direct NFS: channel config is:
channel id [0] local [] path [localhost]
Direct NFS: mount complete dir /localfs on localhost mntport 34962 nfsport 2049
Direct NFS: NFS3ERR 1 Not owner. path localhost mntport 34962 nfsport 2049
Direct NFS: NFS3ERR 1 Not owner. path localhost mntport 34962 nfsport 2049
SHAIKDB>select ts.name tsname,df.name dfname,bigfile,status from v$tablespace ts,v$datafile df where df.ts#=ts.ts#;
TSNAME DFNAME BIG STATUS
---------- ---------------------------------------- --- -------
SYSTEM /u01/app/oracle/SHAIKDB/system01.dbf NO SYSTEM
SYSAUX /u01/app/oracle/SHAIKDB/sysaux01.dbf NO ONLINE
UNDOTBS1 /u01/app/oracle/SHAIKDB/undotbs01.dbf NO ONLINE
USERS /u01/app/oracle/SHAIKDB/users01.dbf NO ONLINE
NFSTBS /nfs/nfstbs01.dbf NO ONLINE
SHAIKDB>col svrname for a10
SHAIKDB>col dirname for a10
SHAIKDB>select * from v$dnfs_servers;
ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX
---------- ---------- ---------- ---------- ---------- ---------- ----------
1 localhost /localfs 34962 2049 0 0
SHAIKDB>select * from v$dnfs_files;
no rows selected
SHAIKDB>create tablespace DNFS datafile '/localfs/dnfs01.dbf' size 100m;
Tablespace created.
SHAIKDB>select * from v$dnfs_files;
no rows selected
SHAIKDB>create table dnfs (col1 number) tablespace dnfs;
Table created.
SHAIKDB>begin
2 for i in 1..10000 loop
3 insert into dnfs values (i);
4 commit;
5 end loop;
6 end;
7 /
PL/SQL procedure successfully completed.
SHAIKDB>select count(*) from dnfs;
COUNT(*)
----------
10000
SHAIKDB>select ts.name tsname,df.name dfname,bigfile,status from v$tablespace ts,v$datafile df where df.ts#=ts.ts#;
TSNAME DFNAME BIG STATUS
---------- ---------------------------------------- --- -------
SYSTEM /u01/app/oracle/SHAIKDB/system01.dbf NO SYSTEM
SYSAUX /u01/app/oracle/SHAIKDB/sysaux01.dbf NO ONLINE
UNDOTBS1 /u01/app/oracle/SHAIKDB/undotbs01.dbf NO ONLINE
USERS /u01/app/oracle/SHAIKDB/users01.dbf NO ONLINE
NFSTBS /nfs/nfstbs01.dbf NO ONLINE
DNFS /localfs/dnfs01.dbf NO ONLINE
6 rows selected.
More info DNFS:
Step by Step - Configure Direct NFS Client (DNFS) on Linux (11g) (Doc ID 762374.1)
http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CWLIN276