Frequently Asked Questions

ASRU is ASM Storage Management Reclaimation utility that allows storage and database administrators to compact the ASM disk group and mark unused storage blocks with zeros so that storage arrays can reclaim the marked physical blocks. This is very useful when ASM disks are thin provisioned LUNs

Database (4)

Oracle database engine calculates and sets this parameter automatically and it is not recommended to adjust this parameter until it is set to a wrong value or suggested by Oracle Support. Various Oracle RDBMS kernel functions use this parameter for various calculations especially Oracle CBO uses this parameter if CPU costing is enabled. If you intentionally increase value of this parameter to a value different from thel # of CPUs calculated by Oracle database engine, then you are inviting performance implications on that database.

ALL_ROWS is the default optimizer mode in Oracle database release 10g and 11g.

For example, if you want to set event 10046 at level 12 for the current process then login to the database as SYSDBA and execute the following commands:

SQL> oradebug setmypid
Statement processed.
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> oradebug event 10046 trace name context off;
Statement processed.
SQL> oradebug tracefile_name
/u01/oracle/products/diag/rdbms/prod/PROD/trace/PROD_ora_31497.trc
SQL> exit

Execute the following statement on the SQL*Plus prompt before executing the INSERT statement.

SET DEFINE OFF

When spooling output of a SQL query returning more than 50000 rows, Oracle automatically add another header section in the spool file after 50000 rows. How can I get rid of this extra header section?

This is hard limit set by the SQL*Plus client but you can try the following workaround. Add following commands into the start of your db script.

set pages 0
set newpage none
set emb on

Execute the following SQL statement against the target database as sysdba:

select decode(instr(metadata,'B023'),0,'64bit Database','32bit Database') "DB Creation" from kopm$;

If we want to enable trace for a running session from a different session, dbms_support package can be handy. Execute the DBMS_SUPPORT package as shown below:

exec dbms_support.start_trace_in_session(sid,serial#,true,true); 

Execute the following SQL statement from the target database as SYSDBA.

select ksppinm, ksppstvl from x$ksppi a, x$ksppsv b where a.indx=b.indx and lower(ksppinm) like ‘_%’;

 

Oracle provide the dynamic performance views that can be used to monitor the progress of parallel queries. Please refer the the article How to monitor progress of running parallel query in Oracle database?

Check the oratab file, which contains list of the databases with the actual physical path to the Oracle Home. The oratab file is located under /etc directory on linux machines and under /var/opt/oracle directory on other major UNIX machines. You can also check the inventory.xml file under oraInventory/ContentsXML folder if the inventory is correct and valid. If you want to identify the Oracle database instances running on the machine then run the 'ps -ef | grep pmon' command on the shell command prompt on the machine.

oracle@node1:~ ps -ef | grep pmon 
oracle 24645 1 0 Jul29 ? 00:00:21 ora_pmon_ORCL1 
oracle 10250 1588 0 12:34 pts/1 00:00:00 grep pmon oracle@node11:~ 

In the above sample, ORCL1 is the name of the database instance running on linux machine node1. If there is only ORCL1 database is installed on machine node1 then oratab file should look like:

ORCL1:/u01/app/oracle/product/db_1:N

YES, you can run multiple Oracle databases in same Oracle Real Application Cluster. You can also run multiple database instances of different cluster databases on same cluster node.

SCN is 6 bytes long so the maximum value is equivalent to 2^48.
One should not need to worry about running out of SCN numbers even if you were to commit 16K transactions per second, it would take more than 500 years to run out of SCN numbers.

The amount of redo data generated by a batch process or data load varies upon different factors such as DML operation is not running in nologging mode, FORCE or SUPPLMENETAL logging is enabled in the database etc.
So to estimate how much redo data, a data load or batch process can generate, one can calculate the redo data generated by executing the single transaction manually and record the ‘redo size’ statistics from V$MYSTAT view and multiply the value with the expected number of transactions in the data load or batch process. Execute the following query to display value of redo size from V$MYSTAT view.

SELECT value FROM v$mystat a, v$statname b WHERE a.statistic# = b.statistic# AND b.name=redo size

 

DBA stands for Data Block Address and RDBA stands for relative data block address.

Lot of users complain that they can not connect to an Oracle database using JDBC connection string when using database service name in it whereas they can connect to the database using service name when connecting to the database using SQLPLUS. There is nothing wrong with the underlying technology but the syntax for using the JDBC connection string is different when using the service name in JDBC connection string. Below are two java code examples having JDBC connection string with the database name and service name.

JDBC Connection String with database name Connection conn = DriverManager.getConnection ("jdbc:oracle:thin:@dbhost:1521:sid_name", "dbusername", "dbpassword"); 
JDBC Connection String with service name Connection conn = DriverManager.getConnection ("jdbc:oracle:thin:@dbhost:1521/service_name", "dbusername", "dbpassword"); 

Prior to Oracle database release 9.2, there is no direct way to identify this information BUT Oracle create directories something like cpu\CPUOct2005\* under the ORACLE_HOME, the patch was applied. So by looking at the directories, one can have idea about the CPU applied on to the Oracle database.

Well from CPUJan2006 onward, if you are running Oracle database release 9.2 or higher then you can execute the following SQL statement against the target database to identify the CPU patches applied to the database.

col action_time for a30 
col action for a6 
col namespace for a9 
col version for a10 
col id for 999999999 
col comments for a10
select * from registry$history;

 

First of all, never set these underscore parameters until suggested by Oracle Support. You MUST rebuild the database if you set these parameters to startup the database because of the following reasons.

  1. By setting these parameters, you are telling Oracle kernel that the rollback segments are corrupted and any uncommitted transactions in it will not be rolled back as part of the crash recovery at the next startup.
  2. The data dictionary will be logically corrupt after bypassing the consistency checks (Result of setting the above said parameter) of the Oracle kernel. Hence, it is VERY VERY important that you MUST rebuild the database to avoid any unforeseen problem later on this database.

 

Execute the following statement on the SQL*Plus prompt before executing the INSERT statement.

SET DEFINE OFF

 

YES, Oracle database server generates the redo data even when you insert data with APPEND SQL hint BUT Oracle database server generates minimal redo data to only record and invalidate the database blocks being impacted by the nologging operations. Needless to say that data inserted without logging is not recoverable.

ALL_ROWS is the default optimizer mode in Oracle database release 10g and 11g.

Oracle database engine calculates and sets this parameter automatically and it is not recommended to adjust this parameter until it is set to a wrong value or suggested by Oracle Support. Various Oracle RDBMS kernel functions use this parameter for various calculations especially Oracle CBO uses this parameter if CPU costing is enabled. If you intentionally increase value of this parameter to a value different from thel # of CPUs calculated by Oracle database engine, then you are inviting performance implications on that database.

For example, if you want to set event 10046 at level 12 for the current process then login to the database as SYSDBA and execute the following commands:

SQL> oradebug setmypid
Statement processed.
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> oradebug event 10046 trace name context off;
Statement processed.
SQL> oradebug tracefile_name
/u01/oracle/products/diag/rdbms/prod/PROD/trace/PROD_ora_31497.trc
SQL> exit

 

For example, if you want to set event 10046 at level 12 for the current process then login to the database as SYSDBA and execute the following commands:

SQL> oradebug setmypid
Statement processed.
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> oradebug event 10046 trace name context off;
Statement processed.
SQL> oradebug tracefile_name
/u01/oracle/products/diag/rdbms/prod/PROD/trace/PROD_ora_31497.trc
SQL> exit

You can use the 'objdump' command on the linux system to identify the address of currently attached SGA as shown below:

objdump -t $ORACLE_HOME/bin/oracle | grep sgabeg
20000000 l *ABS* 00000000 sgabeg

So the first field of the output of the above command is the memory address in HEX where the SGA is attached or will be attached.

 

Oracle documentation has not mentioned such methods but the following command will allow you to start the data guard broker and getting the prompt back without exiting the data guard broker:

dgmgrl sys/password@connectIdentifier "START OBSERVER FILE=oemdbobs.dat;" &

The above command will start the observer and give the prompt back without exiting the observer.

NOTE you can also use logfile option for the dgmgrl command to create a logfile.

 

NO, it is one of the pre-requisites of the Data Guard Configuration that FORCE logging must be enabled on the primary database. If you disable the FORCE logging on the primary database then you can have corrupted data in the standby databases. Please refer Oracle Support note 290161.1

Logical standby will automatically synchronize with the primary database. Please note that the foreign archived logs that arrived between the Point-in-time recovery target time and the current time will have to be re-fetched by the logical standby again, although they may have already existed on-disk at the logical standby database.

Follow the steps given below to complete the switchover:

  • Shutdown all the database instances on your primary site except one database instance.
  • Shutdown all the database instances on physical standby site except the apply instance. In case you are using single instance physical standby site then that will be your apply instance. You should have only one database instance running on each cluster before the switchover.
  • Execute the following statement on SQL PLUS on the existing primary database to covert the primary database to standby role..
		SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO STANDBY WITH SESSION SHUTDOWN;
  • Issue the following statement on the standby site to switch the standby database to the primary database.
		SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY;
  • After successful completion of previous step, execute the following statements on old primary database.
		SQL> SHUTDOWN IMMEDIATE;
		SQL> STARTUP MOUNT; 
  • After successful completion of the step 5, execute the following statement to open the new primary database.
		SQL> ALTER DATABASE OPEN;

 

MTTF(mean-time-to-failure) is the average time that a IT system runs without failing after it has been setup or repaired, whereas MTTR (mean-time-to-repair) is the average time needed to repair a failed IT system.

Execute the following command to report the complete version of the installed rpm:

rpm -qa --queryformat "%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH}) "

It is bit tricky when there are so many files in a directory on UNIX/LINUX system and getting 'Too Many Arguments' is expected. There are different ways to move files to another location rather than directly using the mv command. You can try the following commands:

find source-directory -type f -name *.* -maxdepth 1 -exec mv {} target-directory/ \;

The above command will find all files from the source-directory and then move them into target-directory.

The other quick way to find and move the files is first redirect the output of find command to a shell script then split the shell script and execute the scripts in parallel as shown below

find source-directory -type f -name *.* -maxdepth 1 -exec echo mv {} target-directory/ \; > move-script.sh

In above command we are redirecting the output of find command along with the actual mv command syntax into a move-script.sh shell script. lets suppose we have million files in source-directory hence move-script will have a million rows and we split the move-script.sh further so that each script will have 200000 rows in it.

split -l 200000 move-script.sh

The split command will split the move-script.sh in number of files containing 200000 rows in it and the default names for the split files may be xaa, xab, xac etc. Now give these split files execute permission and execute them with nohup as shown below:

nohup xaa > xaa.log 2>&1
nohup xab > xab.log 2>&1 
etc

 

Execute the following command:

find /yourDir -group groupName -type f -mtime +15 -exec rm -f {} \;

Login to the Oracle Management Server and execute the following command:

emcli create_role -name=NEW_ROLE_NAME

 

Login to the Oracle Management Server and execute the following command:

emcli create_user -name="new_user" -password="new_user_password" -email="first.last@yourdomian.com" -privilege="VIEW_TARGET;host1.us.oracle.com:host"</pre>
<p> </p>"	
											 
											       																			 
											 

 

Connect to the Oracle Management Repository (OMR) database as SYSMAN user and execute the following APIs.

SQL> exec emd_maintenance.remove_em_dbms_jobs;
SQL> exec emd_maintenance.submit_em_dbms_jobs;

Navigate to $OMS_HOME/bin directory and execute the following command:

emctl secure unlock

 

Please refer to article "Deploying Applications on Oracle Real Application Cluster" under Knowledge Base section of ORAXperts.

There are warning messages becasue Oracle could not identify the Cluster Interconnect in the cluster database. Either you have not set the database parameetr cluster_interconnects or nothing is set in the OCR. Easiest way to resolve this is to set the value of the database parameter cluster_interconnects to the actual interconnect IP. Other option is to use 'oifcfg' command to set the physcial interconnect in the Oracle Clusterware(CRS).

Oracle's OPatch internal check for node reach ability fails if there is any kind of login banner is enabled or being displayed upon 'oracle' user login. As a workaround you can set 'LogLevel QUIET' in ~/.ssh/config file on all cluster nodes within the cluster to suppress the banner. Make sure you revoke the changes in ~/.ssh/config file after patching.

Oracle CRS records the third party software information in the ocssd.log logfile under the $ORA_CRS_HOME/log/$HOSTNMAE/cssd directory. Search for text 'skgxn' in ocssd.log file and you will see third party software information there. When Oracle Clusterware is installed on top of third party clusterware, Oracle CRS set the value of css miss count to 600 so this could be another symptom that is evident when Oracle Clusterware is installed on top of third party cluster software such as Sun Cluster, Veritas Cluster etc.

Oracle has released comprehensive documentation about Oracle products including Oracle Real Application Cluster. You can access Oracle's official documentation atOTN. There are few books available in the market on Oracle Real Application Cluster. I personally like this book.

You must set FULL duplex for NIC cards regardless of Oracle Real Application Cluster or not because in HALF duplex mode NIC can not send and recieve network packets at the same time.

Yes, both 9i and 10g cluster databases can co-exist in the same cluster. Becasue Oracle Clusterware is mandatory for the 10g RAC database so when you install Oracle Clusterware it will convert the exisitng 9i RAC srvconfig file to Oracle 10g OCR format and after that both 9i and 10g cluster database will use the OCR file. There is no need to run 9i gsd once Oracle Clusterware is installed.

NO, Oracle almost require same amount of storgae that is required in single instance database BUT it must be shared storage accessible from all cluster nodes. The extra shared storage is required for UNDO tablespace (because in Oracle RAC each cluster database instance uses its own UNDO tablespace) and cluster files such as OCR, Voting etc.

The Oracle software binaries for cluster database uses RAC option, which is not enabled for single database Oracle software binaries but RAC option can be enabled in the single database oracle home by relinking the Oracle software binaries as shown below:

Login as Oracle software owner then change directory to $ORACLE_HOME/rdbms/lib and execut

make -f ins_rdbms.mk rac_on

and finally if the previous commaned executed successfully execute the following command.

make -f ins_rdbms.mk ioracle

Please make sure you shutdown the database instances running from the ORACLE_HOME you are relinking.

 

You can identify the interconnect being used by executing the ORADEBUG IPC command as shown below

Login as sysdba on the database server

SQL> show parameter user_dump_dest

ORADEBUG will write the trace file at this location.

SQL> oradebug setmypid 
SQL> oradebug ipc

The above command will write IPC information in trace file under user_dump_destination. The ipc information will look like.

SSKGXPT 0x1a2392c flags SSKGXPT_READPENDING info for network 0  socket no 10 IP 172.16.195.1 UDP 43759 sflags SSKGXPT_WRITESSKGXPT_UP  info for network 1 socket no 0 IP 0.0.0.0 UDP 0... 

To change the interconnect protocol you need to shut down all Oracle databases and relink the binaries using the following command:

make -f ins_rdbms.mk rac_on ipc_udp ioracl

The alert log can be verified to check whether the right interconnect is used. The following is reported in the alert log if the UDP is used as interconnect protocol

cluster interconnect IPC version:Oracle UDP/IPIPC Vendor 1 proto 2 Version 1.0 

 

It is not suggested and supported to change the properties and behavior of cluster resources starting with ora* in Oracle Real Application Cluster environment. But if you want to try this on application VIP then follow the steps given below:

  • Get the name of the application VIP resource you want to update. eg ebr_vip and stop the resource.
  • create the profile using command

crs_stat -p ebr_vip > $HOME/ebr_vip.ca

  • Update the value of HOSTING_MEMBERS parameter in the .cap file. Most probably it will have node1 in the value.
  • Include node2 in the HOSTING_MEMBERS parameter separated by , eg. HOSTIMG_MEMBERS=node1,node2
  • Update parameter PLACEMENT=favored in the .cap file.
  • Execute command
crs_register ebr_vip -dir -u

The above command will update the resource and next time VIP will failover to the node you have configured in HOSTING_MEMBERS.

 

The best approach to use redundant NICs for interconnect is to bind them using available technology such as IPMP, teaming etc. You can configure failover using these technologies as well. Configure one private IP on these redundant NIC and define that IP in CLUSTER_INTERCONNETS parameter. This approach will allow to utilize the bandwidth of both physical interfaces as well as survive the failure of one physical interface also. If you are setting CLUSTER_INTERCONNECTS parameter without binding the physical interfaces then CLUSTER_INTERCONNECTS initialization parameter requires the IP address . It enables you to specify multiple IP addresses, separated by colons. Oracle RAC network traffic is distributed between the specified IP addresses. In this case no failover or failback is available. It only distribute the traffic among interconnects listed in CLUSTER_INTERCONNECTS.

The objective of having separate software owner is to separate and restrict the privileges based on the job role. For example in big organizations, sysadmin looks after the CRS (because ideally it is a clustering solution rather than a database solution) and should not have access to the Oracle database or Oracle ASM software. Same theory with Oracle ASM software, if you want to separate the duties and privileges of Oracle DBA and storage administrator (managing the storage (ASM) of the database as well), you should create a separate group and software owner for Oracle ASM. By default 'dba' is the OSASM group and Oracle database software owner 'oracle' will have all the privileges to ASM instance, which storage administrator may not like to share because he is responsible for this role. By using separate software owner or groups, you will have liberty to add Oracle database software owner as a member for other groups such as asmdba any time in future and Oracle DBA can access the other components as well.

Grid Computing is a concept that can be implemented using the technology framework provided by Oracle Real Application Cluster 10g, Oracle 10g Services and Oracle ASM. It depends how you implement this to achieve the maximum benefits of the Oracle Grid Computing environment. If you have configured a cluster database with at least two instances running on servers 'A' and 'B' then you can configure Oracle 10g services for different scenarios as given below:

  1. You can manually relocate the service to server 'B' if service is configured to run on 'B' as well
  2. You can configure server side threshold on the services. This is called workload management. Read Oracle Net Service Administrator guide available on OTN for detail information on Oracle 10g services.

If you are keen to learn more about Oracle Grid Computing then read articles about Project MegaGrid at http://www.dell.com/content/topics/global.aspx/alliances/en/megagrid?c=u...

 

Mapping of the database instance to a particular cluster node is defined in the Oracle cluster resource of instance type. Depending upon the preferred node for the resource, CRS start that resource on that respective node. To extract detailed definition of crs resource, you can use

crs_stat -p resource name > $HOME/resourcename.profile

You can list the resources defined by using crs_stat command and resources having 'inst' in their name are the database instance type resources. Below is sample of ORCL1 cluster database instance resource from node1:

NAME=ora.ORCL.ORCL1.inst 
TYPE=application 
ACTION_SCRIPT=/u01/app/oracle/product/10.2.0/db_1/bin/racgwrap 
ACTIVE_PLACEMENT=0 
AUTO_START=1 
CHECK_INTERVAL=600
 DESCRIPTION=CRS application for Instance 
FAILOVER_DELAY=0
 FAILURE_INTERVAL=0 
FAILURE_THRESHOLD=0
 HOSTING_MEMBERS=node1 ....................truncated rest of the text

In above sample output, ORCL1 database instance is meant to run on the node listed in HOSTING_MEMBERS hence mapped to node1. You can also use the SRVCTL to update the Oracle Cluster Registry (OCR) file to tell Oracle Clusterware about the location of Oracle database instance and the ORACLE_HOME for the database instance as shown below.

srvctl add instance -d -i -n

When you modify the location of the Oracle database instances within the cluster, you use the SRVCTL to update the Oracle Cluster Registry hence the underlying Oracle Cluster Resource.

 

There are mainly two hardware components involved while talking about Oracle cluster private interconnect. One is Network Interface Card on the cluster node and the Network Switch (Typically Gigabit Switch) connecting all the cluster nodes within the cluster. You can identify the private network interface card on the machine by running the

oifcfg iflist

command. Once you know the private interconnect interface, ethtool command on linux can extract detail information about interface such as speed, duplex mode etc. On Solaris, you can use any of the following command.

netstat -k 
ndd -get link_speed 
ndd -get link_mode

As I said above that there are two hardware components involved in cluster interconnect. To know the specification of Network Switch, you may need to contact your network administrator.

 

As a super user run the 'ifconfig -a' command on the machine where the network interface card is installed and look at the output of the above command. if you see MTU as 9000 then interface is jumbo frame enabled. In the sample output given below, NIC eth6 is jumbo frame enabled because MTU is set at 9000.

eth6 Link encap:Ethernet HWaddr 00:18:FE:2E:70:05 UP BROADCAST  RUNNING NOARP SLAVE MULTICAST MTU:9000 Metric:1 RX packets:5596 errors:0  dropped:0 overruns:0 frame:0 TX packets:26 errors:0 dropped:0  overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:360280 (351.8  Kb) TX bytes:3296 (3.2 Kb) Base address:0x7000 Memory:fdee0000-fdf00000

 

The best way to recreate the database is using the DBCA. In case you are doing it manually then it depends on situation:

Case 1: The ORACLE_HOME and name of the database being created is same as of the database being dropped

  1. Drop the database (don't delete the init.ora file as you can reuse this if everything is going to be same for the new database)
  2. Recreate the database or contolfile for the new database.

Case 2: The ORACLE_HOME and name of the database being created is different as of the database being dropped

  1. Drop the database
  2. Remove the database from the Oracle Cluster Registry using SRVCTL command.
  3. Recreate the database or contolfile for the new database.
  4. Register the database in Oracle Cluster Registry using SRVCTL

 

Oracle introduced Oracle managed VIP in Oracle Clusterware in Oracle 10g onwards but there has not been any limitation in Oracle 9i RAC to use VIP managed by third party cluster software on UNIX based platforms. Please refer to Oracle 9i Real Application Cluster - How to avoid database users waiting for TCP timeout error when TCP/IP network goes down? article.

NO, user equivalence is not required for normal functioning of Oracle Real Application Clusters BUT must be configured for the installation of Oracle software and patches in the Oracle Real Application Cluster environment.

Without going into much details of the steps, the overall simple approach if you can afford downtime would be:
 

  1. Configure the shared storage to place the Oracle data files and Oracle Cluster files such as OCR and Voting disk.
  2. Install the Oracle Clusterware on all the cluster nodes
  3. Install the Oracle RDBMS software
  4. Create one UNDO tablespace for each target database instance in the existing single instance database.
  5. Configure the cluster database related parameters such as CLUSTER_DATABASE, UNDO_TABLESPACE, REMOTE_LISTENER, THREAD and INSTANCE_NUMBER in the target database initialization parameter file.
  6. Shutdown the single database instance and move the data files to the shared storage
  7. Recreate the control file with the value of MAXINSTANCES parameter grater than the number of actual database instances in the cluster.
  8. Start the first database instance in the new cluster environment and execute the catclust.sql from $ORACLE_HOME/rdbms/admin as sysdba.
  9. Create one redo thread for each database instance in the cluster database and set the instance specific parameter THREAD accordingly for each database instance.
  10. If you want to manage the cluster database instances through Oracle Clusterware, add the cluster database including all the cluster database instances in the OCR using SRVCTL command. Please refer to Oracle Clusterware Quick How Guide under Library section OR you can access the guide directly here

 

OCR is Oracle Cluster Registry file that Oracle Clusterware use to keep and record the information about the registered Oracle Cluster Resources.

For example when DBA adds a new cluster database/database instance in the Oracle Real Application Cluster using the SRVCTL command, SRVCTL commands internally create and records the cluster resource for the database/database instance.

Regardless of third party cluster software, you will have to install the Oracle Clusterware to build the Oracle 10g Real Application Cluster database. Oracle 10g does not require any third party cluster software until there is another application process on the same database server that requires third party cluster software to fail over. In some case you may require third party cluster software if you want to use the cluster file system to place the Oracle data files BUT Oracle do provide the Automatic Storage Management using the Oracle ASM, with no extra cost and very easy to use and manage so you would rarely require third party cluster software if you are using the Oracle 10g Real Application Cluster for the database server only.
Oracle Database 10g Release 2, Oracle Clusterware can be used to manage application processes in the cluster SO why to spend extra money on third party cluster when Oracle is providing everything you need for a cluster database with no extra cost?

NO, all the cluster nodes must be binary compatible and running on same operating system BUT cluster nodes with different sizes are allowed.

As a minimum requirement, you would require one Network Interface Card (NIC) for your public network and one NIC for private network between the cluster hosts. Typically in production environments to achieve the high availability and to meet the SLA, it is recommended to install redundant NICs for public and private network.

You should always use ocfs2console to configure the ocfs2 and O2CB cluster service. Sometime it has been observed that even improper indentation in the cluster.conf file when configured manually, can cause this error.

Executing the following command :- $CRS_HOME/bin/cemutlo -n should display the cluster information.

Yes, technically there should not be any problem to run Oracle 10g RAC database with Oracle 11g Clusterware because Oracle Clusterware will be on highest level in this case.

here is no limit on number of Oracle databases, you can run in a single cluster. You can run as many Oracle databases as you want including the cluster databases in a single cluster as long as OS resources such as CPU, Network bandwidth and memory are available on the machines in the cluster.

White paper "Oracle Homes in an Oracle Real Application Clusters Environment" released by Oracle address this query in details. You can access the white paper here.

Oracle CRS aka Clusterware does not provide any pre configured cluster resources and wrappers that can be used to provide the cold fail over of single instance databases BUT the Oracle Clusterware framework itself allows users to add their own cluster resources to achieve this behavior.

Oracle CRS aka Clusterware does not provide any pre configured cluster resources and wrappers that can be used to provide the cold fail over of single instance databases BUT the Oracle Clusterware framework itself allows users to add their own cluster resources to achieve this behavior.

Please read this article for more information.

Ensure that none of the login script such as bashrc, profile etc are executing mesg operating system command during the login process of operating system account being used for installing Oracle software.

Remove or comment the 'mesg' command from the login scripts.

 

These are cluster control files stored under scls_scr/HOSTNAME/root directory on any Oracle Cluster node. Oracle uses these files to store the run time data for Oracle Cluster such as boot time information in the cssrun and cssfboot files.

Similarly Oracle uses other control files to store different information. Oracle stores the following files under scls_scr/HOSTNAME/root directory:

clsomonpid
crsstart
cssrun
noclsm on
oprocdpid
crsdboot
cssfboot
daemonpid
nocl svmon

 

Oracle Real Application Clusters (RAC) is a database option to create cluster database with a shared cache architecture that overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide highly scalable and available database solutions for all your business applications.

Nope, definitely this is not a bug as this is expected behaviour from Oracle Grid Infrastructure 11.2.0.2 onwards. Oracle has introduced feature to provide redundancy for the private interconnect and this is implementation of this feature. This extra IP is called Highly Available IP. Please refer to this post here.

Starting with Oracle Grid Infrastructure 11.2.0.2, Oracle provide redundancy for private interconnect by using High Available IP cluster resources, which manage these extra private virtual IPs. Oracle can configure four private virtual IPs on a cluster server. This post may give you more information on this.

Oracle has improved node fencing in Oracle Grid Infrastructure 11g Release 2 (11.2.0.2) and onwards and try to kill the processes on the failed node that are capable of doing I/O and restart the Oracle Clusterware rather than rebooting the failed node.

Oracle will reboot the failed node only if it is not able to kill the processes and restart the Oracle Clusterware on the failed node.