Use DCLI to check #em12c agents on Exadata – all at once

For all the benefits of Oracle Enterprise Manager 12c (OEM), there is one draw back when using OEM with Real Application Clusters or Exadata…. checking the status of the EM Agent on each node.  When a node is bounced, some times the EM Agent does not always restart.  When this happens on a single instance box or a RAC; normally you would have to check  each node individually and restart the agent.  With Exadata, Oracle has made this check easier by providing the Distributed Command Line Interface (DCLI).  Using DCLI, an administrator can run commands across all the nodes of the Exadata.

Note: The DCLI is a shell script that allows commands to be ran on multiple machines with a single invocation.

The example I’m going to use in this post is an example of checking the Enterprise Management Agents after rebooting the  computer (db) nodes of an Exadata.

After rebooting each of the nodes, I needed to validate that the EM Agents were successfully restarted.  All the agents were installed on each of the nodes in the same directory structure (/u01/app/oracle/product/agent12c), which makes using DCLI a breeze.   Now that I know where the agents are installed, I need to create a file that lists all the compute (db) nodes in the Exadata; this file name is dbs_group.  The last thing I need is a login for each of the nodes.  On all Exadata systems the Oracle user can be used to login to each of the nodes.

Now that I have all the information I need; I need two command line switches that will be used with the DCLI command.  These switches are –l (login) and –g (group). 

The command that will be used to check all the EM Agents on the Exadata is:

 

dcli -l oracle -g ~/dbs_group /u01/app/oracle/product/agent12c/core/12.1.0.3.0/bin/emctl status agent


Once I run the DCLI command, I get output for all the EM Agents across all the nodes of the Exadata.

[e###db01:oracle:F####1] /home/oracle
> dcli -l oracle -g ~/dbs_group /u01/app/oracle/product/agent12c/core/12.1.0.3.0/bin/emctl status agent
e###db01: Oracle Enterprise Manager Cloud Control 12c Release 3
e###db01: Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
e###db01: —————————————————————
e###db01: Agent Version     : 12.1.0.3.0
e###db01: OMS Version       : 12.1.0.3.0
e###db01: Protocol Version  : 12.1.0.1.0
e###db01: Agent Home        : /u01/app/oracle/product/agent12c/agent_inst
e###db01: Agent Binaries    : /u01/app/oracle/product/agent12c/core/12.1.0.3.0
e###db01: Agent Process ID  : 6638
e###db01: Parent Process ID : 6307
e###db01: Agent URL         :
https://e###db01.acme.com:3873/emd/main/
e###db01: Repository URL    : https://c#####01.acme.com:4904/empbs/upload
e###db01: Started at        : 2014-03-16 02:29:10
e###db01: Started by user   : oracle
e###db01: Last Reload       : (none)
e###db01: Last successful upload                       : 2014-03-17 10:15:41
e###db01: Last attempted upload                        : 2014-03-17 10:15:41
e###db01: Total Megabytes of XML files uploaded so far : 3.84
e###db01: Number of XML files pending upload           : 0
e###db01: Size of XML files pending upload(MB)         : 0
e###db01: Available disk space on upload filesystem    : 29.98%
e###db01: Collection Status                            : Collections enabled
e###db01: Heartbeat Status                             : Ok
e###db01: Last attempted heartbeat to OMS              : 2014-03-17 10:16:56
e###db01: Last successful heartbeat to OMS             : 2014-03-17 10:16:56
e###db01: Next scheduled heartbeat to OMS              : 2014-03-17 10:17:56
e###db01:
e###db01: —————————————————————

e###db01: Agent is Running and Ready
e###db02: Oracle Enterprise Manager Cloud Control 12c Release 3
e###db02: Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
e###db02: —————————————————————
e###db02: Agent Version     : 12.1.0.3.0
e###db02: OMS Version       : 12.1.0.3.0
e###db02: Protocol Version  : 12.1.0.1.0
e###db02: Agent Home        : /u01/app/oracle/product/agent12c/agent_inst
e###db02: Agent Binaries    : /u01/app/oracle/product/agent12c/core/12.1.0.3.0
e###db02: Agent Process ID  : 28588
e###db02: Parent Process ID : 28478
e###db02: Agent URL         :
https://e###db02.acme.com:3873/emd/main/
e###db02: Repository URL    : https://c#####01.acme.com:4904/empbs/upload
e###db02: Started at        : 2014-03-16 02:24:59
e###db02: Started by user   : oracle
e###db02: Last Reload       : (none)
e###db02: Last successful upload                       : 2014-03-17 10:15:40
e###db02: Last attempted upload                        : 2014-03-17 10:15:40
e###db02: Total Megabytes of XML files uploaded so far : 3.18
e###db02: Number of XML files pending upload           : 0
e###db02: Size of XML files pending upload(MB)         : 0
e###db02: Available disk space on upload filesystem    : 38.92%
e###db02: Collection Status                            : Collections enabled
e###db02: Heartbeat Status                             : Ok
e###db02: Last attempted heartbeat to OMS              : 2014-03-17 10:16:59
e###db02: Last successful heartbeat to OMS             : 2014-03-17 10:16:59
e###db02: Next scheduled heartbeat to OMS              : 2014-03-17 10:17:59
e###db02:
e###db02: —————————————————————

e###db02: Agent is Running and Ready
e###db03: Oracle Enterprise Manager Cloud Control 12c Release 3
e###db03: Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
e###db03: —————————————————————
e###db03: Agent Version     : 12.1.0.3.0
e###db03: OMS Version       : 12.1.0.3.0
e###db03: Protocol Version  : 12.1.0.1.0
e###db03: Agent Home        : /u01/app/oracle/product/agent12c/agent_inst
e###db03: Agent Binaries    : /u01/app/oracle/product/agent12c/core/12.1.0.3.0
e###db03: Agent Process ID  : 5834
e###db03: Parent Process ID : 5733
e###db03: Agent URL         :
https://e###db03.acme.com:3873/emd/main/
e###db03: Repository URL    : https://c#####01.acme.com:4904/empbs/upload
e###db03: Started at        : 2014-03-16 02:42:35
e###db03: Started by user   : oracle
e###db03: Last Reload       : (none)
e###db03: Last successful upload                       : 2014-03-17 10:12:39
e###db03: Last attempted upload                        : 2014-03-17 10:12:39
e###db03: Total Megabytes of XML files uploaded so far : 8.54
e###db03: Number of XML files pending upload           : 0
e###db03: Size of XML files pending upload(MB)         : 0
e###db03: Available disk space on upload filesystem    : 39.97%
e###db03: Collection Status                            : Collections enabled
e###db03: Heartbeat Status                             : Ok
e###db03: Last attempted heartbeat to OMS              : 2014-03-17 10:16:38
e###db03: Last successful heartbeat to OMS             : 2014-03-17 10:16:38
e###db03: Next scheduled heartbeat to OMS              : 2014-03-17 10:17:38
e###db03:
e###db03: —————————————————————

e###db03: Agent is Running and Ready
e###db04: Oracle Enterprise Manager Cloud Control 12c Release 3
e###db04: Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
e###db04: —————————————————————
e###db04: Agent Version     : 12.1.0.3.0
e###db04: OMS Version       : 12.1.0.3.0
e###db04: Protocol Version  : 12.1.0.1.0
e###db04: Agent Home        : /u01/app/oracle/product/agent12c/agent_inst
e###db04: Agent Binaries    : /u01/app/oracle/product/agent12c/core/12.1.0.3.0
e###db04: Agent Process ID  : 5853
e###db04:
Parent Process ID : 5734
e###db04: Agent URL         :
https://e###db04.acme.com:3873/emd/main/
e###db04: Repository URL    : https://c#####01.acme.com:4904/empbs/upload
e###db04: Started at        : 2014-03-16 02:53:08
e###db04: Started by user   : oracle
e###db04: Last Reload       : (none)
e###db04: Last successful upload                  : 2014-03-17 10:16:33
e###db04: Last attempted upload                   : 2014-03-17 10:16:33
e###db04: Total Megabytes of XML files uploaded so far : 2.81
e###db04: Number of XML files pending upload           : 0
e###db04: Size of XML files pending upload(MB)         : 0
e###db04: Available disk space on upload filesystem    : 45.75%
e###db04: Collection Status                         : Collections enabled
e###db04: Heartbeat Status                             : Ok
e###db04: Last attempted heartbeat to OMS           : 2014-03-17 10:16:50
e###db04: Last successful heartbeat to OMS          : 2014-03-17 10:16:50
e###db04: Next scheduled heartbeat to OMS           : 2014-03-17 10:17:50
e###db04:
e###db04: —————————————————————

e###db04: Agent is Running and Ready

 

From the output,  I can clearly see that all the EM Agents are monitoring the Exadata  and are in running status.  I really like the DCLI command option, makes life simple especially when needing to check on multiple agents within an environment.  Makes me wonder if I can put DCLI on a traditional RAC system?

Enjoy!

twitter: @dbasolved

blog: http://dbasolved.com

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Pardy DBA

ORA-00001: unique constraint (ORA.BLOG_TAGLINE_PK) violated

Martin Widlake's Yet Another Oracle Blog

Oracle performance, Oracle statistics and VLDBs

HeliFromFinland

Heli's thoughts on Database Designing, Oracle SQL Developer Data Modeler, User Groups etc.

Julian Dontcheff's Database Blog

The good DBA is one who learns from his mistakes, the best DBA is one who learns from other DBA's mistakes

Martins Blog

Trying to explain complex things in simple terms

The Data Warrior

Changing the world, one data model at a time. How can I help you?

Maaz Anjum's Blog

A life yet to be lived...

PeteWhoDidNotTweet.com

Stuff that interests me, if not you!

Uwe Hesse

about Database Technology

%d bloggers like this: