OEM PSU – work around required

Got into the office today to a nice warning from OEM; the warning is the following:

 EM Event: Warning:Management Services and Repository – Average event dequeue time is 108 milliseconds, crossed warning (100) or critical (250) threshold. Dequeue time may get worse if queue table (EM_EVENT_BUS_TABLE) needs maintenance.

 In researching this message within MOS, I realized it was a bug that was associated with bug INCORRECT ALERT MESSAGES (14336237).  This bug is confirmed with internal Oracle note number 1499189.1.  Fortunately, there is a patch for this bug, which I was happy to see; however, what got under my skin was that this patch is rolled into the PSU (1512481.1) for OEM 12.1.0.2.0.  After applying the PSU, OEM is set to 12.1.0.2.1 and has this patch installed.

After confirming that patch 14336237 was in the PSU via the readme.  I also checked the OPatch inventory for the OMS to make sure the patch was there.  Patch 14226237 is applied with the PSU patch set.

$ ./opatch lsinventory | grep 14336237

   14756522, 13894390, 14336237, 14357531, 14571792, 14697257, 14544587.   

 With the PSU patch applied to the OEM environment, the incorrect alert messages should have been resolved.  In this environment, the PSU has been in place for a week and incorrect alerts are still filling my inbox. 

Per the note 1499189.1, there is a workaround.  This work around is basically setting the metric associated with the event message to NULL.  I have provided the workaround here.

Workaround: 

Login to Cloud Contrl console as sysman (or a user with FULL privilege on oracle_emrep target)
  ->Setup -> Manage Cloud Control -> Health Overview
        ->OMS and Repository (Target menu) > Monitoring > Metric and Collection Settings

Change the WARNING and CRITICAL thresholds to NULL for the following metrics:
> Average Event Dequeue Time (Milliseconds)
> Average Event Queue Query Time (Seconds)

      ->Click OK at the top of the page to save the changes.

Overall, I’m kinda disappointed in that fact that the PSU didn’t actually fix this issue although the patch is installed.  My next task it to find out from Oracle if there are any one-off patches that should be applied after the PSU to correct this issue.  I’ll update this thread if there is.

Advertisements

5 comments

  1. Bobby, great post. Thanks for taking the time to outline a solution to a fairly EM Issue.

  2. Sarathbabu · · Reply

    Hi Bobby,
    Can you please check this Patch 14840279 – 12.1.0.2.1 Patch Set Update for Oracle Management Service.

    Thanks
    Sarath

    1. Hi Sarathbabu,

      You are correct, the PSU patch set I referenced is 14840279. The patch (14336237) that fixes this event warning is part of the PSU.

      I have applied this PSU (14840279) in 3 diffrent enviornments with the same results. The workaround is valid to help resolve the issue. I still need to follow up with Oracle to see if there is a one-off.

      Thanks for reminding me and ensuring that I put the correct PSU patch number with the post.

      Thanks

      Bobby

  3. Hi Bobby,

    We received the same warning e-mail today but we already have the patch applied. So, before I contact Oracle, I would like to know if you followed up with Oracle already. If yes, please let me know what they suggested and what actually fixed this – if fixed, at all.

    For now, I have followed temporary workaround of setting the specified thresholds to ‘null’

    Regards,
    Karan

    1. Karan,

      Just got your message. If the patch has been applied and you are still getting the message/email, I would follow up with Oracle. I have not seen the message/email again in the environments I monitor since the patch has been applied.

      Thanks

      Bobby

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Mrs. Jaye's Class Blog

Coordinate Algebra & Advanced Algebra

Pardy DBA

ORA-00001: unique constraint (ORA.BLOG_TAGLINE_PK) violated

Martin Widlake's Yet Another Oracle Blog

Oracle performance, Oracle statistics and VLDBs

HeliFromFinland

Heli's thoughts on Database Designing, Oracle SQL Developer Data Modeler, User Groups etc.

Julian Dontcheff's Database Blog

The good DBA is one who learns from his mistakes, the best DBA is one who learns from other DBA's mistakes

Martins Blog

Trying to explain complex things in simple terms

The Data Warrior

Changing the world, one data model at a time. How can I help you?

Maaz Anjum's Blog

A life yet to be lived...

PeteWhoDidNotTweet.com

Stuff that interests me, if not you!

%d bloggers like this: