ORACLE EM DBA

Problems, Solutions, Test Cases from my professional and personal experience in technologies like Oracle Enterprise Manager 13 c,Oracle Enterprise Manager 12c, Oracle Configuration Manager and some other stuff.

Category: Uncategorized (page 1 of 6)

Host Monitoring using Oracle Enterprise Manager 13c [Webinar]

In this blog I have shared the recent webinar which I did for “AIOUG-North India Chapter“. This webinar is related to the “Host Monitoring” using 13c Cloud Control. More details about this session is mentioned below. In-case you have missed it, you can watch the recorded session on YouTube Page of “AIOUG-North India Chapter”.

Session Details:-

Abstract: In this session we will learn how effectively you can monitor servers/hosts in your organization through new “Oracle Enterprise Manager 13c Cloud Control”. Some New features which were missing in previous release 12c Cloud Control.

Profile: Deepak Sharma is working as Sr. Application Administrator I, with experience of 7+ years. He has worked on technologies like Oracle Enterprise Manager 11g Grid Contol, Oracle Enterprise Manager 12c Cloud Control, Oracle Enterprise Manager 13c, Oracle Database, Oracle Configuration Manager and Sql Server. He is currently working with “Oceaneering International Inc.” from last 5 years and before that he has been employee of “Bebo Technologies” for 2 years. He is Co-Founder of AIOUG-North India Chapter and core team member of AIOUG.

Webinar URL: https://www.youtube.com/watch?v=58s_Mf-k7m0

Demo URL: https://www.youtube.com/watch?v=ECNOpfQTCn8

 

AIOUG NI Chapter-Over The WWW

Youtube: https://www.youtube.com/user/aiougnic

Official Website: http://www.aioug.org/NorthIndiaChapter.php

Facebook: https://www.facebook.com/AIOUG.NIC

Twitter: https://twitter.com/nichapter_aioug

Blog: https://aiougnic.wordpress.com/

Google+ : https://plus.google.com/+AIOUGNorthIndiaChapter/

Do subscribe to our pages for regular updates.

Thanks

ADDKS

Step by Step document to Apply PSU to OEM12c

In this document I have mentioned the step by step process which you have to follow in order to apply PSU to OEM12c. The example shown in this document is for OEM12cR4 which is my current environment. I have documented this process as the step involved are slightly different that normal steps for database patching, OMS patching involves some additional steps.

 

Step 1: Check OMS Details as it will be needed in later steps.

Download the latest PSU from Oracle Support and move it to OEM Server.

Step 2: Check OMS Details as it will be needed in later steps.

[oracle@hanoemap1 bin]$ ./emctl status oms -details

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation. All rights reserved.

Enter Enterprise Manager Root (SYSMAN) Password :

Console Server Host : hanoemap1.emdeepaksharma.com

HTTP Console Port : 7790

HTTPS Console Port : 7803

HTTP Upload Port : 4890

HTTPS Upload Port : 4904

EM Instance Home : /app/oracle/product/middleware/gc_inst/em/EMGC_OMS1

OMS Log Directory Location : /app/oracle/product/middleware/gc_inst/em/EMGC_OMS1/sysman/log

OMS is not configured with SLB or virtual hostname

Agent Upload is unlocked.

OMS Console is unlocked.

Active CA ID: 1

Console URL: https://hanoemap1.emdeepaksharma.com:7803/em

Upload URL: https://hanoemap1.emdeepaksharma.com:4904/empbs/upload

 

WLS Domain Information

Domain Name : GCDomain

Admin Server Host : hanoemap1.emdeepaksharma.com

Admin Server HTTPS Port: 7103

Admin Server is RUNNING

 

Oracle Management Server Information

Managed Server Instance Name: EMGC_OMS1

Oracle Management Server Instance Host: hanoemap1.emdeepaksharma.com

WebTier is Up

Oracle Management Server is Up

 

BI Publisher Server is Up

BI Publisher Server named ‘BIP’ running at local URL: https://hanoemap1.emdeepaksharma.com:7803/xmlpserver

BI Publisher Server Logs: /app/oracle/product/middleware/gc_inst/user_projects/domains/GCDomain/servers/BIP/logs/

BI Publisher Log : /app/oracle/product/middleware/gc_inst/user_projects/domains/GCDomain/servers/BIP/logs/bipublisher/bipublisher.log

[oracle@hanoemap1 bin]$

 

Step 3: WebLogic encrypted configuration and key files.

Create directory to store property file.

[oracle@hanoemap1 ~]$ cd /app/oracle/product/middleware/oms12cr4/oms/

[oracle@hanoemap1 oms]$ mkdir -p oms_encr

Run the following script to create the WebLogic encrypted configuration and key files.

<OMS_HOME>/OPatch/wlskeys/createkeys.sh –oh <full path of platform OMS Oracle Home> -location <location to put the encrypted files>

[oracle@hanoemap1 bin]$ cd /app/oracle/product/middleware/oms12cr4/oms/OPatch/wlskeys

[oracle@hanoemap1 wlskeys]$ ./createkeys.sh -oh /app/oracle/product/middleware/oms12cr4/oms -location /app/oracle/product/middleware/oms12cr4/oms/oms_encr

Please enter weblogic admin server username: weblogic

Please enter weblogic admin server password: formula1

Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key…

Trying to get configuration and key files for the given inputs…

This operation will take some time. Please wait for updates…

User configuration file created: /app/oracle/product/middleware/oms12cr4/oms/oms_encr/config

User key file created: /app/oracle/product/middleware/oms12cr4/oms/oms_encr/key

‘createKeys’ succeeded.

[oracle@hanoemap1 wlskeys]$

Step 4: Create the property file with the following entries:

Enter the details of the WebLogic Administrator Server URL retrieved in STEP 1

WLS Domain Information

Domain Name : GCDomain

Admin Server Host : hanoemap1.emdeepaksharma.com

Admin Server HTTPS Port: 7103

Admin Server is RUNNING

 

The output displayed contains the WebLogic Adminstrator Server details. Here is an example on how to construct the URL with these output details. Property file should contain below mentioned information and should look similar to the following once completed:

AdminServerURL=t3s://<host address from where admin server is running>:<port of the admin server>

AdminConfigFile=<‘config’ file location>

AdminKeyFile=<‘key’ file location>

 

Following is the example of how a property file (constructed by the above mentioned guidelines) should appear:

AdminServerURL= https://hanoemap1.emdeepaksharma.com:7103

AdminConfigFile=/app/oracle/product/middleware/oms12cr4/oms/oms_encr/config

AdminKeyFile=/app/oracle/product/middleware/oms12cr4/oms/oms_encr/key

 

Create directory for property file followed by creation of text file as mentioned below

mkdir –p  /app/oracle/product/middleware/property_file

cd /app/oracle/product/middleware/property_file

vi property_file.txt

 

Add below mentioned entries in “property_file.txt”

AdminServerURL= https://hanoemap1.emdeepaksharma.com:7103

AdminConfigFile=/app/oracle/product/middleware/oms12cr4/oms/oms_encr/config

AdminKeyFile=/app/oracle/product/middleware/oms12cr4/oms/oms_encr/key

 

Step 5: Analyze the PSU

Make sure you have the latest OPatch for your OMS. The analyze process will fail with error if the OPatch is not latest.

Run the following command on OMS instance home to ensure that you do not have any issues with configuration and binary prerequisite checks. Please note: the OMS should be online when you do this step only.

export ORACLE_HOME=$OMS_HOME

export PATH=$ORACLE_HOME/OPatch:$ORACLE_HOME/bin:$PATH:

Navigate to the patch folder and analyze the patch.

[oracle@hanoemap1 middlware]$  cd /app/oracle/product/middleware/oms12cr4/oms/22132672

While Installing the System patch

$ opatchauto apply -analyze  [–property_file <location of property file>]

[oracle@hanoemap1 22132672]$$ opatchauto apply -analyze -property_file “/app/oracle/product/middleware/property_file/property_file.txt”

[oracle@hanoemap1 22132672]$ opatchauto apply -analyze -property_file /app/oracle/product/middleware/property_file/property_file.txt -invPtrLoc /app/oracle/product/middleware/oms12cr4/oms/oraInst.loc  OPatchAuto.OMS_DISABLE_HOST_CHECK=true

 

OPatch Automation Tool

Copyright (c) 2014, Oracle Corporation.  All rights reserved.

OPatchauto version : 11.1.0.12.9

OUI version        : 11.1.0.12.0

Running from       : /app/oracle/product/middleware/oms12cr4/oms

Log file location  : /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/opatch2017-01-20_16-45-34PM_1.log

OPatchauto log file: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatchauto/22132672/opatch_oms_2017-01-20_16-45-36PM_analyze.log

Configuration Validation: Success

Running apply prerequisite checks for sub-patch(es) “22132672” and Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”…

Please monitor OPatch log file: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/22132672_Jan_20_2017_16_45_34/ApplyPrereq2017-01-20_16-46-55PM_2.log

Sub-patch(es) “22132672” are successfully analyzed for Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”

Complete Summary

================

All log file names referenced below can be accessed from the directory “/app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/2017-01-20_16-45-34PM_SystemPatch_22132672_1”

Prerequisites analysis summary:

——————————-

The following sub-patch(es) are applicable:

  Oracle Home Name   Sub-patches                                     Log file

  —————- ——–  ———–                                          ——–

           oms12c3            22132672                                         22132672_ApplyPrereq2017-01-20_16-46-55PM_2.log

Log file location: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatchauto/22132672/opatch_oms_2017-01-20_16-45-36PM_analyze.log

OPatchauto succeeded.

[oracle@hanoemap1 22132672]$

 

Step 6: On Successful analysis, Apply PSU. If not, address the errors, then proceed with next steps.

Stop OMS

[oracle@hanoemap1 22132672]$ emctl stop oms

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation. All rights reserved.

Stopping WebTier…

WebTier Successfully Stopped

Stopping Oracle Management Server…

Oracle Management Server Successfully Stopped

Oracle Management Server is Down

 

Apply Patch

[oracle@hanoemap1 22132672]$ opatchauto apply -property_file /app/oracle/product/middleware/property_file/property_file.txt

 

OPatch Automation Tool

Copyright (c) 2014, Oracle Corporation. All rights reserved.

OPatchauto version : 11.1.0.12.9

OUI version : 11.1.0.12.0

Running from : /app/oracle/product/middleware/oms12cr4/oms

Log file location : /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/opatch2017-01-20_17-04-10PM_1.log

OPatchauto log file: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatchauto/22132672/opatch_oms_2017-01-20_17-04-11PM_deploy.log

Configuration Validation: Success

Running apply prerequisite checks for sub-patch(es) “22132672” and Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”…

Please monitor OPatch log file: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/22132672_Jan_20_2017_17_04_10/ApplyPrereq2017-01-20_17-05-37PM_2.log

Sub-patch(es) “22132672” are successfully analyzed for Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”

To continue, OPatchauto will do the following:

[Patch and deploy artifacts] : Apply sub-patch(es) [ 22132672 ] to Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”;

Apply RCU artifact with patch “/app/oracle/product/middleware/oms12cr4/oms/.patch_storage/22132672_Dec_16_2015_23_31_36/original_patch”;

Register MRS artifact “omsPropertyDef”;

Register MRS artifact “procedures”

Do you want to proceed? [y|n]

y

User Responded with: Y

Applying sub-patch “22132672” to Oracle Home “/app/oracle/product/middleware/oms12cr4/oms”…

Please monitor OPatch log file: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/22132672_Jan_20_2017_17_04_10/apply2017-01-20_17-07-53PM_4.log

Updating repository with RCU reference file “/app/oracle/product/middleware/oms12cr4/oms/.patch_storage/22132672_Dec_16_2015_23_31_36/original_patch”

Registering service “omsPropertyDef” with register file “/app/oracle/product/middleware/oms12cr4/oms/sysman/metadata/omsProperties/definition/BIPHAPropDefinition.xml”…

Registering service “procedures” with register file “/app/oracle/product/middleware/oms12cr4/oms/sysman/metadata/procedures/linuxpatch/PatchLinux.xml”…

Registering service “procedures” with register file “/app/oracle/product/middleware/oms12cr4/oms/sysman/metadata/procedures/linuxpatch/ULNPatchLinux.xml”…

 

Complete Summary

================

All log file names referenced below can be accessed from the directory “/app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/2017-01-20_17-04-10PM_SystemPatch_22132672_1”

Patching summary:

—————–

Binaries of the following sub-patch(es) have been applied successfully:

Oracle Home Name Sub-patches Log file

—————- ———– ——–

oms12c3 22132672 22132672_apply2017-01-20_17-07-53PM_4.log

Deployment summary:

——————-

The following artifact(s) have been successfully deployed:

Artifacts Log file

——— ——–

SQL rcu_applypatch_original_patch_2017-01-20_17-08-57PM.log

MRS-omsPropertyDef emctl_register_omsPropertyDef_2017-01-20_17-10-38PM.log

MRS-procedures emctl_register_procedures_2017-01-20_17-10-43PM.log

MRS-procedures emctl_register_procedures_2017-01-20_17-10-48PM.log

Log file location: /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatchauto/22132672/opatch_oms_2017-01-20_17-04-11PM_deploy.log

OPatchauto succeeded.

 

Start OMS

[oracle@hanoemap1 22132672]$ emctl start oms

Oracle Enterprise Manager Cloud Control 12c Release 4

Copyright (c) 1996, 2014 Oracle Corporation. All rights reserved.

Starting Oracle Management Server…

Starting WebTier…

WebTier Successfully Started

Oracle Management Server Successfully Started

Oracle Management Server is Up

Starting BI Publisher Server …

BI Publisher Server Already Started

BI Publisher Server is Up

[oracle@hanoemap1 22132672]$

 

Step 7: Verify Applied Patches

 

[oracle@hanoemap1 22132672]$ opatch lsinventory

Oracle Interim Patch Installer version 11.1.0.12.9

Copyright (c) 2017, Oracle Corporation. All rights reserved.

Oracle Home : /app/oracle/product/middleware/oms12cr4/oms

Central Inventory : /app/oracle/oraInventory

from : /app/oracle/product/middleware/oms12cr4/oms/oraInst.loc

OPatch version : 11.1.0.12.9

OUI version : 11.1.0.12.0

Log file location : /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/opatch2017-01-20_18-29-52PM_1.log

OPatch detects the Middleware Home as “/app/oracle/product/middleware/oms12cr4”

Lsinventory Output file location : /app/oracle/product/middleware/oms12cr4/oms/cfgtoollogs/opatch/lsinv/lsinventory2017-01-20_18-29-52PM.txt

——————————————————————————–

Installed Top-level Products (1):

EM Platform (OMS) 12.1.0.4.0

There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch 22132672 : applied on Fri Jan 20 17:08:47 EST 2017

Unique Patch ID: 19559467

Patch description: “EMBP Patch Set Update 12.1.0.4.160119”

Created on 16 Dec 2015, 23:31:36 hrs PST8PDT

Bugs fixed:

20010360, 20346629, 18599481, 19459173, 20646440, 18830050, 18244680

19614772, 20492788, 18932882, 20646540, 19355307, 20263906, 20556489

20106145, 14301574, 18721562, 17714229, 19328391, 19535114, 19861538

19794113, 19604817, 20596437, 19892848, 19218456, 19978077, 18505219

17843191, 20345612, 20748748, 20024779, 18256983, 19329809, 19300103

17987955, 18351725, 18903783, 19430853, 19845101, 20870437, 20421152

20223681, 21437683, 20019184, 20263188, 19069940, 21545754, 19861412

19136663, 18446798, 19474685, 18765545, 18038059, 17783148, 19509780

20838602, 18041873, 19030447, 20861693, 20933667, 19574756, 19258481

18938561, 17812648, 20377474, 20251039, 19572261, 18945007, 20307428

18388324, 19500609, 20751948, 22324488, 17991694, 19137411, 20369347

19474929, 19592446, 18710124, 20536095, 21358443, 19352118, 19557054

18726708, 19595813, 19170861, 19685443, 18353162, 19289270, 19367214

18906451, 19458672, 20284029, 18490444, 19766881, 20021966, 18509136

20739088, 21450977, 18695961, 19511544, 19812966, 20093530, 20576148

20806405, 20623258, 19055251, 19586898, 20260177, 19323634, 21462217

19941819, 18725891, 19824587, 20672821, 19780847, 18359621, 21067380

19393211, 19211048, 20505155, 18476364, 18490468, 18247849, 19071602

19068674, 18490455, 19524568, 20616490, 19061168, 19077861, 20031287

18756725, 19281252, 21061986, 18977955, 18485889, 20695395, 18521731

18411036, 19775544, 18900197, 17834366, 20460507, 19545505, 21330117

19823996, 20645335, 20633770, 17407918, 18706824, 20568225, 18805935

19376082, 19146514, 21429472, 20985421, 18490432, 18230284, 19694744

18726208, 20044313, 18516298, 18391707, 20642633, 18999400, 20692416

19362386, 20392036, 21495795, 19452280, 20623344, 21093418, 22132672

18356966, 19055979, 20617058, 19830994, 20030493, 19823761, 18834216

19196979, 19785223, 18389542

——————————————————————————–

OPatch succeeded.

[oracle@hanoemap1 22132672]$

 

Hope this document will help you apply patches in your OEM12c environment.

 

Thanks

ADDKS

OEM13c Release 2 Step by Step Upgrade

This is the step by step document to upgrade OEM12cR4 to OEM13cR2. Mentioned below are some important prerequisites, make sure to you do not skip anyone of them before starting your upgrade.

Prerequisites

· Check the certification matrix to make sure your existing environment is certified for this upgrade. I have already gather the required details.

· EM13cR2 OMS requires Oracle 12.1.0.2 as repository database.

· EM13cR2 agents don’t support Enterprise Linux earlier than version 6.

· Back up your repository database and OMS.

· Another important point to take is the timezone of Repository Database and OMS machine. Since in our case we have both OMS and Database on the same machine so it does not matter. However if the timezone is not same then, Upgrade will fail at ‘Plug-ins Perquisites check’ step.

 

Certification Matrix

certification_matrix13cr2

Since our Existing environment is on RHEL Linux 6 so the certification matrix is based on that.

certification_matrix13cr2_1

There are few plugin’s which have been obsoleted in new release 13c. Check if there are any obsolete plug-ins in your current system. Un-deploy obsolete plugins if exists both from OMS and Agent before proceeding with upgrade to EM13.2:

  • EMC Symmetrix Array (oracle.em.sesy)
  • EMC Celerra Storage (oracle.em.secs)
  • EMC CLARiiON Array (oracle.em.secl)
  • Microsoft Biz Talk Server (oracle.em.smbs)
  • Exalogic Elastic Cloud Infrastructure (oracle.em.soee)

 

Upgrade Process.

Step 1: Copy EMKey to the Management Repository.

Note: EMKey is used during upgrade, EM13c will need to access EMKey. { secret key to encrypt password}

$OMS_HOME/bin/emctl config emkey -copy_to_repos

Step 2: Make sure you have “optimizer_adaptive_features” disabled in your repository database, if not run the below mentioned SQL command.

SQL> alter system set optimizer_adaptive_features=false scope=spfile;

 

Step 3: If you have upgraded your database recently, make sure that database compatibility parameter equal to the database version.

SQL> alter system set compatible=”12.1.0.2.0″ scope=spfile

To make the new changes take effect, please restart your database.

Step 4: Download the OEM13cR2 Setup from Oracle Website. Use the below mentioned URL to download 6 files included in setup.

http://www.oracle.com/technetwork/oem/grid-control/downloads/oem-linux64-3237624.html

Step 5: Copy Software on the installation server.

Make you copy all 5 files in one single folder. NOTE: Please do not unzip any file, EM13c installation doesn’t need you to unzip the files. In EM13c we have “em13200_linux64.bin” as executable we have to run this file as “oracle” user.

image1

Step 6: Run the Setup and enter MOS Details.

image2

 

The first screen that pops-up will request your credentials for MOS. This is optional not mandatory if you want to receive updates from Oracle you can put your credentials or you can leave it blank as well. “CLICK NEXT” to proceed.

Step 7: Software Update (Optional)

Next step is the software update window, here you will be prompted if you want to check for any new updates available for your new release on which you ware upgrading. Since 13c is released recently so it already contains all latest updates, still if you want to check you can do that by simply providing your MOS credentials as shown below in the screenshot.

image3

image4

Step 8: Pre-requisite Checks

At this step setup performs a check to make sure all the pre-requisites are passed successfully. Make sure you do not have Warning or Failed status for any one of the pre-requisites. Some warnings can be ignored but make you fix the “Failed” pre-requisites before proceeding to the next step as it can FAIL you upgrade later.

image5

 

Step 9: Installation Types

In this step you will be prompted to select the “Installation Type”, since we are doing the upgrade we will select “Upgrade as existing Enterprise Manager System”. For upgrade there is only one option available “One-System Upgrade”.

Select the radio button as shown in the screenshot below and click on “NEXT”.

image6

 

Step 10: Installation Details.

In this step you will be prompted to enter the middleware home location for new Oracle Enterprise Manager 13c. Make sure you provide the correct location as this is the directory where all binaries for OEM13c going to be stored.

image7

 

Step 11: Database Connection Details.

Here you have to provide the connection details for your OMR database. Since we are doing the upgrade of our existing environment, “Connect Descriptor” field is already filled. You have to provide “SYS and SYSMAN” passwords.

image8

EM13c gives as option to disable DDMP during the upgrade.

If you are planning to stop your EM for POST upgrade maintenance then it is recommended to DISABLE DDMP jobs. Deferred Data Migration (DDMP) is a post-upgrade activity to migrate the historical data of your old Enterprise Manager. Depending on how much data stored, it may take same time.

Also, before proceeding to next step make you have you have already backed-up your OMR. Select the check-box and click on “NEXT”.

Before you proceed, make sure you have applied latest PSU on your database to avoid failure during your repository upgrade.

Note: If you haven’t stopped the OMS and agent running on your OMS and OMR server (if OMS and database are on separate machines) , please stop them now. Once OMS and Agent are down then click on NEXT.

Run the below mentioned commands to stop OMS and Agent respectively.

$OMS_HOME/bin/emctl stop oms -all
$AGENT_HOME/agent_inst/bin/emctl stop agent

Although you stop the agents, you may still get a warning about them. Ignore it and continue to upgrade.

 

Step 12: Database Configuration Check.

When you click on “NEXT” in “Database Connection Details” window in the last step, it performs come checks for the existing database.

image9

image10

 

You may or may not receive warning depending on your database configuration.

image11

Some of the warnings can be fixed automatically by the installer when you click “YES”, some of them you have to fix manually. In-order to avoid your upgrade to FAIL in middle make sure you fix all the warnings and then click “NEXT”.

image12

 

Step 13: Plug-in Upgrade, select additional Plug-in’s if you want to install any of them during this installation..

This displays all your existing plug-ins which will be upgraded during this process. Please read thoroughly and click on “NEXT”. Also if you want to install and additional plugin’s in this install, select from the list.

image13

 

Step 15: Enter Weblogic Details

Please enter the details of Weblogic server for your OEM13c installation. Make sure you choose new location for your OMS instance outside your Middleware home. Also enter your weblogic password.

image14

 

Step 16: Shared Location Details for BI Publisher

If you already had BI Publisher installed and configured in your existing environment then BI Publisher automatically gets upgraded during upgrade to OEM13c. The checkbox shown in the below screenshot will be checked and grayed out by default only if BI Publisher is already configured.

image15

 

Step 17: Ports Configuration

Do not make any changes until the mentioned ports are already being used. Accept the default ports and click on NEXT.

image16

 

Step 18: Review Details

Review the information you entered, and then click the upgrade button to start upgrade process. The time for this upgrade will depend on hardware of your system.

image17

image18

image19

image20

image21

Step 19: Run root.sh script as ROOT user to complete the installation. [root.sh script is located in the middleware home]

image22

 

Step 20: Final & Final Details

image23

 

Step 21: POST Upgrade Steps.

Login to your 13c console using the sysman user and perform the POST-upgrade tasks. Navigate to the “upgrade agents” page under the “setup menu” to upgrade your agents.

13c_console

license

home_pagehome_page

 

Regards

ADDKS

Moved WordPress blog to my website “emdeepaksharma.com”

Readers,

I have moved my WP blog to my new website “emdeepaksharma.com”, I am publishing all my new posts over there, hope you will like this. Need you support and suggestions are always welcome.

Thanks

Deepak Sharma

Back to OEM Basics – Agent12c Blocked by OMS [Agent Resynchronization]

This blog is a part of my “Back to OEM Basics” series. In this I have explained in details how easily you can solve your “Agent blocked by OMS” issue. With the help of logs and screenshots I have made it more simple to understand.

Lets got step by step and see how you can fix your problem and can Unblock an blocked agent.


Oracle Agent usually gets blocked when it is Out-Of-Sync with the OMR repository database.

 

Step 1: How to check if agents are blocked, this can be simply done by login-in to OEM.

 

Please login to OEM and navigate to “Setup-> Manage Cloud Control –> Agents” as shown in the screenshot.

 

agent_blocked0

 

The “Manage Cloud Control” AGENTS home page will give you list all of the installed agents, you can sort the list using the “STATUS” column as shown in the below screenshot.

 

image1

 

Step 2: Login to Agent server and check the logs to verify the cause.

 

I checked agent logs on the server where agent was BLOCKED and this is what I found.

 

Agent Logs:-

2016-09-20 10:13:04,906 [9825:547A0FF8:GC.SysExecutor.1036 (Ping OMS)] INFO - attempting initial heartbeat

2016-09-20 10:13:04,919 [9825:547A0FF8] INFO - The agent has been blocked by the OMS.

2016-09-20 10:13:04,919 [9825:547A0FF8] INFO - Reason the OMS blocked the agent: Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console.

2016-09-20 10:13:41,530 [9743:2F24D3F3] WARN - Subsystem (Ping Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@21a09189

2016-09-20 10:13:41,531 [9743:2F24D3F3] WARN - Subsystem (Upload Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@653b1740

2016-09-20 10:14:41,530 [9822:669C9195] WARN - Subsystem (Ping Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@354632cf

2016-09-20 10:14:41,531 [9822:669C9195] WARN - Subsystem (Upload Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@6cde3ddf

2016-09-20 10:15:41,532 [9836:6A4F37D3] WARN - Subsystem (Ping Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@39af50a4

2016-09-20 10:15:41,532 [9836:6A4F37D3] WARN - Subsystem (Upload Manager) returned bad status of oracle.sysman.gcagent.tmmain.lifecycle.AgentSystemMonitor$TaskStatus@9bce0a

2016-09-20 10:16:04,938 [9759:755A5435:GC.SysExecutor.1028 (Ping OMS)] INFO - attempting initial heartbeat

2016-09-20 10:16:04,948 [9759:755A5435] INFO - The agent has been blocked by the OMS.

2016-09-20 10:16:04,948 [9759:755A5435] INFO - Reason the OMS blocked the agent: Agent is out-of-sync with repository. This most likely means that the agent was reinstalled or recovered. Please contact an EM administrator to unblock the agent by performing an agent resync from the console.

 

Step 3: Check status of agent by running “emctl status agent” command.

 

D:oracleproductagent12cr3agent_instbin> emctl status agent

Oracle Enterprise Manager Cloud Control 12c Release 2

Copyright (c) 1996, 2012 Oracle Corporation.  All rights reserved.

—————————————————————

Agent Version     : 12.1.0.2.0

OMS Version       : (unknown)

Protocol Version  : 12.1.0.1.0

Agent Home        : D:/oracle/product/agent12cr3/agent_inst

Agent Binaries    : D:oracleproductagent12cr3core12.1.0.2.0

Agent Process ID  : 3416

Parent Process ID : 2524

Agent URL         : https://HANHRRP3.oiiad.com:3872/emd/main/

Repository URL    : https://hanoemap1.oii.oceaneering.com:4904/empbs/upload

Started at        : 2016-09-16 22:43:30

Started by user   : HANHRRP3$

Last Reload       : (none)

Last successful upload                       : (none)

Last attempted upload                        : (none)

Total Megabytes of XML files uploaded so far : 0

Number of XML files pending upload           : 690

Size of XML files pending upload(MB)         : 1.13

Available disk space on upload filesystem    : 47.09%

Collection Status                            : Collections enabled

Heartbeat Status                             : Agent is blocked

Last attempted heartbeat to OMS              : 2016-09-20 10:43:05

Last successful heartbeat to OMS             : (none)

Next scheduled heartbeat to OMS              : 2016-09-20 10:46:05

—————————————————————

Agent is Running and Ready

 

Step 4: Navigate to Agent home page in Oracle Enterprise Manager Console to verify and rectify issue.

 

agent_blocked1

 

The error message in console as well clearly shows that agent is Blocked and need to “unblock” it manually.

 

Step 5: Navigate to “Resynchronization” under “Agent” on the home page as show in the screenshot below.

 

agent_blocked2

 

Step 6: Select unblock agent Under “Agent Resynchronization” make the checkbox is checked and click on “Continue”.

 

agent_blocked3

 

 

Step 7: This confirms that “Agent Resynchronization” job has been executued successfully.

 

You can monitor the progress of the job by clicking the Job Name {eg. RESYNC_20160920104720}

 

agent_blocked4

 

Step 8: Check RESYNC Job Details and verify  if it is completed.

 

agent_blocked5

 

Step 9: Navigate back to agent home page and verify Agent status.

It should be “UP & RUNNING” after the RESYNC job has completed successfully.

 

agent_blocked6

 

This way you can easily un-block all of your Blocked agents and make them again fit for monitoring your targets.

Stay tuned for more “BACK TO OEM BASICS” Topics.

 

Thanks

ADDKS

Silent Installation of Oracle 12C Agent on Linux Server

I know its very late but it was pending from my side since long. This is step by step silent installation of Oracle12c Agent on any linux machine. I have tried by best to make your job easy.

Please follow the below mentioned steps to Install Oracle 12c agent.


 

Step 1: Login to OEM server and then swtich to “oracle” user, using “sudo su – oracle“. Navigate to OMS home.

cd “/app/oracle/product/middleware/OMS12CR4/oms/bin”.

 

Step 2: Create a temporary directory {setup} under “tmp” where agent setup will be downloaded.

“mkdir -p /tmp/setup”

 

Step 3: Download Agent setup using below mentioned procedure.

Procedure is mentioned below:-

————————————————

[oracle@testdb1 bin]$ pwd

/app/oracle/product/middleware/OMS12CR4/oms/bin

[oracle@testdb1 bin]$ ./emcli login -username=sysman -password=formula1

Login successful

 

[oracle@testdb1 bin]$ ./emcli sync

Synchronized successfully

 

[oracle@testdb1 bin]$ ./emcli get_supported_platforms

———————————————–

Version = 12.1.0.3.0

Platform = Linux x86-64

———————————————–

Version = 12.1.0.4.0

Platform = Linux x86-64

———————————————–

Platforms list displayed successfully.

 

[oracle@testdb1 bin]$ ./emcli get_agentimage -destination=/tmp/setup -platform=”Linux x86-64″ -version=12.1.0.3.0

 

=== Partition Detail ===

Space free : 3 GB

Space required : 1 GB

Check the logs at /app/oracle/product/middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_2016-08-05_00-57-38-AM.log

Downloading /tmp/setup/12.1.0.3.0_AgentCore_226.zip

File saved as /tmp/ setup /12.1.0.3.0_AgentCore_226.zip

Downloading /tmp/ setup /12.1.0.3.0_PluginsOneoffs_226.zip

File saved as /tmp/ setup /12.1.0.3.0_PluginsOneoffs_226.zip

Downloading /tmp/ setup /unzip

File saved as /tmp/ setup /unzip

Agent Image Download completed successfully.

[oracle@testdb1 bin]$ pwd

================================================================================

Step 4:  Copy setup from OEMAP1 server to LOCAL machine where agent needs to be installed. You can do this using WINSCP or any other file transfer.

 

==================================================================================

 

Step 5:  Login to server as “oracle” user and create folder of Agent Installation.

mkdir – /app/oracle/product/agent12cr3

==================================================================================

 

Step 6:  Navigate to location where agent setup was copied and run the below mentioned command.

 

[oracle@testdb1 agent_sw]$ ./agentDeploy.sh AGENT_BASE_DIR=/app/oracle/product/agent12cr3 OMS_HOST=oemap1.test.com EM_UPLOAD_PORT=4904 AGENT_REGISTRATION_PASSWORD=formula123

 

Validating the OMS_HOST & EM_UPLOAD_PORT

Executing command : /app/oracle/product/agent12cr3/core/12.1.0.3.0/jdk/bin/java -classpath /app/oracle/product/agent12cr3/core/12.1.0.3.0/jlib/agentInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/OraInstaller.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12cr3/core/12.1.0.3.0 /app/oracle/product/agent_sw /app/oracle/product/agent12cr3 -prereq

Validating oms host & port with url: http://testdb1.oii.test.com:4903/empbs/genwallet

Validating oms host & port with url: https://testdb1.oii.test.com:4903/empbs/genwallet

Return status:3-oms https port is passed

Unzipping the agentcoreimage.zip to /app/oracle/product/agent12cr3 ….

12.1.0.3.0_PluginsOneoffs_226.zip

Executing command : /app/oracle/product/agent_sw/unzip -o /app/oracle/product/agent_sw/12.1.0.3.0_PluginsOneoffs_226.zip -d /app/oracle/product/agent12cr3

Executing command : /app/oracle/product/agent12cr3/core/12.1.0.3.0/jdk/bin/java -classpath /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/OraInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/xmlparserv2.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/srvm.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/emCfg.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/jlib/agentInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/share.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12cr3/core/12.1.0.3.0 /app/oracle/product/agent_sw /app/oracle/product/agent12cr3 /app/oracle/product/agent12cr3/agent_inst AGENT_BASE_DIR=/app/oracle/product/agent12cr3

 

Executing agent install prereqs…

Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -prereqchecker -silent -ignoreSysPrereqs -waitForCompletion  -prereqlogloc /app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy -entryPoint oracle.sysman.top.agent_Complete -detailedExitCodes PREREQ_CONFIG_LOCATION=/app/oracle/product/agent12cr3/core/12.1.0.3.0/prereqs  -J-DAGENT_BASE_DIR=/app/oracle/product/agent12cr3

Prereq Logs Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/prereq<timestamp>.log

Agent install prereqs completed successfully

 

Cloning the agent home…

Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -clone -forceClone -silent -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12cr3/core/12.1.0.3.0   AGENT_BASE_DIR=/app/oracle/product/agent12cr3 AGENT_BASE_DIR=/app/oracle/product/agent12cr3 OMS_HOST=testdb1.oii.test.com EM_UPLOAD_PORT=4903 AGENT_REGISTRATION_PASSWORD=formula123 -noconfig  ORACLE_HOME_NAME=agent12c4 -force b_noUpgrade=true AGENT_PORT=-1

Clone Action Logs Location:/app/oracle/oraInventory//logs/cloneActions<timestamp>.log

Cloning of agent home completed successfully

 

Attaching sbin home…

Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -attachHome -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12cr3/sbin ORACLE_HOME_NAME=sbin12c4 -force

Attach Home Logs Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/AttachHome<timestamp>.log

Attach home for sbin home completed successfully.

 

Updating home dependencies…

Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -updateHomeDeps -waitForCompletion HOME_DEPENDENCY_LIST={/app/oracle/product/agent12cr3/sbin:/app/oracle/product/agent12cr3/core/12.1.0.3.0} -invPtrLoc /app/oracle/product/agent12cr3/core/12.1.0.3.0/oraInst.loc -force

Update Home Dependencies Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/UpdateHomeDeps<timestamp>.log

Update home dependency completed successfully.

 

Performing the agent configuration…

Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runConfig.sh ORACLE_HOME=/app/oracle/product/agent12cr3/core/12.1.0.3.0 RESPONSE_FILE=/app/oracle/product/agent12cr3/core/12.1.0.3.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true

Configuration Log Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/cfgfw/CfmLogger<timestamp>.log

Agent Configuration completed successfully

 

The following configuration scripts need to be executed as the “root” user.

#!/bin/sh

#Root script to run

/app/oracle/product/agent12cr3/core/12.1.0.3.0/root.sh

To execute the configuration scripts:

  1. Open a terminal window
  2. Log in as “root”
  3. Run the scripts

Agent Deployment Successful.

Agent deployment log location:

/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/agentDeploy_2016-08-05_01-06-08-AM.log

Agent deployment completed successfully.

 

==================================================================================

 

Step 7:  Secure Agent using below mentioned command.

 

[oracle@testdb1 agent_sw]$ cd /app/oracle/product/agent12cr3/agent_inst/bin/

[oracle@testdb1 bin]$ ./emctl secure agent

Oracle Enterprise Manager Cloud Control 12c Release 3

Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.

Agent successfully stopped…   Done.

Securing agent…   Started.

Enter Agent Registration Password :

Agent successfully restarted…   Done.

EMD gensudoprops completed successfully

Securing agent…   Successful.

 

==================================================================================

 

Step 7:  Add internal targets existing on the server and upload.

[oracle@testdb1 bin]$ ./emctl config agent addinternaltargets

Oracle Enterprise Manager Cloud Control 12c Release 3

Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.

 

[oracle@testdb1 bin]$ ./emctl upload

Oracle Enterprise Manager Cloud Control 12c Release 3

Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.

—————————————————————

EMD upload completed successfully

[oracle@testdb1 bin]$

 

Thanks

ADDKS

 

OMS decided to shutdown the agent : EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED

Yesterday I came across an issue when I was installing Oracle Agent 12cR3 on my TEST  environment,  the installation ended with a new error message for me this time. It was for the first time in 3 years while working on Oracle Enterprise Manager I faced this Issue.

This was the error message:- ” ERROR: Agent Configuration Failed SEVERE: emctl start agent command has failed with status1

I have added the complete installation logs to make picture more clear.

[oracle@testdb1 linux_x64]$ ./agentDeploy.sh OMS_HOST=testdb1.oii.test.com EM_UPLOAD_PORT=4903 AGENT_BASE_DIR=/app/oracle/product/agent12c AGENT_REGISTRATION_PASSWORD=formula123

Validating the OMS_HOST & EM_UPLOAD_PORT
Executing command : /app/oracle/product/agent12c/core/12.1.0.4.0/jdk/bin/java -classpath /app/oracle/product/agent12c/core/12.1.0.4.0/jlib/agentInstaller.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/OraInstaller.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12c/core/12.1.0.4.0 /app/oracle/product/agent_sw/linux_x64 /app/oracle/product/agent12c -prereq
Validating oms host & port with url: http://testdb1.oii.test.com:4903/empbs/genwallet
Validating oms host & port with url: https://testdb1.oii.test.com:4903/empbs/genwallet
Return status:3-oms https port is passed
Unzipping the agentcoreimage.zip to /app/oracle/product/agent12c ....
12.1.0.4.0_PluginsOneoffs_226.zip
Executing command : /app/oracle/product/agent_sw/linux_x64/unzip -o /app/oracle/product/agent_sw/linux_x64/12.1.0.4.0_PluginsOneoffs_226.zip -d /app/oracle/product/agent12c
Executing command : /app/oracle/product/agent12c/core/12.1.0.4.0/jdk/bin/java -classpath /app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/OraInstaller.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/xmlparserv2.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/srvm.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/emCfg.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/jlib/agentInstaller.jar:/app/oracle/product/agent12c/core/12.1.0.4.0/oui/jlib/share.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12c/core/12.1.0.4.0 /app/oracle/product/agent_sw/linux_x64 /app/oracle/product/agent12c /app/oracle/product/agent12c/agent_inst AGENT_BASE_DIR=/app/oracle/product/agent12c
Executing agent install prereqs...
Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runInstaller -debug -ignoreSysPrereqs   -prereqchecker -silent -ignoreSysPrereqs -waitForCompletion  -prereqlogloc /app/oracle/product/agent12c/core/12.1.0.4.0/cfgtoollogs/agentDeploy -entryPoint oracle.sysman.top.agent_Complete -detailedExitCodes PREREQ_CONFIG_LOCATION=/app/oracle/product/agent12c/core/12.1.0.4.0/prereqs  -J-DAGENT_BASE_DIR=/app/oracle/product/agent12c
Agent install prereqs completed successfully
Cloning the agent home...
Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runInstaller -debug -ignoreSysPrereqs   -clone -forceClone -silent -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12c/core/12.1.0.4.0   AGENT_BASE_DIR=/app/oracle/product/agent12c OMS_HOST=testdb1.oii.test.com EM_UPLOAD_PORT=4903 AGENT_BASE_DIR=/app/oracle/product/agent12c AGENT_REGISTRATION_PASSWORD=formula123 -noconfig  ORACLE_HOME_NAME=agent12c3 -force b_noUpgrade=true AGENT_PORT=-1
Cloning of agent home completed successfully
Attaching sbin home...
Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runInstaller -debug -ignoreSysPrereqs   -attachHome -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12c/sbin ORACLE_HOME_NAME=sbin12c3 -force
Attach home for sbin home completed successfully.
Updating home dependencies...
Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runInstaller -debug -ignoreSysPrereqs   -updateHomeDeps -waitForCompletion HOME_DEPENDENCY_LIST={/app/oracle/product/agent12c/sbin:/app/oracle/product/agent12c/core/12.1.0.4.0} -invPtrLoc /app/oracle/product/agent12c/core/12.1.0.4.0/oraInst.loc -force
Update home dependency completed successfully.
Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runConfig.sh ORACLE_HOME=/app/oracle/product/agent12c/core/12.1.0.4.0 RESPONSE_FILE=/app/oracle/product/agent12c/core/12.1.0.4.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true

ERROR: Agent Configuration Failed SEVERE: emctl start agent command has failed with status1
Agent Deploy Log Location:/app/oracle/product/agent12c/core/12.1.0.4.0/cfgtoollogs/agentDeploy/agentDeploy_2016-08-04_22-40-59-PM.log

[oracle@testdb1 linux_x64]$

I checked the log file as mentioned in the last line and this I what I came across.

——————————————————————————————————————————————

INFO: Executing command: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runConfig.sh ORACLE_HOME=/app/oracle/product/agent12c/core/12.1.0.4.0 RESPONSE_FILE=/app/oracle/product/agent12c/core/12.1.0.4.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true
INFO: Setting the invPtrLoc to /app/oracle/product/agent12c/core/12.1.0.4.0/oraInst.loc
INFO:
INFO: perform - mode is starting for action: configure
INFO:
INFO: ** Agent Port Check completed successfully.**
INFO: SEVERE: emctl start agent command has failed with status1
INFO: Follow the below steps to manually start the agent and add the host and agent targets:
INFO: 1. Start the agent by executing the command: /app/oracle/product/agent12c/agent_inst/bin/emctl start agent
INFO:  2. Add the targets by executing the command:/app/oracle/product/agent12c/agent_inst/bin/emctl config agent addinternaltargets.
INFO:
INFO: perform - mode finished for action: configure
INFO:
INFO: You can see the log file: /app/oracle/product/agent12c/core/12.1.0.4.0/cfgtoollogs/oui/configActions2016-08-04_10-41-54-PM.log
INFO: Plugin homes:
INFO: Plugin homes:
INFO: /app/oracle/product/agent12c/core/12.1.0.4.0/oui/bin/runConfig.sh ORACLE_HOME=/app/oracle/product/agent12c/core/12.1.0.4.0 RESPONSE_FILE=/app/oracle/product/agent12c/core/12.1.0.4.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true completed with status=3
SEVERE:
ERROR: Agent Configuration Failed
SEVERE:
ERROR: Agent Deployment Failed

 

I followed the steps as mentioned in the log file and tried to start the agent. It failed again, to be honest I was expecting that.   😉

My Concern:-

Same setup is working fine for my production OEM why it is failing again and again for Test OEM environment. I used the same setup to install and configure agent on multiple servers which were being monitored by my PRODUCTION OEM12c and they were installed and configured successfully without any errors.

So why now it was failing for OMS in my TEST OEM12c environment.

Then last option left for me was to explore the trace files and see what exactly is causing issue. So I started digging deeper in the log and trace files until I reached common error message in all of them.


 

My findings.

— EMState agent

----------------------------------------

----- Thu Aug  4 23:01:15 2016::27430::Auto tuning the agent at time Thu Aug  4 23:01:15 2016 -----
----- Thu Aug  4 23:01:16 2016::27430::Finished auto tuning the agent at time Thu Aug  4 23:01:16 2016 -----
----- Thu Aug  4 23:01:16 2016::27430::Launching the JVM with following options: -Xmx128M -server -Djava.security.egd=file:///dev/./urandom -Dsun.lang.ClassLoader.allowArraySyntax=true -XX:+UseLinuxPosixThreadCPUClocks -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+UseCompressedOops -----
----- Thu Aug  4 23:01:16 2016::27430::Agent Launched with PID 27484 at time Thu Aug  4 23:01:16 2016 -----
----- Thu Aug  4 23:01:16 2016::27484::Time elapsed between Launch of Watchdog process and execing EMAgent is 1 secs -----
----- Thu Aug  4 23:01:16 2016::27430::Previous Thrash State(-1,-1) -----
2016-08-04 23:01:16,750 [1:main] WARN - Missing filename for log handler 'wsm'
2016-08-04 23:01:16,757 [1:main] WARN - Missing filename for log handler 'opss'
2016-08-04 23:01:16,759 [1:main] WARN - Missing filename for log handler 'opsscfg'
OMS decided to shutdown the agent because of the following reason sent from OMS:  EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED
----- Thu Aug  4 23:01:29 2016::27430::Checking status of EMAgent : 27484 -----
----- Thu Aug  4 23:01:29 2016::27430::EMAgent exited at Thu Aug  4 23:01:29 2016 with return value 0. -----
----- Thu Aug  4 23:01:29 2016::27430::writeAbnormalExitTimestampToAgntStmp: exitCause=NORMAL : restartRequired=0 -----
----- Thu Aug  4 23:01:29 2016::27430::EMAgent was shutdown normally. -----
----- Thu Aug  4 23:01:29 2016::27430::Exiting watchdog loop
-----

gcagent.log

----------------------
2016-08-04 23:01:19,441 [49:905DEBE9] WARN - improper ping interval (EM_PING_NOTIF_RESPONSE: SHUTDOWN EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED)
2016-08-04 23:01:19,443 [57:CB968A7D:GC.SysExecutor.2 (ShutdownFromPing)] INFO - OMS decided to shutdown the agent because of the following reason sent from OMS:  EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED
2016-08-04 23:01:19,443 [1:3305B9] INFO - Agent is shutting down
2016-08-04 23:01:19,444 [49:905DEBE9] WARN - improper ping interval (EM_PING_NOTIF_RESPONSE: SHUTDOWN EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED)
2016-08-04 23:01:19,444 [1:3305B9] INFO - Invoking SHUTDOWN (1) on Internal Metrics
2016-08-04 23:01:19,444 [1:3305B9] INFO - Done: SHUTDOWN (1) on Internal Metrics
2016-08-04 23:01:19,444 [1:3305B9] INFO - Invoking SHUTDOWN on Miscellaneous
2016-08-04 23:01:19,444 [1:3305B9] INFO - Done: SHUTDOWN on Miscellaneous


I gave one try more but this time with proper procedure. So I followed my own document (blog) for “Silent Installation of Agent” and this time it worked and agent was installed and configured successfully.

What was the issue ? as OMS mentioned in the Error logs.  “ OMS decided to shutdown the agent because of the following reason sent from OMS:  EM_PLUGIN_MISMATCH_AND_AGENT_NOT_YET_MANAGED

SOLUTION:- 

The setup which I used earlier was downloaded from the production environment this was the compatible with EM Plugin in the production not with test environment and this was the reason why it was giving error again and again.

When I downloaded agent from the test environment and deployed it got configured successfully.

 

Procedure is mentioned below:-

————————————————

[oracle@testdb1 bin]$ pwd
/app/oracle/product/middleware/OMS12CR4/oms/bin
[oracle@testdb1 bin]$ ./emcli login -username=sysman -password=formula1
Login successful

[oracle@testdb1 bin]$ ./emcli sync
Synchronized successfully

[oracle@testdb1 bin]$ ./emcli get_supported_platforms
-----------------------------------------------
Version = 12.1.0.3.0
Platform = Linux x86-64
-----------------------------------------------
Version = 12.1.0.4.0
Platform = Linux x86-64
-----------------------------------------------
Platforms list displayed successfully.

[oracle@testdb1 bin]$ ./emcli get_agentimage -destination=/tmp/deepak -platform="Linux x86-64" -version=12.1.0.3.0

=== Partition Detail ===
Space free : 3 GB
Space required : 1 GB
Check the logs at /app/oracle/product/middleware/gc_inst/em/EMGC_OMS1/sysman/emcli/setup/.emcli/get_agentimage_2016-08-05_00-57-38-AM.log
Downloading /tmp/deepak/12.1.0.3.0_AgentCore_226.zip
File saved as /tmp/deepak/12.1.0.3.0_AgentCore_226.zip
Downloading /tmp/deepak/12.1.0.3.0_PluginsOneoffs_226.zip
File saved as /tmp/deepak/12.1.0.3.0_PluginsOneoffs_226.zip
Downloading /tmp/deepak/unzip
File saved as /tmp/deepak/unzip
Agent Image Download completed successfully.
[oracle@testdb1 bin]$ pwd

[oracle@testdb1 agent_sw]$ ./agentDeploy.sh AGENT_BASE_DIR=/app/oracle/product/agent12cr3 OMS_HOST=testdb1.oii.test.com EM_UPLOAD_PORT=4903 AGENT_REGISTRATION_PASSWORD=formula123

Validating the OMS_HOST & EM_UPLOAD_PORT
Executing command : /app/oracle/product/agent12cr3/core/12.1.0.3.0/jdk/bin/java -classpath /app/oracle/product/agent12cr3/core/12.1.0.3.0/jlib/agentInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/OraInstaller.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12cr3/core/12.1.0.3.0 /app/oracle/product/agent_sw /app/oracle/product/agent12cr3 -prereq
Validating oms host & port with url: http://testdb1.oii.test.com:4903/empbs/genwallet
Validating oms host & port with url: https://testdb1.oii.test.com:4903/empbs/genwallet
Return status:3-oms https port is passed
Unzipping the agentcoreimage.zip to /app/oracle/product/agent12cr3 ....
12.1.0.3.0_PluginsOneoffs_226.zip
Executing command : /app/oracle/product/agent_sw/unzip -o /app/oracle/product/agent_sw/12.1.0.3.0_PluginsOneoffs_226.zip -d /app/oracle/product/agent12cr3
Executing command : /app/oracle/product/agent12cr3/core/12.1.0.3.0/jdk/bin/java -classpath /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/OraInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/xmlparserv2.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/srvm.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/emCfg.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/jlib/agentInstaller.jar:/app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/jlib/share.jar oracle.sysman.agent.installer.AgentInstaller /app/oracle/product/agent12cr3/core/12.1.0.3.0 /app/oracle/product/agent_sw /app/oracle/product/agent12cr3 /app/oracle/product/agent12cr3/agent_inst AGENT_BASE_DIR=/app/oracle/product/agent12cr3

Executing agent install prereqs...
Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -prereqchecker -silent -ignoreSysPrereqs -waitForCompletion  -prereqlogloc /app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy -entryPoint oracle.sysman.top.agent_Complete -detailedExitCodes PREREQ_CONFIG_LOCATION=/app/oracle/product/agent12cr3/core/12.1.0.3.0/prereqs  -J-DAGENT_BASE_DIR=/app/oracle/product/agent12cr3
Prereq Logs Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/prereq<timestamp>.log
Agent install prereqs completed successfully

Cloning the agent home...
Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -clone -forceClone -silent -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12cr3/core/12.1.0.3.0   AGENT_BASE_DIR=/app/oracle/product/agent12cr3 AGENT_BASE_DIR=/app/oracle/product/agent12cr3 OMS_HOST=testdb1.oii.test.com EM_UPLOAD_PORT=4903 AGENT_REGISTRATION_PASSWORD=formula123 -noconfig  ORACLE_HOME_NAME=agent12c4 -force b_noUpgrade=true AGENT_PORT=-1
Clone Action Logs Location:/app/oracle/oraInventory//logs/cloneActions<timestamp>.log
Cloning of agent home completed successfully

Attaching sbin home...
Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -attachHome -waitForCompletion -nowait ORACLE_HOME=/app/oracle/product/agent12cr3/sbin ORACLE_HOME_NAME=sbin12c4 -force
Attach Home Logs Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/AttachHome<timestamp>.log
Attach home for sbin home completed successfully.

Updating home dependencies...
Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runInstaller -debug   -ignoreSysPrereqs -updateHomeDeps -waitForCompletion HOME_DEPENDENCY_LIST={/app/oracle/product/agent12cr3/sbin:/app/oracle/product/agent12cr3/core/12.1.0.3.0} -invPtrLoc /app/oracle/product/agent12cr3/core/12.1.0.3.0/oraInst.loc -force
Update Home Dependencies Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/UpdateHomeDeps<timestamp>.log
Update home dependency completed successfully.

Performing the agent configuration...
Executing command: /app/oracle/product/agent12cr3/core/12.1.0.3.0/oui/bin/runConfig.sh ORACLE_HOME=/app/oracle/product/agent12cr3/core/12.1.0.3.0 RESPONSE_FILE=/app/oracle/product/agent12cr3/core/12.1.0.3.0/agent.rsp ACTION=configure MODE=perform COMPONENT_XML={oracle.sysman.top.agent.11_1_0_1_0.xml} RERUN=true
Configuration Log Location:/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/cfgfw/CfmLogger<timestamp>.log
Agent Configuration completed successfully

The following configuration scripts need to be executed as the "root" user.
#!/bin/sh
#Root script to run
/app/oracle/product/agent12cr3/core/12.1.0.3.0/root.sh
To execute the configuration scripts:
  1. Open a terminal window
  2. Log in as "root"
  3. Run the scripts
Agent Deployment Successful.
Agent deployment log location:
/app/oracle/product/agent12cr3/core/12.1.0.3.0/cfgtoollogs/agentDeploy/agentDeploy_2016-08-05_01-06-08-AM.log
Agent deployment completed successfully.

[oracle@testdb1 agent_sw]$ cd /app/oracle/product/agent12cr3/agent_inst/bin/
[oracle@testdb1 bin]$ ./emctl secure agent
Oracle Enterprise Manager Cloud Control 12c Release 3
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
Agent successfully stopped...   Done.
Securing agent...   Started.
Enter Agent Registration Password :
Agent successfully restarted...   Done.
EMD gensudoprops completed successfully
Securing agent...   Successful.

[oracle@testdb1 bin]$ ./emctl config agent addinternaltargets
Oracle Enterprise Manager Cloud Control 12c Release 3
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.

[oracle@testdb1 bin]$ ./emctl upload
Oracle Enterprise Manager Cloud Control 12c Release 3
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully
[oracle@testdb1 bin]$

This time as well I learnt something new and I thought of sharing it with you so I did. Hope this will help you some time in future.

Thanks
ADDKS

Add Targets in OEM12c Declaratively specifying target monitoring properties.

In my last blog I have explained a step by step procedure, how we can add a target in Oracle Enterprise Manager 12c using “GUIDED” process. In this blog I have explained how we can add target in OEM12c using “DECLARATIVE” process. Let first understand what is the difference between “GUIDED” and “DECLARATIVE” process first.

In GUIDED process we have seen that agent on that host helps this process to discovers the target of particular type  that meets the selection criteria like in this case “Oracle Database, Listener and Automatic Storage Management”, once all these targets are discovered you just need to put the credentials and other monitoring configuration will be already filled during discovery.

However if we talk about the “DECLARATIVE” process, in this process we limit our input information and we want OEM to discover targets based on the limited information supplied to it. Lets see how this process works. Mentioned below are the steps:

Step 1: Navigate to the page “Add Targets Manually“.

image1

Step 2: Select method “Add Targets Declaratively by Specifying Target Monitoring Properties” and select the “Target Type” .

 

image4

Step 3: Select the Monitoring Agent. {The host on which the database instance/target (in this case) is located.}

image5

Step 4: Click on “Add Manually” to proceed further.

image6

Step 5: Add database instance properties, this includes not only the Monitoring configuration but also the “Target Name” and “Database System“.

image7

 Note:  The major difference between GUIDED Process and Declarative Process comes at this step. In Guided Process we just have to specify the password for monitoring user{DBSNMP}, rest of the information was filled automatically when Agent discovers the target. However while adding target by declaratively specifying the properties, we have to manually enter all these values.

Step 6: Test Connection, once you have filled all the values and make sure the connection is successful. Click NEXT to proceed further.

image8

Step 7: Review details and click on “Submit” to add this target.

image9

Now you can find the target listed under the “Targets -> Databases“.

 

Regards

ADDKS

Add Targets in OEM12c Using Guided Process

In this Blog I have explained step by step procedure to discover and add targets  on a HOST on which Agent is already installed.

Note: Oracle Agent installation is mandatory on the host for which you want to discover and push targets in OEM.

  • Navigate to the “Add Targets Manually” screen using the menu at the top-right of the screen (Setup > Add Target > Add Targets Manually).

image1

  • Select the “Add Targets using Guided Process” option, select the target types to be discovered (eg. lets say in this blog I have taken “Oracle Database, Listener and Automatic Storage Manager”) and click the “Add Using guided Discovery…” button.

image10

 

  • Under “Database Discovery: Search Criteria Agent” enter the host name on which target exists and agent in installed. Click the “Continue” button.

image11

 

  • Click on “Next” button.

image12

  • Click the “Configure” icon for any discovered targets and enter the required details.

image13

image14

  • Specifying Global Target Properties for Notifications and Administrations Groups and click “OK”.

image15

  • When all the configuration steps are complete, click the “Submit” button, followed by the “Save” button, then finally the “OK” button.

image9

You can now see the recently added “database Instance” target listed on the relevant target {Databases} screen (Targets > Databases).

 

Thanks

ADDKS

 

OTNYathra2016 – My first Experience as Speaker

In this blog I have share my experience of OTNyathra2016 as a SPEAKER.

It was for the first time when beside the managing the entire event along with my Gurus Mr. Aman Sharma and Mr Navneet Upneja, I was there as a Speaker in the same event. Yes for the first time in a National Level Event I was nominated as Speaker and I had to deliver session on “Oracle Enterprise Manager 13c“.

Was I worried, YES a lot. It would sound weird but there were lots of ups and downs going on in my mind. I thought many  times to take my name back but then I took it as challenge and  thought no matter what I will speak. I knew some day I have to start so why not now.

Moreover it was for the first time being the Regional chapter of parental body AIOUG, AIOUG-North India Chapter took the responsibility for conducting this event. There was lot of pressure on our minds but yes with the hard work of entire team of North India Chapter our event was a GRAND SUCCESS. We broke the record of last 7 years with attendance of 130 delegates.

Special thanks to our Volunteers who helped us promoting this event on Social Media. Thank You so much guys. {Pavan Mishra, Vijay, Vinay and Anoop}

 

Back to experience as Speaker. 🙂

My session was at the last, when people are in mood of just rushing back to their homes. Few of them already left,  well I would say it was not actually a BAD start, there were around 20 people sitting in that room to attend my session. Before starting my session I had made my mind that I will not get confused, worried or shiver and I guess I succeeded in that. My both Gurus Mr Aman Sharma and Mr Navneet Upneja, came there in my session one by one.

The topic was new “Oracle Enterprise Manager 13c – New Features”, of course this release has a lot of new feature but I just took three domain and explain new or enhanced things in them. There were many cross questioning from the delegates some of which I was able to answer and some of them I did not know as I myself had not explored that feature. Overall it all went well and I would say my first experience was not that Bad to get upset.

My mentors Aman and Navneet both told me about some flaws and area of improvements, and I’ll  make sure not to make those mistakes again. I would like to thank both of them for there forever support and guidance.

 

Will be back soon with stuff on OEM13c, till then TC.

 

DSC_0315

Older posts

© 2017 ORACLE EM DBA

Theme by Anders NorenUp ↑