WLSDM: Smart Dashboard & Monitoring Console Help

Native WebLogic Monitoring & Diagnostics

WLSDM Help

v3.2.2 | Released on 2017.11.04

Introduction

WLSDM is an enterprise "WebLogic console extension" which enables monitoring for WebLogic JMX MBean metrics, all the WebLogic domain assets (Health, Servers, Applications, Data Sources, JMS… etc.) and back-end systems monitoring (JDBC, EJB, JAXWS WebServices, Servlets). It is very easy to create alarm and notification definitions by using WLSDM metric browser. WLSDM can store any WebLogic metric values historically and also can generate graphical reports. WLSDM is a WebLogic Console extension and runs under WebLogic console on Admin server.

1. Important WLSDM Features (WebLogic Monitoring & Management)

2. WLSDM Functionality and Easy Usage

By using WLSDM, there is no need to write WLST scripts for getting WebLogic MBean metrics and scheduling it. Let WLSDM take care everything. WLSDM can access any values on WebLogic console.

3. Administrative and DevOps WLSDM Features


Tips and Tricks: Access WLST Web Console and run custom WLST scripts by using WLSDM's WLST Web Console. It is possible to store and centralize custom WLST scripts on WLSDM.

4.WLSDM SOA Module: Monitoring, Diagnostics & Reports

Completing WLSDM Wizard

WLSDM has a robust wizard which is aimed for setting up a WebLogic domain’s monitoring configurations easily and fast. This wizard is active only for the first login.WLSDM reads the WebLogic domain and offers important metrics to monitor.

WLSDM Quick Configuration Wizard

Step 1: Welcome: Wizard introduction

WLSDM Wizard Banner

Step 2: License Information

After accepting the license agreement, WLSDM offers two options:

  1. Evaluate for 30 days:

    If you do not have a valid WLSDM license key, choose this option for the evaluation. After 30 days WLSDM will ask for a valid license.

  2. Enter license key:

    If you have a valid WLSDM license key, choose this option and enter the appropriate license information. The license key will be checked off-line and activated instantly if it is a valid key.

Step 3: Alarm Configuration

In step 3, the SMTP settings and email recipients can be configured easily. These settings can be left blank, but it is highly recommended to configure these settings successfully.

By setting up email configuration WLSDM will send notifications and HTML email alerts.

Use "Send" button for testing SMTP settings. If you configured the SMTP settings successfully, you will get a test email as below.

wizard smtp settings wlsdm weblogic monitoring
If any error occurs then check wlsdm.log file which is located under $WL_DOMAIN_HOME/WLSDM/logs folder.

Step 4: Reading Domain

WLSDM reads WebLogic domain and lists the domain resources to configure monitoring settings.

Step 5: Domain Resources

There are four main groups which WLSDM offers to monitor: Servers, Deployments, Data Sources and JMS

Select domain resources to monitor. All important domain resources are selected by default. These can be unselected in this step.

reading domain wlsdm weblogic monitoring

Step 6: Metric Configuration

This is most important step and highly recommended to complete carefully. WLSDM offers important WebLogic MBean metrics to monitor according to domain’s available resources.

Complete Step-6 settings as:

1. Check metric values
All available metrics are grouped by WebLogic servers. All the sub instance threshold values can be set on main metric record. Click on threshold edit-box and see current instance values.

metric configuration wlsdm weblogic monitoring
2. Set threshold values
Give reasonable threshold values according to the WebLogic domain characteristic.
3. Enable/Disable alarm
It can be done by using ON/OFF button on "Alarm" column.
4. Delete metric instance
It is also possible to remove the metric instance.
5. Reset values and configurations
All the configurations, values and deletion processes can be reset by clicking on refresh icons on metric group tabs.

Step 7: Monitoring & Diagnostics

There are three Monitoring&Diagnostic settings in this step. These are "Log Inspector", "Response Times Monitoring", "Back-end Systems" monitoring configurations.

Firstly, select profiling servers for Monitoring & Diagnostics.


monitoring and diagnostics wlsdm weblogic monitoring

Type Description Production Systems Best Practice
Log Inspector Select at least one profiling server. Select only one managed server among managed servers has same duty and settings. (i.e. cluster members)
Response Times Select at least one profiling server. Select only one managed server among managed servers has same duty and settings. (i.e. cluster members)
Back-end Systems WLSDM agent must be installed. Only supported and WLSDM agent installed servers are listed. Select at least one profiling server. Select only one managed server among managed servers has same duty and settings. (i.e. cluster members)

All the Monitoring & Diagnostics settings (thresholds, WLDF queries, back-end events... etc.) can be edited on "Configuration > Monitoring & Diagnostics" page later.

Step 8: Finish

It is unable to access to the wizard page after completing WLSDM Quick Configuration Wizard steps. But all the configurations and values can be listed or updated on Smart Dashboard, "Configuration > Metric Browser", "Configuration > Monitoring & Diagnostics" pages.

Runtime {WLSDM} Folder Structure

When WLSDM installed to a WebLogic domain, it creates "WLSDM" folder under WebLogic domain's home folder. Directory layout of WLSDM folder:

wlsdm folder layout
wlsdm folder layout v2

Folder Name Relative Path Description
db ${WL_DOMAIN_HOME}/WLSDM/db WLSDM's embedded database folder. WLSDM uses embedded Apache Derby Database which comes with wlsdm.war file. There is no operational cost for maintaining WLSDM database, the console extension takes care everything.
config ${WL_DOMAIN_HOME}/WLSDM/config Configuration folder which consists of html email "templates" folder and main WLSDM config file wlsdm.config.xml
config/templates ${WL_DOMAIN_HOME}/WLSDM/config/templates HTML email templates folder. Edit these files for your HTML customizations.
logs ${WL_DOMAIN_HOME}/WLSDM/logs WLSDM default log folder. There are two log files, one is wlsdm.db.log the other one is wlsdm.log which is the main log file.
temp ${WL_DOMAIN_HOME}/WLSDM/temp This folder is using by WLSDM log inspector module for sending alert emails which have log attachments.
dumps ${WL_DOMAIN_HOME}/WLSDM/dumps WLSDM pofiling dump files are stored in this folder. Sub folders are: threaddump, jfr, heap, diagnostic_images
scripts ${WL_DOMAIN_HOME}/WLSDM/scripts Used by WLST Web Console and WLST scripts are stored in this folder.
archive ${WL_DOMAIN_HOME}/WLSDM/archive WLSDM archive folder. Purged data are stored in this folder.
monitorWLAdmin ${WL_DOMAIN_HOME}/WLSDM/monitorWLAdmin Standalone Java application for monitoring WebLogic Admin Server. Works as plug and play!

WLSDM Console Usage

1. General Usage

Main WLSDM menu is located on the left side and default is collapsed. It can be expand/collapse by menu icon.

main menu wlsdm weblogic monitoring

Page operations icon is located on the right top of every pages. Visual page settings and page functions can be list on page operations menu.

page operations1 wlsdm weblogic monitoring

Summary metric notification window is located on the right top of every page.

notifications1 wlsdm weblogic monitoring

2. Smart Dashboards Usage

Health Dashboard

Health dashboard page is the home page of WLSDM and lists all the WebLogic domain resources at one page. All the states and health values is listed for per WebLogic server instance.


health page wlsdm weblogic monitoring

Health Notifications

WLSDM checks these health and states periodically. If the health is not "OK" and state's value is not "RUNNING" for any resource, WLSDM generates "Health ALARM" and sends email notifications as below.


server health wlsdm weblogic monitoring

Using Health Dashboard

Health alarm notifications can be enabled/disabled for each resources (Servers, Applications, Data Sources and JMS) by clicking on alarm icon on the lists.


health alarm enable wlsdm weblogic monitoring

Default Dashboards

Servers, Deployments, Data Sources and JMS Servers dashboards are default dashboards which are created at wizard steps.

Dashboard page operations are as below:

  • Can update dashboard name
  • Can change "Metric Load Range" historically (Minimum 1 minute)
  • Can change automatic "Page Refresh Frequency" (Minimum 30 seconds)
  • Can change number of charts per line (Minimum 1 chart)
  • Can change chart heights (Minimum 100px)
  • Can change grid display status (Show / Hide)

dashboard page operations wlsdm weblogic monitoring

Chart Usage on Dashboard Pages

All metric and chart operation icons are located on the top right of charts.

metric graph options wlsdm weblogic monitoring
metric and chart options wlsdm weblogic monitoring
  • View Current Hogger/Stuck Threads: Opens HOGGER/STUCK threadpool modal window and prints HOGGER/STUCK threads. Available for HoggingThreadCount and StuckThreadCount Charts
  • Metric JMX MBean Information: Opens JMX MBean Information window and shows MBean object documentation
  • Reload Data: Reloads metric values for the current chart (Loads the current values of WebLogic MBeans)
  • Metric & Chart Options: Opens metric options modal window. Configure metric settings, actions, thresholds... etc.
  • Open Metric Reports (New Window): Opens Reporter page and queries historical JMX MBean data from WLSDM database
  • Print: Print window is opened and it is possible to save the charts as PDF document
  • Show/Hide: Opens/Closes chart display
  • Close: Removes the chart occurrence on the page. When page refreshes the chart is loaded again.
  • Drag and Drop: Use drag button to rearrange chart placement

Metric and Chart Options

Opens the "Metric and Chart" Options window. All the operations can be done on this window easily.

  • Can change "Metric Chart Type"
  • Alarm settings (ON / OFF)
  • Action settings (Thread Dump, Heap Dump, Java Flight Recorder Dump, WLDF Diagnostic Image)
  • Synchronization settings (Controlling extended WebLogic servers)
  • Monitoring metric instance health
  • Can change metric's dashboard settings to show on which dashboard to display

    Important Feature: A metric can be assign multiple dashboard on "Edit Dashboards" modal window as below.

    assign_dashboards_wlsdm_weblogic_monitoring

Generic DevOps MBean

WLSDM DevOps MBean Logic?

WLSDM generates a Dynamic MBean instance by using com.wlsdm.genericmbean.DevOpsMBean class and creates / populates its MBean attributes by user defined script output. All user defined scripts are stored in the WLSDM database and each script works once for each selected server through monitoring frequency. Each script / SQL statement has its own execution timeout and this value cannot be less than one second or cannot be more than thirty seconds. (Recommended / Default value is five seconds)

Generated DevOps MBean will be assigned to selected smart metric dashboard(s) on the configuration wizard and chart(s) will be visible on its smart dashboard. These MBean metrics are alive like any other MBean attributes (WebLogic metrics i.e. OpenSocketsCurrentCount).

MBeans are the managed Java objects and they represent Java classes. A Java class cannot contain two or more attributes with the same name. Script outputs are the source of MBean attributes. There two options to place script outputs in charts.

1. Attributes in Different Chart: Each script output attribute will be placed on different charts. Generates one same MBean instance with the multiple MBean attribute name.

2. Attributes in Same Chart: All the script output attributes will be place in the same chart. Generates multiple different MBean instance with the same MBean attribute Name. This option creates multiple DevOps MBean instances for each attribute with the given name, i.e. ${YourMBeanName}_${AttrName}, on wizard.

Single instance multi attribute MBean
Multi instance single attribute MBean

DevOps MBean generation and monitoring threads are independent and different processes. Each process has its own process controller and execution frequency. DevOps monitoring frequency value is editable on " Configuration > WLSDM System Settings" page.

Script Types and Usage


1. OS Executables: sh, bash, bat, cmd

Key / value pairs are the expected output format for the OS executable scripts. Sample output(echo) is listed below:

#attributeName=attributeValue
TotalCompletedOrders=2017
TotalWaitingOrders=35

Note: MBean attribute names cannot contain non alphanumeric characters except underscore ( _ ). Non-alphanumeric characters must be replaced.

Sample Shell Script - 1 : Disk usage of a Linux server

#Author: WLSDM
df -H | grep -vE '^Filesystem|tmpfs|lv|dev' | awk '{ print $5"="$4}' | sed 's/%//' | sed '1s/\//ROOT/'

Sample Shell Script - 1 Output :

ROOT=85
data=67
u01=76

Sample Shell Script - 2 : Ping database and calculate database respone time using WebLogic API (weblogic.jar)

#Author: WLSDM | 2017

#Variables
_dbStatus=0 
#0:DB_DOWN,1:DB_UP

#start DB Ping
_startTime=$(($(date +%s%N)/1000000))
_pingResult=`java -cp weblogic.jar utils.dbping ORACLE_THIN $YOUR_USER_NAME $YOUR_PASSWORD $YOUR_IP:$YOUR_PORT/$YOUR_SERVICE_NAME`
_endTime=$(($(date +%s%N)/1000000))
_dbResponseTime=`expr $_endTime - $_startTime`

if [[ "$_pingResult" == *"Success!!!"* ]]; then
  _dbStatus=1
  #echo "Database connection SUCCESS!!!"
  #DB_UP
  echo DatabaseStatus=$_dbStatus
else
  _dbStatus=0
  #echo "Database connection FAILURE!!!"
  #DB_DOWN
  echo DatabaseStatus=$_dbStatus
fi
echo DatabaseResponseTime=$_dbResponseTime

Sample Shell Script - 2 Output :

DatabaseStatus=1
DatabaseResponseTime=1218

2. SQL Script

WLSDM supports two parsing method for SQL script results as MBean attribute generation. As OS Executable Script output, key / value pairs are the expected output for SQL statement results.

  1. Horizontal Fetch (Column Name Based)
  2. Vertical Fetch (Row Value Based)
WLSDM SQL Script Result Parse Methods


2.1. Horizontal Fetch (Column Name Based)

Table column or SQL statement result column names are accepted as MBean attribute name (key) and first row of column values are accepted as MBean attribute value. Column names and column orders must be static and SQL result set must have only ONE row data.

select 
    AttributeName1, AttributeName2 
from 
    MyTable
where
    ROWNUM=1
AttributeName1 AttributeName2
AttributeValue1 AttributeValue2

2.2. Vertical Fetch (Row Value Based)

Each row (in result set) accepted as attribute key / value pair. First column value is attribute name and second column value is attribute value.

select 
    Column1, 
    count(*) as Column2 
from 
    MyTable
where 
    Column1 <= 3 -- static result
group by Column1
order by 1
Column1 Column2
AttributeName1 AttributeValue1
AttributeName2 AttributeValue2
AttributeName3 AttributeValue3

Sample SQL Script : Active sessions grouped by machine (Oracle Database)

select 
    status, 
    count(*) 
from 
    gv$session 
group by status
order by 1
STATUS COUNT(*)
ACTIVE 54
INACTIVE 21

HOW TOs

1. How to export/import WLSDM settings, definitions and configurations for multiple WLSDM installation as a standard setup?

  1. Go to "WLSDM System Settings" page
  2. Open "Page Operations" menu then click "Export WLSDM Configuration" button
  3. Switch to new WLSDM installation in different/antoher WebLogic domain
  4. In the begining of "WLSDM Wizard" click "Import WLSDM Config" button then upload the exported file

2. How to add JMX MBean objects (metrics) to dashboards via WLSDM MBean Browser?

  1. Go to "Metric Settings & JMX Browser" page
  2. Click "Add New JMX MBean Metrics" from page operations menu
  3. Search metric then select "Server, Metric Group and Metric" from the MBean Browser
  4. Configure metric options then click "Add To List" button to "Save"
  5. Newly added metric is going to be available in the assigned smart dashboard(s)

3. Smart Dashboards Display Settings

  1. Go to "Smart Dashboards" pages (i.e. Smart Dashboards > Servers)
  2. Configure smart dashboards display settings by using "Page Operations" menu
  3. Q: How to reorder metric charts? A: Click "Reorder Metric Chart Positions" button to open "Reorder Metrics" modal window
  4. Drag and drop metrics and rearrange orders in "Reorder Metrics" modal window

4. How to change "Back-end Monitoring" alarm thresholds?

  1. Go to "Monitoring & Diagnostic Page"
  2. Go to "Back-end Systems" tab and click "Edit button"
  3. Edit alarm thresholds in "Modal Window"

5. How to create "Custom Log Monitoring" in LogInspector?

  1. Go to "Monitoring & Diagnostics" page
  2. Click "Add New Log Monitoring" from page operations menu
  3. Set "Log Inspector Monitoring Name" then choose "Log Type" as "Create New"
  4. Add "Search String"
  5. Select server(s) to monitor log files
  6. Configure tail line count and delivery settings
  7. Save newly added "Log Monitoring" definition

6. How to create "User Defined Actions / Scripts"?

  1. Go to "Monitoring & Diagnostic" page
  2. Open "Page Operations" menu then click "New User Defined Actions / Script" button
  3. Set name and executable target file path then click "Save"
  4. Go to Smart Dashboards (i.e. Servers) and open "Metric & Chart Options" window for the relevant MBean (i.e. StuckThreadCount)
  5. Click on "Actions - Control Field" button and open "Actions" modal window
  6. Select newly added "User Defined Action" (i.e. RestartManagedServer) and move to the right "Applied Actions" box
  7. Save action. If the metric value exceeds the threshold new actions will be fired/executed asynchronous by WLSDM

7. How to activate "Downtime Job" for the planned WebLogic maintenance works to stop WLSDM monitoring or/and notifications?

  1. Go to "Monitoring & Diagnostics" page
  2. Click "New Scheduled Job / Downtime" button from page operations menu
  3. Select "Downtime" and configure downtime settings
  4. Downtime will be activated when the cron time is valid