Mimic GGSCI with Python

Since the release of Oracle GoldenGate 12.2 (Classic) and Oracle GoldenGate 12.3 (Microservices), users have been able to interact with Oracle GoldenGate using RESTful APIs.  I’ve written about this interaction in previous posts.  With the option to use RESTful APIs, the flexibility that is introduced into Oracle GoldenGate is huge.  Everything from simple administration to daily monitoring can be scripted.  With this in mind, I decided to finish some items I tabled a few years ago (more will come).  

As I’ve been doing a pure Oracle GoldenGate (Microservices) project recently, it is interesting to see how quickly the number of deployments can multiple.  In a single project; running a hub-architecture, there are thirty-two (32) deployments on a single host.  Out of these thirty-two deployments, there are eight (8) Adminstration Services with an extract and replicat in each.  This means there are sixteen (16) extracts and replicats to be monitored.  Being able to quickly view these is key to supporting the environment.

Note: Obey files can be written to do something similar, but the down side is that the password has to be passed in clear text within the obey file.

With obey files being a bit clunky, scripting languages like Perl, Go, and Python are a natural choice to take over.  In this case, I chose Python.   By using Python, I’m able to quickly take the REST APIs and consume them.  Then have the output displayed similar to what use to be in GoldenGate Service Command Interface (GGSCI) (below).

In this format, I can quickly see the status, lag, and lag at checkpoint of the extract and replicats that are running in the deployment. I also added the metric of the last time the process was started.  In the end, by using the REST APIs, I or even you can extend Oracle GoldenGate and bend it to your needs.  The script that I wrote for this purpose is below.  You are more than welcome to use it as written at your own risk.  

REST API Info

If you want more information on the Oracle GoldenGate REST APIs, visit the link here.

Python Script

To use the below script, you will need to have Python3 installed and then install the “requests” module using the pip install process.  

#!/usr/bin python
# Copyright (c) 2018-2019 Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
#
# Since: March 2020/Jan 2023
# Author: Bobby Curtis <[email protected]>
# Description: Check GoldenGate Processes
#
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER.
#

import requests
import json
import os

vUrl = ["https://<hostname_or_ip>:<port>"]
vAdmin = “<gg_user>"
vPass = “<password>"

requests.packages.urllib3.disable_warnings()

def cls():
	os.system('clear')

def display_header(url):
	print("#############################################################”)
	print(" GoldenGate Process Status : " + url)
	print(“")
	print ("#############################################################")

def proc_status (proc_name, url, proc_type):
	if (proc_type == "extract”):
		infoUrl = url + "/" + proc_name + "/info/status”
		#print(infoUrl)
		response1 = requests.get(infoUrl, verify=False, auth=(vAdmin, vPass))
		jsonResponse = response1.json()
		vStatus = jsonResponse['response']['status’]
		vLastStart = jsonResponse['response']['lastStarted’]
		vLag = jsonResponse['response']['lag’]
		vLagAtCheck = jsonResponse['response']['sinceLagReported’]
		print("Process Name\t" + "Status\t\t" + "Lag\t" + "\tLag At Check\t" + "Last Start Time”)
		print(proc_name.upper() + "\t\t" + vStatus.upper() + "\t\t" + str(vLag) + "\t\t" + str(vLagAtCheck) + "\t\t" + vLastStart)
		#print(json.dumps(response1.json(), indent=4))
	else:
		infoUrl = url + "/" + proc_name + "/info/status”
		#print(infoUrl)
		response1 = requests.get(infoUrl, verify=False, auth=(vAdmin, vPass))
		jsonResponse = response1.json()
		vStatus = jsonResponse['response']['status’]
		vLastStart = jsonResponse['response']['lastStarted’]
		vLag = jsonResponse['response']['lag’]
		vLagAtCheck = jsonResponse['response']['sinceLagReported’]
		print(proc_name.upper() + "\t\t" + vStatus.upper() + "\t\t" + str(vLag) + "\t\t" + str(vLagAtCheck) + "\t\t" + vLastStart)
		#print(json.dumps(response1.json(), indent=4))

def check_status (proc_type):

if (proc_type == "extract”):
	display_header(xUrl)
	hubUrl = xUrl + "/services/v2/extracts”
	#print(hubUrl)
else:
	hubUrl = xUrl + "/services/v2/replicats”
	#print(hubUrl)

response = requests.get(hubUrl, verify=False, auth=(vAdmin, vPass)).text
response_info = json.loads(response)
#print(response_info)

i = 0
while i < len(response_info['response']['items']):
  proc_status(response_info['response']['items'][i]['name'], hubUrl, proc_type)
  i += 1

#Main
cls()
for xUrl in vUrl:
	check_status("extract”)
	check_status("replicat")

The script above can be extend to monitor more than one deployment by adding string values to the vURL variable.  For every Administration Service plus port number added in the array will return the output for the dpeloyment and the extracts and replicats that is running there.  What this output will not show you the status of the Oracle GoldenGate Services that would be seen through AdminClient.  Meaning there is a give-n-take on the approach you are trying to monitor with this python script.

There are more improvements that can be done to this script, but for the project where this will be use it provides the needed output.

Enjoy!!

Please follow and like: