WMB for beginners

WMB is an ESB from IBM. To learn WMB or any ESB tool, you need to understand the concepts of ESB and Integration and then the vendor specific products. This post is for those who are new to ESB world and is looking for some instructions or directions to learn WMB basics.

To begin with, go through the basics and concepts of Integration and service Bus. Read the below articles to learn concepts of application integration and messaging patterns.

EAI – enterprise application integration
EAI Basics
messaging patterns
EAI vs ESB vs SOA

10 Key features in IIB V 10

IBM Integration Bus V10 was released quite some time from now. IIB V10 is the second release of Integration Bus family after Websphere message broker is re branded as IBM integration Bus. The new release is having some popular new features. Popular in the sense, these features were demanded over communities or RFEs.

I would like to write the 10 new features which I like the most from IIB V10 in this post. These are my personal choice and I would not say these are the only key features in IIB V10.

1. Removal of the WebSphere MQ prerequisite

This is one of the strategical move to decouple two IBM products. Like few other vendor ESB products, messaging engine is not tightly coupled with the ESB. This can be a nice feature where an organization is not using IBM MQ for their MOM, but heavily uses Webservices or other protocols.

Monitor and Capture Queue depth on WMQ

Introduction

Monitoring a queue on WMQ. This is a kind of very basic activity on an MQ infrastructure. And there are plenty of sample programs written in Java and other languages available free on internet to get a queue depth of a particular queue. Websphere MQ installation itself provides sample programs on various languages and is available under the WMQ Install Dir/Tools.

The program here is of no much difference. Its a sample utility to monitor a queue for a particular time period on particular interval and capture the metrics in to an excel sheet for easy reference. It found useful during a performance testing on WMB/WMQ where you are loading 10000’s of messages at a time to queue and would like to capture the details of the message consumption rate.

Monitor queue depth using Java

This utility program written in Java is having property file used to configure the  MQ Env details such as Queue manager name, host name,  port etc. and the monitoring time details such as duration and interval in seconds. This utility used WMQ Java PCF API to connect to Queue manager and collect the queue depth and queue handle (open input count and output count) details. These details will be written to an excel sheet for easy analysis.

You can package this PI to an executable JAR file and can use to monitor multiple queues parallel.

Processing Accounting and Statistics data for WMB

Introduction:
Accounting and statistics data in WMB will help us to get the performance metrics of the message flow such as number of threads, max time to process a message, message max size, CPU utilization, product performance, how many messages processed etc. These data can be taken in a granular level (node level) as well. Data can be captured for a defined duration to analyse the performance metrics.
Accounting and statistics data can be captured using WMB explorer or by commands. And the data captured can be viewed in granular/tabular form via WMB explorer.

The performance impact is minimal when collecting these data with basic level. But when we capture advanced level (Node level) details, performance impact may occur.

Challenges:
Collecting and viewing the statistics via WMB (MQ) explorer may not provide the results properly as it may hang for some time or user interruptions are required to save the metrics, pause, stop etc in the WMB explorer. When administrating a remote broker, it may be too slow to display the snapshot data in the explorer.

Solution:
To overcome these, we can capture the accounting data using the command line utilities. By using the commands, we can capture either snapshots (every 20 secs emitted) or archives (every 60 mins by default) accounting info.
By default the archive interval is 60 min. However it can be changed by mqsichangebroker command
Eg:

mqsichangebroker  -v statisticsMajorInterval

IIB V 9.0, System Administration- IBM Certification (C2180-405)

This is about the IBM certification for IBM Integration Bus Administration. Recently I got the opportunity to took this certification and I am sharing the experience about the same.

About the test
The test is structured like the other IBM Websphere certifications. There’s 56 questions of which you need 34 to pass. There are 5 categories represented:

Section 1 – Planning, Installation and Migration (14%)
Section 2 – Administration and Configuration (32%)
Section 3 – Security (18%)
Section 4 – Performance Monitoring and Tuning (16%)
Section 5 – Problem Determination (20%)

Refer the certification page Certification page

Preparing for the certification:

The exam is not that surprising and if you follow the certification topics listed on http://www-03.ibm.com/certify/tests/eduC2180-405.shtml religiously.

Performance testing of IIB Flows using JMeter

Introduction

JMeter is an open source testing software from Apache Software foundation. JMeter can be used to test functional, regression/load/performance cases. The common tools which are used to test IIB flows like RFHUtil, SOPAUI, NetTool etc are found very good for unit testing or functional testing scenarios. JMeter fits well for load testing or automated testing of IIB flows.

This article is not intended to provide the details of JMeter software or how to use it. This would like to cover how to configure JMeter to do the load testing of IIB message flows.

JMeter can be download from Apache Website. Follow the steps mentioned in the User manual to configure JMeter for your environment.

Test Scenarios
JMeter can be used to test the commonly used protocols such as MQ, HTTP, SOAP, TCP etc. Hence the IIB flows having input as any of these protocols can be taken for load testing using the JMeter.

A JMeter Test Plan may contain Thread Groups, logic controllers, sample generating controllers, listeners, timers, assertions, and configuration elements. Refer the JMeter User manual to know about each of these components.

In the following examples, we are using Thread groups, Samplers, Listeners elements frequently.
There are four testing scenarios covered here to address load testing of IIB flows. These were based on the input node’s protocol used. In the coming sections, we will cover how to configure JMeter to do the load testing of message flows with Input as MQ Input, HTTP Input, SOAP Input and TCPIP Server nodes.

Message flow transactions in IIB

Transaction handling in IIB is a topic which is widely discussed in forums and is been well documented in Info center as well. I am just trying to summaries those information here in this topic based on a request from a reader.

A transaction can be defined as set of activities that are executed as a Single Unit Of Work. Means, if any one of the actions got failed, all the actions which are already performed will be rolled back. A transaction can be termed as success only when all the individual actions got processed without any error.

According to IBM info center, a transaction can be termed as

A transaction describes a set of updates that are made by an application program, which must be managed together. The updates might be made to one or more systems. The updates made by the program are controlled by the environment in which the program executes, and either all are completed, or none. This property of a transaction is known as consistency: transactions might have other properties of atomicity, isolation, and durability.

The normal terms which are used when describing the transaction behavior in general are :
Transaction manager
The resource which co ordinate with all other resources in the Unit Of Work. The transaction manager will be able to co ordinate or communicate with all the other resources involving the Unit Of Work. Example is a Websphere Queue Manger which will act as a transaction manager in XA coordinated transactions with supported database resources. Here the Queue manager act as the transaction manager for MQ and DB resources.
Note: The default transaction manager in distributed systems is the broker and is known as broker- coordinated transactions or partially coordinated transactions.
If the coordination is provided by an external transaction manager (MQ or DB) which supports XA protocols to interact with individual resource managers, is known as Globally coordinated transactions.
You need to perform additional setup to configure the globally coordinated setup.
The external transaction manager causes a performance overhead over the default transaction manager (broker) set up.
In z/OS, the transactions are always Globally coordinated by default.

2014 in review

The WordPress.com stats helper monkeys prepared a 2014 annual report for this blog.

Here’s an excerpt:

A New York City subway train holds 1,200 people. This blog was viewed about 5,200 times in 2014. If it were a NYC subway train, it would take about 4 trips to carry that many people.

Click here to see the complete report.

Route To Label Node Routing mode behavior

Route to Label node in IIB/WMB can be used to dynamically route the message based on the message content. The combination of Route to Label and the Label node will do this dynamic routing. In order to use route to Label node, we need to set the destination list in the Localenvironment message tree using a compute node before passing the message to the route to label node. In the Basic tab of the route to Label node, there is the mode property specified and by default it is been selected as ‘Route to Last‘ This post is to elaborate the Routing mode behavior in bit detail. As per the InfoCenter, this property signifies:

This property controls how the RouteToLabel node processes the items in the local environment that is associated with the current message . Valid values are: • Route To First: removes the first item from the local environment. The current message is routed to the Labelnode that is identified by labelName in that list item. • Route To Last (the default): removes the last item from the local environment. The current message is routed to the Label node that is identified by labelName in that list item.

To Elaborate this in more details, let me introduce a sample message flow having a compute node attached before the route to label node and the compute mode set to ‘LocalEnvironment and Message’.

Extract configurable properties from a BAR file

In an automated build environment using ANT scripts, there might be configuration changes for the same bar file according to the different environments.

eg: The DSN Name, Email server details, Back end server socket details, various flow UDPs etc might vary according to environment such as Staging, UAT, production etc.

In the ANT scripts, these properties can be overridden using some property files. The configurable values can be extracted using the mqsireadbar command. mqsireadbar command will extract all the keywords which are configurable in a BAR file.

From the output of mqsireadbar command, we can take those properties which need to be configured for different environment.

I have created a utility flow which will take the broker.xml as input and will outputs a .property file which can be used for the ANT scripts. This utility flow extracts all the configurable properties from the broker xml and will provide a well formatted property file output useful for the scripts.

This property file can be used as an input for the mqsiapplybaroverride command to override the properties of the BAR file.

eg: mqsiapplybaroverride -b  BarFileName -p PropertyFileName

The PI can be accessed from the URL here