Message stores in WebSphere Application Server

 Message stores in WebSphere Application Server

MESSAGE STORES

Message stores are important in the operation of messaging engines. To host queue-type destinations, a messaging engine includes a message store where, if necessary, it can hold messages until consuming applications are ready to receive them, or preserve messages in case the messaging engine fails. Each messaging engine has one and only one message store. This can either be a file store or a data store.
A message store enables a messaging engine to preserve operating information and to retain those objects that messaging engines need for recovery in the event of a failure.

A messaging engine preserves both volatile and durable data in its message store. Volatile data is lost when a messaging engine stops, in either a controlled or an uncontrolled manner. Durable data is available after the server restarts. A messaging engine stores various types of data, including messages, transaction states, and communication channel states.

When started, a messaging engine obtains configuration information from the WCCM ( WebSphere Application Server Common Configuration Model) repository. A messaging engine retrieves all other data from its own file store or data store.

There are currently no facilities available for migrating from a data store to a file store.
• File stores
• Data stores

File stores

File stores use a file system to preserve operating information and to persist the objects that messaging engines need for recovery in the event of a failure.
A file store is a type of message store that directly uses files in a file system through the operating system. The data storage in a file store is split into three levels:
1. Log file
2. Permanent store file
3. Temporary store file.

Log file :
This file contains information about currently active transactions and data that is not yet written to a store file. It is a circular log and its file size is static while a messaging engine is running, but can be changed if required. A restart of the messaging engine is required for the changes to take effect. The size of the log file limits the maximum size of a message that can be sent.

Permanent store file :
This file contains permanent data that is retained after the restart of the messaging engine, such as persistent messages, queue data, and information about the storage and transmission of persistent messages.
The permanent store file can be configured to have a maximum and minimum size, or to be unlimited in size. The file can grow from the minimum size (or as required in the unlimited case) but will never shrink (even if the maximum size is set lower than its current size). The file sizes can be changed in the administration console, but a restart of the messaging engine is required for the changes to take effect.
Similar to a file system, when data is deleted from the store, the data in the file is not deleted, only the directory information is updated. This means that if a message is consumed, the message data may still be present in the store file, but the directory information that includes this data in the store is updated to reflect the fact that it is deleted.

Temporary store file :
This file contains temporary data that is not retained after the restart of the messaging engine, such as nonpersistent messages that were spilled to the file store to release memory from the JVM heap. The temporary store file contents are truncated when the messaging engine starts.
The temporary store file can be configured to have a maximum and minimum size, or to be unlimited in size. The file can grow from the minimum size (or as required in the unlimited case) but will never shrink (even if the maximum size is set lower than its current size). The file sizes can be changed in the administration console, but a restart of the messaging engine is required for the changes to take effect.
Similar to a file system, when data is deleted from the store, the data in the file is not deleted, only the directory information is updated. This means that if a message is consumed, the message data may still be present in the store file, but the directory information that includes this data in the store is updated to reflect the fact that it is deleted.

You can configure where the file store files must be placed. By default, the file store uses a subdirectory in the following path:${USER_INSTALL_ROOT}/filestores/com.ibm.ws.sib/${ME_NAME}. The file store directory contains two other directories; the log directory that contains the log file and the store directory that contains both the PermanentStore and TemporaryStore files.

Data stores

A data store is a message store that uses a relational database. A messaging engine uses a data store to store operating information in the database, as well as to preserve essential objects that the messaging engine needs for recovery in the event of a failure.

A data store consists of the set of tables that a messaging engine uses to store persistent data in a database. See Data store tables for a list of the tables that comprise a data store. All the tables in a data store are held in the same database schema. You can create multiple data stores in the same database, provided that you use a different schema name for each data store.

The one-to-one relationship between a messaging engine and a data store means that every messaging engine must have its own data store. A messaging engine uses an instance of a JDBC data source to interact with the database that contains the data store for that messaging engine. The relationship between a messaging engine and its data store. illustrates these relationships.

A messaging engine uses a JDBC data source to manage the interaction with a data store, which is held in a database
All the tables in the data store must be stored in the same schema. You can create more than one data store in a database, provided that you use a different schema name for each data store. Although every messaging engine uses the same table names, its relationship with the schema gives each messaging engine exclusive use of its own tables.

Data store topologies :
You have several options for the relative location of a data store and its messaging engine. The topology also defines the relationship of a data store with other data stores.

The following options affect your choice of data store topology:
1. The data store can either run on the same node as its messaging engine or on a remote node.
2. The data store can either have a dedicated database or it can share a database with other data stores.


Reference: IBM Knowledge Center.

Federated Repository

Federated Repository

The majority of my clients set up their LDAP settings in WebSphere by going to [Security – Global Security – Federated Repositories] and then never look at it again after that. They don’t really understand what the back-end is – well, here is a crash course:

Federated – The definition:

From late Latin foederatus, based on foedus, foeder- ‘league, covenant.’

Adj. 1. federated – united under a central government. Federate / united – characterized by unity; being or joined into a single entity; “presented a united front”

OK, what does this mean? When you installed WebSphere you were asked about an admin account and a password to assign to it – by default that account is called [wasadmin] though you can change it to anything you want. That user name and password is saved in a FILE BASED directory structure in the Deployment manager and replicated out to all federated nodes. When you add an LDAP directory then the Files based (the thing you see defined as [defaultWIMFilesBasedRealm] are federated meaning that now they are BOTH together part of a SINGLE directory entity that all WebSphere applications will utilize as a single unit for the purpose of user account look-ups and authentication.

The Files Involved:

wimconfig.xml

Is located in the [deployment manage profile]\config\cells\[cellname]\wim\config folder. This file contains the federated directory setting definitions. So the files based directory (more details below) and the LDAP directory/directories are all defined and configured in this file. As this file is an XML file, each directory is defined inside the <config:repositories and the </comfig:repositoes> items.

Let’s look at the example from my training WebSphere environment:

<config:repositories xsi:type=”config:FileRepositoryType” adapterClassName=”com.ibm.ws.wim.adapter.file.was.FileAdapter” id=”InternalFileRepository” supportPaging=”false” messageDigestAlgorithm=”SHA-1″>

<config:baseEntries name=”o=defaultWIMFileBasedRealm”/>

</config:repositories>

<config:repositories xsi:type=”config:LdapRepositoryType” adapterClassName=”com.ibm.ws.wim.adapter.ldap.LdapAdapter” id=”TTrainDom01″ isExtIdUnique=”true” supportAsyncMode=”false” supportExternalName=”false” supportPaging=”false” supportSorting=”false” supportTransactions=”false” supportChangeLog=”none”certificateFilter=”” certificateMapMode=”exactdn” ldapServerType=”DOMINO” translateRDN=”false”>

<config:baseEntries name=””/>

<config:loginProperties>uid</config:loginProperties>

<config:loginProperties>mail</config:loginProperties>

<config:loginProperties>cn</config:loginProperties>

<config:ldapServerConfiguration primaryServerQueryTimeInterval=”15″ returnToPrimaryServer=”true” sslConfiguration=””>

<config:ldapServers authentication=”simple” bindDN=”ldapaccess” bindPassword=”{xor}Dz4sLCgwLTtubWx+”

connectionPool=”false” connectTimeout=”20″ derefAliases=”always” referal=”ignore” sslEnabled=”false”>

<config:connections host=”ldap.intranet.toalsys.com” port=”389″/>

</config:ldapServers>

</config:ldapServerConfiguration>

This shows the two entries I have in my environment:

• The default file based repository identified by the ID <id=”InternalFileRepository”>
• My Domino based LDAP repository identified by the ID <id=”TTrainDom01″>

Gotcha #1: User Name and Password is Open

This wimconfig.xml contains the user name and encoded password for the LDAP bind account. Note the choice of words … ENCODED, not ENCRYPTED.

If you want to know the password for my training LDAP account copy the encoded password above and go to this link by Andrew Jones:http://www.poweredbywebsphere.com/decoder.html (thanks Andrew, I send all my clients to your site for further info and learning!)

If his is a production environment I have now gained access to an account in your environment, possibly an account that has update/write rights to the LDAP directory ….. all by looking at one file. If you are like 99.9% of my clients you are compromised:

1. SECURE YOUR SERVER, LOCK DOWN YOUR FILE SYSTEM
2. DON’T USE ADMIN OR PERSONAL ACCOUNTS TO BIND TO LDAP
3. DON’T RE-USE THE SAME PASSWORD FOR ALL YOUR ACCOUNTS

Sound obvious, doesn’t it?

 Gotcha #2: Rogue LDAP entries

If you have ever tried to change an LDAP directory, replace and entry in WebSphere you might have run into the issue that you suddenly can’t log into WebSphere anymore after you made the changes. Why? Well, you need to understand that sometimes when you make changes, those old entries don’t disappear totally – they are left behind and impact you.

Remember the part about FEDERATED above? If not ALL directory entries here (in this file, not what shows in the IBM Console) are accessible and functioning, then the federated directory that you are trying to access will not work and you cannot authenticate. It is the Three Musketeer principle: “All for One, One for All”

Gotcha #3:

Some changes can’t be made in the interface. I had a client that mistakenly entered an LDAP directory as Microsoft AD but it was Domino. They tries to clean it up in this file but it still was not working and they could not log in ….. well, the wimconfig.xml contains allot of directory type specific settings which are set by the type: <ldapServerType=”DOMINO” > .. My advice is to remove the incorrect entry and enter a NEW entry at the same time and then make sure the old incorrect one is gone from the wimconfig.xml. DO NOT manually try to clean this up (other than remove the entry) as you might end up destroying thewimconfig.xml and making your environment unusable.

Remember Dr. Vic’s rule #2 above? Make back-ups before any changes to WebSphere security settings.

HOW TO RECOVER A LOST PASSWORD

WEBSPHERE: WASADMIN – HOW TO RECOVER A LOST PASSWORD

I had the case recently where was working on a WebSphere 7 environment that was being uncooperative and the previous creator had not taken any backups and the documentation was rather thin . . . one of my most favorite scenarios to walk into.

I found myself with three sets of documents that all stated a different wasadmin password and none of them worked and the client had never bothered to set up any secondary WBM console admins. A dilemma I had to solve.

Encoding vs Encrypting

You might know that all sensitive information about security is entered into thesecurity.xml document that can be found at [$WAS_HOME]/profiles/[profile name]/config/cells/[cell name] folder. In Windows this might equate to:

C:\IBM\Websphere\AppServer\profiles\Dmgr01\config\cells\cell01\security.xml

Linux/AIX would likely be something like:

/usr/IBM/WebSphere/AppServer/profiles/Dmgr01/config/cells/cell01/security.xml

This document contains the name and password information for the primary admin account for the WebSphere cell – in most cases that will be the default account [wasadmin]. The password is, however, not encrypted but rather encoded. Encryption would use an encryption key to hash the password and without that key you would not be able to retrieve it. Encoding however is a whole other deal – the coding/decoding information is integral to WebSphere itself and is the same for any install anywhere in the world. That means if you encode the same password anywhere, the resulting hash will be exactly the same no matter which server you do it on.

Now, this is not great security in and upon itself and I will not go into details on this – other than it is really important to lock down the physical access to to any WebSphere server you are in charge of, all the way down to file rights …. or you might regret it at some later time.

How to Decrypt:

I am not the first blogger out there that is writing about this, but nobody every wrote it out for Windows servers so I am going to concentrate on that OS right now, and most of the blog entries out there are for older versions and the proces has changed since. Here some of the articles that I have read over the last few years Robert Farstad, Robert Maldon,  and a few more . . . . google the conent here and you will find them.

Here some basic details:

• WebSphere Version: 7.0.0.21 (the process is the same for any V 7.x server)
• $WAS_HOME=C:\IBM\WebSphere\AppServer

Step 1: find the wasadmin information

Open the security.xml, find the entry for the encrypted password: it always starts with {xor}, in my case it is:

userId=”wasadmin” password=”{xor}LDo8LTor”

Step 2: Find your WAS Version Specific Java Plug-in Folder:

In my case it was:

C:\IBM\WebSphere\AppServer\deploytool\itp\plugins\com.ibm.websphere.v7_7.0.2.v20110524_2321\

Step 3: Find your java home and open a command prompt

In my case this equates to

C:\IBM\WebSphere\AppServer\java\bin\

Change to this folder in the command prompt you opened.

Step 4: Run the Password Encoder/Decoder:

This is where you need the folder location and the encoded password you looked up in the previous steps.

In C:\IBM\WebSphere\AppServer\java\bin\ run the following command

java –java.ext.dirs=C:\IBM\WebSphere\AppServer\deploytool\itp\plugins\com.ibm.websphere.v7_7.0.2.v20110524_2321\wasJars\ -cpsecurityimpl.jar:iwsorb.jar com.ibm.ws.security.util.PasswordDecoder{xor}LDo8LTor

This above command is one long command string (it might wrap depending on your screen) and it will create the following output in the command prompt:

encoded password == “{xor}LDo8LTor”, decoded password == “secret”

The process for Linux/AIX is basically the same, however the folder structure will be different. The commands are about the same but depending on which version of Linux you are running the Java switches might need some fidlding – though the base does not change.

Security!

Next to being helpful to retrieve lost passwords, this article hopefully also shows you just how important good physical security for your WebSphere servers is – don’t think that just because you have a log-on or locked down root that you are safe.