How To Handle / Parse / Split Large XML Files ?

We understood that we cannot fit a 10GB file into only 1GB of memory or 4GB of memory i.e. RAM size.

Even for the 77Mb file it might not be enough depending on your implementation.

Memory issue is a common problem for applications that need to handle large XML files.

The XML parser of BW works roughly like a DOM parser.

Meaning, it loads the whole XML file into memory. This is not really design for working with huge XML.

For reading a large XML file you will need to use a different XML parser that does not load the whole XML (like stax or sax or other ?). For this you need to use "java code".

Design steps:

Step1 : Get the stax jar file and java code from open source links stax xml parser.

Step2:  Do the necessary steps for copy the jar file in tibco library folders.
Step3:  Design one subprocess with java Code and configure the parameters as required for Jar file.

Step4:  Design the Main Process File Pollar, callprocess and Parse XML.

Step5:  Select the subprocess (which we designed in Step2) in main process call process.

Solution example posted in this blog with title BW solution for Large XML Files.



 

Error : out-of-memory errors received by a Wait For JMS Topic Message

How should we fix out-of-memory errors received by a Wait For JMS Topic Message due to events with no Candidate Key matches?

A. A shorter process timeout should be configured in the Input tab of the activity.

B. A shorter event timeout should be configured in the Message Event tab of the activity.

C. The size of the LRU cache should be reduced in the Advanced tab of the activity.

D. The Candidate Key should be configured as "match-any."

The correct answer is option B.

Explanation:

Event Timeout in Message Event Tab specifies the amount of time (in milliseconds) a message waits if it is received before this activity is executed.

If the event timeout expires,an error is logged and the event is discarded.

If no value is specified in this field, the message waits indefinitely.

If we specify a shorter timeout,then we can meet the requirement given in the question.

TIBCO ADB Adapter Error - “Failed to load shared library”

When starting an Adapter Service from the TIBCO Designer Adapter Tester, you might encounter this error: “Failed to load shared library, library name: adb55.dll”.

Fix this error with the following steps:

1. Open up C:\WINDOWS\system32.
2. Open up the TIBCO RV home directory (it might be C:\tibco\tibrv\8.1\bin).
3. Copy the libeay32.dll and the ssleay32.dll from the TIBCO RV home directory to the system32 directory.
4. Say yes if you are prompted to replace the files.

Restart your TIBCO Designer and try again.

EMS vs RV

• EMS is server based while RV uses a communications daemon on every machine.
• Depending on the requirements for speed and bandwidth, flexibility, security,reliability, availability and message passing between sub-nets one might require EMS or RV or a hybrid of both. 
• Both TIBCO RV and TIBCO EMS provide authentication, authorization and secure communication. Tibco RV and TIBCO EMS both provide secure socket layer (SSL) support for all communication. Where Web access is used, HTTPS is supported as well.
• In conclusion the EMS (JMS) hub-and-spoke model provides a better choice for the majority of EAI problems and RV is best left to its niches: high volume, high performance, 1-to-very many. 
• Generally use RV for speed topic / broadcast messages and EMS / JMS the more you are worried about assuring messages get to a receiver.  
• You can tune both products to go faster and more reliable. RV is basically broadcast via UDP, so needs RV Remote Daemons to cross subnets. You can bridge EMS to RV if you have specialised areas which you want to speed up.

Creating a TIBCO Hawk Domain

Creating a TIBCO Hawk Domain -

Create a TIBCO Hawk domain if you wish to isolate groups of TIBCO Hawk agents into independent monitoring sets.

Remember that each TIBCO Hawk agent can belong to only one TIBCO Hawk domain and cannot communicate with the TIBCO Hawk Display program on other domains.

The default is to monitor an entire network as one group. To use the default, omit the hawk_domain option.

Testing a Domain -

To test the domain option, temporarily set the TIBCO Hawk domain on a single system to a name such as test, then start (or restart ) the TIBCO Hawk Display.

If you entered a unique domain name, you only one discovered host should appear in the TIBCO Hawk Display main window.

This shows that only the local agent and the local TIBCO Hawk Display are using the test domain.

If the domain is as desired, you can now install it on the network systems where you want to assign specific computers to that domain.

All systems assigned to that TIBCO Hawk domain appear on the Display as part of that domain.

When finished with testing, change back to the default domain and restart the TIBCO Hawk Display to verify that it was correctly restored.

Catch activity vs Error path

In TIBCO, DB Adapters are C based and BusinessWorks is java based applications.

Hence we can't catch all the exceptions from Adapter in BW using Error path.

So we need to use either "catch" or "Write to Log" to know the error in BW. 

Scenario on EMS Server start up

If we have config file and duplicate queue/topic names are present in that conf file, so can we start this EMS server with that conf file ?

Yes,when starting EMS Server, if the duplicate queue/topic names are present in respecitve conf files, while starting up, EMS Server displays the duplicate names and the server will get start.

If  the duplicate entries are present in bridges.conf then what will happen while starting up the EMS server ?

duplicate entries are present in bridges.conf then it  displays the line number where it got the duplicate entry and server will not get start.

How can add a machine to DOMAIN from command line ?

Below are the steps to add a machine to domain from command line.

Step 1: Go to C:/tibco/tra/5.6/template/domainutility/cmdline

Step 2: Create a new file by copying the AddMachine.xml to another file name.
Ex: cp AddMachine.xml AddMachine_XX.xml

Step 3: Edit AddMachine_XX.xml, enter domain related specific entries.

Step 4: Go to C:/tibco/tra/5.6/bin, execute the following command.

./domainutilitycmd -cmdFile C:/tibco/tra/5.6/template/domainutility/cmdline/AddMachine_XX.xml –verbose

Step 5: Finish.

How can change the Global Variables value at runtime ?

To change the Global Variables value at runtime, add a property like below in the application .tra file and restart the application.

tibco.clientVar.SharedResource/Connections/MaxSessions=16

xpath function "tib:render-xml" vs "Render XML" activity

• We can use tib:render-xml function instead of Render XML activity if we are not modifying XML structure.
• We can't set encoding with the function, where as with activity we can.
• By default, Render XML activity do the XML validation, not Schema validation.