1) Best practise would be to use is File adapter, If we need to process a large file of size around 25MB.
The Tibco file adapter parses the file and send it over to Tibco BW Engine. Whenever the filesize is larger than 25MB the Tibco BW process hangs or give java.lang.OutOfMemoryError.
2) If the file is delimeter separated type (fixed format or comma separated), then we can use "Parse Data" palette to read the file in subsets. check the option "Manually specify start record". we can make a loop for this palette, where we can pass the 'startRecord' value as 100, 200, 300 in each loop. So that each loop reads the file with specific 100 lines. Then we can process these 100 lines within loop.
3) we may need to increase Heap Size as well in run time,we can see the setting in tibco administrator while deploying the project.
For heap size go to process service instance of that project->monitoring->server settings->maximum heap size
For threads(max jobs) go to process archive.par->advanced below we can see TIBCO BW Process Configuration there we can set the max jobs for this read file process to 8.
we can try setting EnableMemorySavingMode=true .
The Tibco file adapter parses the file and send it over to Tibco BW Engine. Whenever the filesize is larger than 25MB the Tibco BW process hangs or give java.lang.OutOfMemoryError.
2) If the file is delimeter separated type (fixed format or comma separated), then we can use "Parse Data" palette to read the file in subsets. check the option "Manually specify start record". we can make a loop for this palette, where we can pass the 'startRecord' value as 100, 200, 300 in each loop. So that each loop reads the file with specific 100 lines. Then we can process these 100 lines within loop.
3) we may need to increase Heap Size as well in run time,we can see the setting in tibco administrator while deploying the project.
For heap size go to process service instance of that project->monitoring->server settings->maximum heap size
For threads(max jobs) go to process archive.par->advanced below we can see TIBCO BW Process Configuration there we can set the max jobs for this read file process to 8.
we can try setting EnableMemorySavingMode=true .
