Core Content Only

Archive for July, 2008

This is just a quick introduction to batch loading content into content server with the Batch Loader application. The application can be run from the command line as well as in graphical mode. I normally run it in graphical mode.

I also like to put the check mark in the box to produce an errors file. This file is kind of neat, if anything fails it is placed in that file where you can tweak the metadata and then point batch loader at that file to attempt loading all the content that failed to load. Why might things fail? Perhaps you forgot one of the required fields like dSecurityGroup or perhaps you performed a check-in using a particular metadata value that did not exist in a validated list.

Usually you will need to add a setting to the intradoc.cfg file called Batch Loader User Name. This setting in your configuration might look like this:

BatchLoaderUserName=sysadmin  

I am sure you could come up with better values to put in this field besides sysadmin, but for the sake of this demonstration/test this should get you running pretty easily.

NOTE: intradoc.cfg, not config.cfg.

Now you must construct the batch load file. Nothing scary here, just a text file. This file contains key=value pairs one to a line of metadata and values and uses hash/pound characters (#) to begin lines that are comments, like this:

# This is a comment
Action=insert
dDocType=ADACCT
dDocTitle=Product Details
dDocAuthor=sysadmin
dSecurityGroup=Public
primaryFile=<path to file>
dInDate=7/23/2008
dDocName=TestContentID
<<EOD>>

What if you wanted to perform a metadata only check in? Try something like this:

# This is a comment
Action=insert
dDocType=ADACCT
dDocTitle=Product Details
dDocAuthor=sysadmin
dSecurityGroup=Public
#primaryFile=<path to file>
dInDate=7/23/2008
dDocName=TestContentID
createPrimaryMetaFile=true
<<EOD>>

Sometimes we encounter the need to perform a search in content server that requires a lot of work with the <OR> query operator. Between that and a few other variables it might be possible to craft a query that is too long for content server. This happens especially with generated queries. If you are using universal query syntax or a verity engine for your search engine you might also encounter this. If the data you are searching on is metadata based you might be better off temporarily switching over to a database based search on the fly to find your results. By adding this setting to the query string you can change how content server interprets your search:

SearchQueryFormat=DATABASE

In most installations the Search Query Format setting will default to UNIVESAL. Now that you have switched over to a database based search you can set your query string using the IN operator for a much shorter query converting from something like this:

QueryText=xType <MATCHES> `A` <OR> xType <MATCHES> `B`
<OR> xType <MATCHES> `C` <OR> xType <MATCHES> `D`

Into:

QueryText=xType IN ('A', 'B', 'C', 'D')

If you’re on a Database Full Text setup to start with you can still do this type of thing AND use full text like this:

QueryText=( xType IN ('A', 'B', 'C', 'D')) and (<ftx>keyword</ftx>)

Perhaps the best thing you can do is at least take a moment to evaluate why you must have so much ‘or’ based logic in your query in the first place. Or is almost always one of the most costly instructions.

If you are trying to squeeze additional performance out of your production servers you might look into adding the setting DisableSharedCacheChecking. This would be placed in your config.cfg file or under general configuration through the admin server. As always, it may help to perform some kind of analysis or metrics gathering prior to adding the configuration and then re-analyze after configuration implementation. What this boils down to is content server watching dynamic html resource includes in components to see if they need to be reloaded.

DisableSharedCacheChecking=true

In other words: if in place on a production server (where you would not be changing includes like you would a development instance) this setting can alleviate a measureable amount of file system activity. This should translate into a performance gain in some fashion.  Or to say this as confusingly as possible: a reduction in performance degradation�I just thought that would be fun to say.

Potential flamebait warning: And as we all know, disks are slooowww. Maybe I just hook the disks up wrong?  I hope I hooked the disks up…

Using “View Server Output” from the admin server can tell you a lot about what is going on inside your content server. The server output can accumulate so much information it can easily be overwhelming. To help with this we are going to take a closer look at the tracing sections and how we can expand the built in list of sections to allow us to trace and troubleshoot the code we build for content server. The information detailed in server output depends on your configuration setup in the System Audit page:

The comma separated list of active tracing sections can be saved into config.cfg under TraceSectionsList. The checkbox next to Full Verbose Tracing would represent TraceIsVerbose in the config.cfg file. Descriptions of these can be found in the IDOC Script Reference Guide. Recall your config.cfg file is in the <install>/config directory, not in the bin directory. The drop down list is filled with sections you can use to narrow or expand your tracing output with.

The server output does not show all activity from all of history. It only displays the most recent activity, and if you are planning on dumping out the full contents of a DataBinder you probably will miss some of the data. On windows, it comes in handy during a full-on-debug-session to add UseRedirectedOutput to your config.cfg file. This will dump the log out to a file on disk. Do not forget to turn this off later or you may run out of disk space or enjoy some decreasing performance metrics.

Excellent, with this basic information under our belt we can now look at how you might add a trace section to that drop down. Say for example you have some code you would like to instrument with tracing so you can see it in server output. Just how might you add your custom-section to the drop down list?

So, having replaced the defaulted information with my specific information I end up with the adjusted resource file looking something like this:

<tr>
<td>TraceSample</td>
<td>Example of custom section tracing</td>
<td>false</td>
</tr>

Having used component manager or component wizard to ENABLE my component (you enabled yours right?) and restarted my content server (you restarted yours right?) I can now see my new custom tracing section in the drop down list in the System Audit page:

Whew. If you have made it this far please stick around a little longer, we are finally ready for the big payoff. We understand tracing now, we understand how to add sections to be traced. So, please, please just tell us how to instrument the code to take advantage of this already! To make use of this in IdocScript use the trace function like this:

IdocScript Tracing

<$trace("message", "#console", "CustomTraceSectionName")$>

So, for the example section I created I would write something like:

<$trace("Trace Test!", "#console", "TraceSample")$>

One word of caution, sometimes when you copy code like this off the web the quotes get out of whack (technical term!). So if you have problems, when you paste this in try retyping the quotes. Or, once installed you could use the test template included in the example component to see how things work.

Java Tracing

If you wanted to use this in Java code you can use SystemUtils.trace(“CustomTraceSectionName”, “Your Message”); to instrument your Java code for tracing. Inside your Java code you can also take advantage of the “Full Verbose Tracing” checkbox setting by testing the value of SystemUtils.m_verbose, like this:

if (SystemUtils.m_verbose)
{
SystemUtils.trace("TraceSample"
 , "Tracing from my java code against my custom trace section");
}

You can also dump information about an exception with some code like this:

SystemUtils.traceDumpException("filestore"
, "Error encountered saving file to file store.", e);

Additional References

IDOCScript Online Reference Guide (HTML Version)
IDOCScript Online Reference Guide (PDF Version)

Online Documentation (Version 10.1.3.3.0)

1. Release Notes
2. Install Guide
3. Administration

Online Blogs & References

1. Understanding SSPU by John Sims
2. Fixing FTP with SSPU by John Sims
3. Useful SSPU Logging from Web Monkey Magic

Now that have all these tools hanging on our belt and you have went to these various places and learned all about SSPU (you did right?) what exactly do I have to add? During some recent publishing we ran into a problem where Cascading Style Sheets and parts of HTML pages were getting replaced when they should not have. An example of the improper transformation might look like:

background: url(/original/path/coolpic.gif)

into

background url(/sspu/translated/path/coolpic.gif)

The difference is fairly subtle, the colon is getting dropped. Bad, bad. Several other issues exist but are all similar in nature. Luckily, patch p51051816 exists. I have been unable to find the patch on METALINK at this point but one Service Request later and they shipped it right out. The patch itself is very simple to apply, and the readme.txt details out the six step process.

Oh, and yes the patch did fix the problem!