Message Processing HowTo - Monitoring file contents

Implement needed operations

NOTE: For the location of source files see extending cobalt.

For monitoring you have to create a custom message processing step which has to inherit the MessageProcessingStepBase and override its Execute method. This sample uses FlatfileRegexFilter (also included in source projects) for parsing the transferred file using regular expressions.

01public class FileLogger : MessageProcessingStepBase
02{
03    private readonly FlatfileRegexFilter _regexFilter;
04 
05    public FileLogger(MessageProcessingStepConfig config): base(config)
06    {
07        Trace.TraceInformation(String.Format("Initializing FileLogger with parameters: {0}", config));
08        try
09        {
10            XmlDocument doc = new XmlDocument();
11            doc.LoadXml(config.CustomParameters);
12 
13            XmlNode monitorElement = doc.SelectSingleNode("//Monitoring");
14            int skipHeader = Int32.Parse(monitorElement.SelectSingleNode("//SkipHeader").InnerText);
15            string regex = monitorElement.SelectSingleNode("//Expression").InnerText;
16            string logFile = monitorElement.SelectSingleNode("//LogFile").InnerText;
17 
18            _regexFilter = new FlatfileRegexFilter(new FileRepository(logFile), regex, skipHeader);
19        }
20        catch (Exception ex)
21        {
22            Trace.TraceInformation(String.Format("Exception occurred while trying to initialize FileLogger: {0}", ex.Message));
23            throw;
24        }
25    }
26 
27    public override FileInfo Execute(FileInfo inputFile, ExtendedTransferContext context)
28    {
29        using (FileStream fs = inputFile.OpenRead())
30        {
31            _regexFilter.ReadStream(fs, context.SourceEndpointContext.FileItem.Name);
32        }
33        return inputFile;
34    }
35 
36    public override void Dispose(){}
37}   

Constructor initializes file monitoring parameters which are in config.CustomParameters. Parameters is a single string containing a simple XML structure which is parsed to to get required values for creating FlafileRegexFilter.

FlatfileRegexFilter requires an implementation of IMonitoringDataRepository as constructor parameter. The implementing class FileRepository takes a monitoring file path as a constructor parameter. Monitoring data is written into that file.

FlatfileRegexFilter class' ReadStream method reads contents of a transferred file one row at a time. Once the row is read and parsed, the results are passed to FileRepository class' RecordRow(string[] captures) method for writing them into the monitoring file separated with tabs.

01     
02public class FileRepository : IMonitoringDataRepository
03{
04    private string logFileName;
05 
06    public FileRepository(string fileName)
07    {
08        logFileName = fileName;
09    }
10 
11    public void RecordRow(string[] captures)
12    {
13        Trace.TraceInformation("Recording row");
14        StringBuilder sb = new StringBuilder();
15        foreach (string capturedValue in captures)
16            sb.Append(String.Concat(capturedValue, "\t"));
17 
18        sb.Append(Environment.NewLine);
19        File.AppendAllText(logFileName,sb.ToString());
20    }
21}

Step 2: Configure Cobalt

Once the assembly is deployed, you need to configure Cobalt to use the new custom MessageProcessing step. To do this, just create a new Cobalt connection point as described in the manual and:

Select MessageProcessing and set the first Step type to Custom.

Under the Custom element:

Set the AssemblyName to the assembly name where the custom class resides in, e.g. Frends.Cobalt.Extensions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.
Set the ClassName to class name of custom class, e.g.Frends.Cobalt.Extensions.FileLogger.
Set the Parameters to <Monitoring xmlns=""><SkipHeader>2</SkipHeader><Expression>([^;]*);([^;]*);([^;]*);</Expression><LogFile>C:\Temp\cobalt.log</LogFile></Monitoring>.
In this example parameters a provided via single XML string. Parameter element descriptions:
- SkipHeader = How many rows at the beginning of transferred file are considered to be as header and are skipped before starting monitoring of file contents.
- Expression = Regular expression set of rules by how the file contents are being monitored.
- Logfile = Location of log file where monitoring results are being written.
  - If log file is not present, it is being created during trasfer monitoring.
  - If log file is present, monitoring results are appended to the end of the existing file.

Step 3: Create monitored file

In this example we expect the monitored file to be in .csv format and that it has 2 header rows. After header the actual content has 5 cells on each row, but with the expression given in monitoring parameters only first 3 cells, separated with a tabulation, are wanted to be monitored in the log file.

Create a file with the following contents in Cobalt input directory. Name the file to match your Cobalt Source FileName pattern.

ACME inc;Invoicing data;October 2009; Customer;Address;Total sum;EMail;Additional information; John Doe;Concrete Jungle 12 A 5;43.35;john.doe@example.org;John is a cool dude; Jane Doe;Hill Street 124;59.32;jane.doe@example.org;Cooks good pizza;

Message Processing HowTo - Monitoring transfer file contents

Implement needed operations

Step 1: Compile and deploy the assembly

Step 2: Configure Cobalt

Step 3: Create monitored file

Step 4: Create and execute the file transfer routine