FRENDS Iron Documentation

Contents

• Getting Started
  • Introduction to FRENDS Iron
  • What is FRENDS Iron?
    • Enabling batch processing
    • Batch features also streamline other development
    • Highly extensible process model
  • Installing FRENDS Iron
  • Starting the FRENDS Iron processing engine service
• Developing FRENDS Iron Solutions
  • FRENDS Iron Process model and elements
    • Routines
    • Tasks
    • Schedules
    • Connection points
    • Connection point groups
    • Actions only supported in the FRENDS MMC UI
  • Technical details
    • Task parameter references
    • Encrypted connection point fields
    • Creating your own task workflows
    • Workflow persistence
    • Tracking activities
    • Ready-made WF elements
    • BizTalk Server integration
    • Logging to database with FileTable
    • StreamInsight supported
    • Support for custom columns in Web UI
  • Troubleshooting
    • Finding the reason for the error: viewing logs
    • Contacting FRENDS Support
    • Common errors and fixes
• FRENDS Iron Management
  • FRENDS architecture
  • Providing high availability for FRENDS Iron host
  • Configuring FRENDS Iron
  • Batch element import with deployment tool

Getting Started

Introduction to FRENDS Iron

FRENDS Iron is a process management and systems integration product developed by Frends Technology. It provides enhanced integration options for a customer while being based on latest industry standards.

FRENDS Iron is designed for heavy-duty process-based integration tasks. It can communicate with a large set of enterprise applications but its connection features scale down easily to include embedded devices, small databases, point-of-sale (POS) systems etc. FRENDS Iron provides both message-based brokering capabilities as well as tools for batch-based transfers.

FRENDS Iron consists of three functional parts:

• FRENDS Iron service, which executes parameterized batch processes according to schedule
• FRENDS Log service, which processes log messages from FRENDS or BizTalk processes or any third party application
• FRENDS Web UI, the main user interface for the system, allowing for creating and monitoring FRENDS processes.

FRENDS Iron utilizes Windows server software such as SQL Server, BizTalk Server and Service Bus for Windows Server

What is FRENDS Iron?

FRENDS Iron is a stand-alone process engine, with scheduling and parameterizing functionality. It can work together with BizTalk Server, providing several features on top of it.

Enabling batch processing

FRENDS Iron is a flexible and versatile tool. It is primarily meant for enabling enterprise batch processing, especially to systems with multiple passive endpoints that you need to send data to or receive data from.

Currently, BizTalk does not handle the following situations well:

• You cannot schedule business processes to run at specific times. You can only define schedules for receive locations and send ports. These schedules can only be open once a day. Say you need to schedule a batch to run four times a day, every six hours, and only on workdays - you need to create your own scheduler. You also need to have some kind of message to actually trigger the execution, so you need a process of your own to create the message and send it to the receive location, like a script scheduled using Windows Task Scheduler
• Defining and managing endpoint groups to send data can be tedious, especially with multiple groups and more than 10 send ports in a group - the send ports need to be added to the group one by one. When removing a send port from many groups, you need to remove them one by one.
• Only one-way send ports can be grouped - there is no way to get a response from multiple endpoints. Solicit-Response from multiple endpoints requires loops and dynamic ports in an orchestration.
• Monitoring executed batch processes with multiple sub-processes can be a pain: the process execution details can be found in HAT or BizTalk Explorer queries, but how will you know which processes belong to the same batch? Grouping different batch processes to BizTalk Applications can give some relief (but forces you to purchase the Enterprise version of BTS)

FRENDS Iron provides the features needed for the above mentioned situations:

• You can define and reuse complex schedules easily: schedules can have different time windows, with individual settings for season, recurrence, open days and delay.
• Endpoints can be easily grouped, e.g., dynamically using a query.
• As the endpoint configuration and process triggering is external to BizTalk, you can reuse simple BizTalk orchestrations to handle solicit-response message exchanges with any port type.
• All execution data and parameters of executed schedules, triggers and processes are stored in a single database - all grouped together. The batch execution status per each process can be monitored easily from the user interface or a custom reporting site. Failed executions are easily noticed and, as all their data is stored to the database, the reason for the failure can be analyzed quickly.

Batch features also streamline other development

You can also use FRENDS Iron for other business tasks.

• The need for doing things according to a schedule is not limited to batch processes. For instance, maintenance processes can be scheduled to run on specific dates and times. With FRENDS Iron, you can also schedule BizTalk receive locations and send ports.
• You can launch BizTalk processes from FRENDS Iron UI simply by clicking a mouse button.
• The messages sent to BizTalk can be easily defined and maintained in FRENDS Iron connection

Highly extensible process model

As FRENDS Iron uses its own process engine, the actual triggering process can contain any number of custom steps, e.g., fetching data from a local database or file and publishing execution results to a server or via email. In fact, FRENDS Iron can be used independently of BizTalk for simple processes that would be cumbersome to do with BizTalk and usually would be done with custom code.

Installing FRENDS Iron

FRENDS Iron is installed using a Windows Installer installation package. It installs the needed libraries and executables. After installing, you need to run the Configuration Tool to set up the Windows services, databases and other settings. The installer suggests you do this at the end of the installation.

Please see the accompanying Readme file for up-to-date information on the installation and basic configuration. For details on how to configure FRENDS Iron, see section Configuring FRENDS Iron or the FRENDS Iron Configuration Tool Documentation.

Starting the FRENDS Iron processing engine service

Normally the service is first started by the configuration tool or on automatically on system startup, but if for some case the service is not running, you can start it manually. There are multiple ways of starting the FRENDS Iron service, you may for example use the FRENDS Iron Management console, command line or Window's Services applet. The following images show how to accomplish this.

In the Services applet: Open the Services applet and find the FRENDS Acc Service service from the Services list. Right click it and select Start.

Commandline: Open a commandline prompt with administrative rights (Run as administrator) and write the following command: net start FrendsAccService

Developing FRENDS Iron Solutions

Creating a new FRENDS Iron process is straight-forward, as demonstrated in the Getting Started: First create a new routine and choose a task to execute. Then you can add a schedule and parameterize the task as needed. Then just start the execution manually or activate the routine to be executed according to the schedule.

However, when creating and modifying your own tasks, schedules and other FRENDS Iron elements, first you need to have a basic understanding on the FRENDS Iron process model and elements, which is the topic of the next section. After that, you may want to look more closely to the technical details of FRENDS Iron, for instance on topics FRENDS architecture.

The FRENDS Web user interface and the actions supported by it are documented in FRENDS Web UI documentation.

FRENDS Iron Process model and elements

Even though FRENDS Iron can use BizTalk Server to execute business processes, it also has a process engine of its own. The process engine can run tasks based on simple .NET classes as well Windows Workflow Foundation (WF) 3.5 workflows. You can then combine these tasks to processes called routines. The routine processes can have simple branches and run steps in parallel. You can start routines from the FRENDS Web UI or define schedules for starting them at a given time.

Routines bring the other FRENDS Iron elements together:

• Tasks, which define the actual executable process,
• Schedules, which define the times when a routine can run,
• Connection points and connection point groups, which define endpoint specific parameters

Routines

Routines define the executable integration processes: Routines contain the information on the sequence of activities to execute (tasks) and the schedule according to which they are executed.

A routine can have one or more process steps that are executed according to the process graph connections: each step can have success and error connections to one or more other steps. When deciding which steps to execute, the scheduler checks the state of the step: if the step is successful (state is OK or ActionSkipped), the steps connected to the finished step with a success connection are considered, otherwise the steps in the possible error branches are considered. Steps with no more possible connections will be marked as Canceled.

Error branches are not mandatory: if a step does not have one and it fails, the process execution will stop and the rest of the steps will be canceled. Also, if task fails with an unhandled exception (and the Treat exception as failure is not chosen), the execution of the routine instance will end. See the section on instance states for more details on what the possible end states are.

You can run two or more steps in parallel simply by adding multiple connections from a single step. To wait for the parallel steps to finish before continuing to a follow-up step, you simply add connections from each of the parallel steps to the follow-up step. The follow-up step will only run once all of the preceding tasks have finished.

However, please note that the follow-up step will run even if one of the connections to the preceding tasks is taken: e.g. if the task is waiting for three steps with success connections, it will run even if only one of them is finished successfully. If you need to wait for all of the parallel tasks to finish successfully, you need to check for the results of the preceding tasks in the follow-up step and decide what to do if not all of the steps have been finished successfully.

Task instances for a CP group that are executed concurrently will all be allowed to start and finish, even if one of the instances fails. The next task may be started (or canceled) only after all task instances in the group have finished. If any of the instances in the group has failed, the possible error branch will be followed.

You can give parameters to the tasks in routines, allowing you to create reusable processes. You can also define endpoint specific parameters in a reusable way as connection points, and group them for launching many executions at once.

All routine execution data is stored to the FRENDS database. You can view this data with the Routines view in the FRENDS Web UI; see section Run and monitor routine execution for details. You can also create reports from the data using SQL Reporting Services, for instance.

Using tasks as conditions or for polling

Sometimes it is necessary to check for some preconditions before doing an action: for example, it could be useful the check if the remote host is up and running before trying to send the message to it, avoiding an unnecessary error if the host is temporarily down. You may also try to poll a remote directory for files to transfer, and if there are no files, end the task doing nothing.

For the first case, you can create a custom precondition task that sets its end state Failed if the precondition is not met. Then you just drag the error branch to the step for that action. As the failure is not really an error, you can set the Ignore failure in routine end state option set for the task, so the routine instance is not unnecessarily marked as failed.

Polling tasks that do nothing should return the ActionSkipped return value, so they end in the ActionSkipped end state. For instance, the FRENDS Cobalt task can end in ActionSkipped state if no files were transferred.

Running routines on multiple hosts in a farm

By default, routines can be executed on any host in a farm. Individual task instances of the same routine can also be executed on any host in a farm by default.

Sometimes it may be necessary to limit a routine to run only on certain hosts in a farm. For example, if an FTP server allows connections only from specific IP addresses, a routine can be configured to run only on hosts within the accepted IP address range. You can do this by setting the Allow routine to execute only on specified hosts option in the user interface.

For more information on running FRENDS Iron in a farm, see section Farming below.

Scheduling routines

Routines are usually scheduled to be executed at given times. To understand how schedules work, it is good to know some things about how the schedules are processed.

1. The FRENDS Iron scheduler polls the database once every second for active routines with schedules with an open time window, i.e. their start time and date have passed (and the possible end time has not), and the current date is one of the days when the schedule should execute, as defined by the recurrence type.
2. For all of these routines, the FRENDS Iron scheduler will create and launch a new routine instance. The routine instance will have a task instance for each task defined for the routine, and launch the first tasks - the ones with no incoming connections.
   If the routine task uses a CP group, one task instance is created for each connection point in the group. All task instances in the same group are launched at the same time.
3. Finally, for each routine, the FRENDS Iron scheduler will calculate the next time the time window will be open and store that to the database. It is this pre-calculated value that the FRENDS Iron scheduler uses in its polling loop. This makes the poll more efficient, but also means you should not try to modify the values in the database directly! Otherwise, you may cause routines not being launched at the correct times.

One thing to note from this is that routines are scheduled with only best effort guarantees: the routine will be started if the schedule is open when the database is polled. Because the poll delay is one second by default, this can mean that the actual execution of a routine may start 1-2 seconds later than the actual scheduled time. Therefore you should not create too short time windows or expect the routines will start at exactly the given time.

The next scheduled execution is shown in the Routine view, in the Next Scheduled Execution column. If the routine is not active (i.e. is not scheduled for execution), the column shows "Not active"; also, if there are no schedules for the routine, the column shows "No Schedules"; and finally if the schedule season start date is in distant future (more than one year from now) or if the season end date has already passed, the column shows "No open schedules".

Once a routine has been launched, it will be allowed to start and execute. This means that if you launch a routine with a large number of tasks (e.g. using a large CP group), the execution of the task instances and hence the routine instances may take a long time, continuing even after the time window has ended. If this can be a problem, the routine executions should be scheduled more evenly in time, e.g. by splitting large CP groups and scheduling them separately.

There are limits on how many routines and task instances are executed and launched at one time:

• Maximum number of task instances running at a time (by default 100, see setting WfMaxTotalRunningInstances)
• Maximum number of running task instances per routine (by default 50, see setting WfMaxRunningInstancesPerRoutine)

NOTE: The limit on maximum number of task instances running at a time also limits the number of task instances that are launched: If the number of waiting, launched or running task instances in a farm is more than hundred times the WfMaxTotalRunningInstances setting, new routine instances will not be launched. New instances will again be launched once the running instance count goes under the limit. This limit is meant to make sure the number of launched task instances does not overwhelm the scheduler.

In addition, you can limit a routine to run only one scheduled instance at a time by setting the Run only one scheduled instance at a time option for the routine. For instance, if a routine is scheduled to execute every 10 minutes, but the routine instance takes 13 minutes to complete, a new routine instance will be scheduled only after all the task instances of the previous routine instance have finished, i.e. about 14 minutes from the start of the previous instance. There are some things to note about this feature:

• If you start a routine manually e.g. with the "Run Once" action from the FRENDS Web UI, the new instance will be executed, even if the setting was turned on and a previous instance was already running. This is because the setting is only checked for scheduled routine instances.
• A routine instance can still have many concurrent task instances running, if you use a connection point group.
• The previous routine instance may still show as Launched when launching the new routine instance, because the scheduler actually checks the number of running task instances.

The routine schedules are checked automatically once every minute. If you change the routine schedules, e.g. add new schedules or change the start time of one, the changes will therefore take effect in within one minute.

NOTE: Changes to schedules used in a routine may mean a routine that has executed for that day, will execute again. Example outcomes of this are described in the table below.

Routine's next execution time depends on the schedule type and the configuration of the schedule. Following tables describe how routines will be executed depending on the schedules. Scheduling behavior is similar when creating new routine or importing, activating or resetting existing routine's schedules.

Multiple schedules

A routine can have many schedules. In this case, the routine will run once for each time window once they are open.

Note that only one time window can be started at a time. Especially, if two or more schedules have the same start time, only the first one evaluated (in the order of creation time) is considered. Therefore, you should not use overlapping schedules: if the routine has many schedules, make sure they all have end times and the time periods do not overlap.

Repeating (polling) schedules

Sometimes you need to start a task that polls some system e.g. once every hour. For this, you can define a schedule to repeat after a given delay during the period between the start and end times. The maximum repeat delay is 23 hours and 59 minutes, because the maximum time window length is 24 hours - if you need to define longer repeat periods, you should use the recurrence cycle options to define the days when the schedule is open. E.g. if you need a routine to execute once every day, there is no need to have its schedule with a 'Repeat every' setting. Only the 'Recurrence' needs to be set to 'Daily' and to 'Recur every' 1 days.

The repeat times within the open period are calculated from the local schedule start time: for example, if the schedule starts at 12:00, and should repeat every 10 seconds, the executions will be scheduled at 12:00:00, 12:00:10, 12:00:20 etc. End times for repeating schedules are not inclusive. This means that a routine with repeating schedule with start time set to 12:00 and end time to 12:30 with repeat interval of 10 minutes will only be executed at 12:00, 12:10 and 12:20 but not at 12:30.

If the FRENDS Iron service has been stopped for a while, when it is started and there are repeating open schedules that should have been executed, they will be launched immediately. The next execution will again use the calculated repeat time.

Note that when a routine has multiple schedules, repeating schedules take precedence over any other schedules because they are assumed to repeat for the entire time window they are open. For instance, if a routine has two schedules, one that starts at 9:00 and ends at 17:00 and repeats every 25 minutes, and another that starts at 16:00 and ends at 20:00, the routine will not run at 16:00 because the second schedule is overlapped by the first one - it will only run at 17:00, when the first schedule has ended. Because of this, you should not use overlapping repeating schedules, because their executions may be in unwanted times.

Importing dates to exclude for a schedule

You can import iCalendar .ics files for schedules so they won't be open on certain dates, for example on holidays.

• Import reads all the events from all calendars.
• Event's UTC start time will be used and this will be converted to local time. Only the date of this local time will be used.
• If there are many events on the same day only one entry will be used.
• For repeating events only the first event will be used.

NOTE: Dates with time zone defined are not supported and these will result in an error message "There was an error while parsing the file. Check manual for supported file types." Lines like DTSTART;VALUE=DATE-TIME;TZID=Europe/Helsinki:20130114T090357 should be converted to DTSTART;VALUE=DATE-TIME:20130114T090357Z.

Daylight saving time effects

The routine schedules are checked according to the server local time, which also takes possible daylight saving time (DST) adjustments into account. This means that if you schedule a routine to run every day at 12:00, it will execute at that time, whether DST is in effect or not.

However, because the scheduler uses the local time with adjustments, the time periods when DST is starting or ending may cause routines to be scheduled a bit differently than normally: In spring, when DST is starting, the clocks are turned forward, and one hour gets skipped. The opposite happens in autumn, when the clocks are turned back, and there is one additional hour. The effects of these adjustments for different schedules are shown in the tables below.

When the DST is starting, one hour (in EU, the hour from 3 to 4 AM) will be skipped, i.e. the clock will skip from 2:59:59 to 4:00:00. For routines this means that the schedules cannot be open during this period, and they are adjusted as follows:

When the DST is ending, one hour will be added to the day, i.e. the clock will be turned back from 4:00 to 3:00. Therefore the local time will be 3:00 twice for that day. For routines that start or stop at this ambiguous hour, the ambiguity is resolved by always choosing the first possible occurrence.

DST changes also affect repeating schedules. Because the repeats are calculated from the schedule's local start time, the repeats will start to adhere to the adjusted local time as soon as a new schedule is opened. The table below shows an example of a polling routine that is scheduled to execute every three hours for the entire day with different start times.

If the schedule start time or end time is adjusted due to the DST change, a warning will be logged to the event log for every time the schedule is checked, i.e. every minute. If the schedule would be skipped, i.e. the schedule starts and stops during the invalid period (3:00-3:59:59), an error is logged.

Monitoring and controlling routines

You can monitor the execution of routines, as well as start and terminate routine instances and activate and deactivate routine schedules from the FRENDS Web UI. See the Monitoring routine execution and Controlling routines sections in the FRENDS Web UI documentation for more details.

Instance states

A routine instance can either be executing or finished. When a routine instance is executing, it has task instances that are executing or waiting to be executed. Once all task instances have finished executing, the routine instance will also be finished and take the end state of latest non-ignored task instance end state. The following tables describe these states.

Executing instance states:

State	Icon	Description
Waiting		FRENDS Iron service is currently under heavy load and the task instance is queued to be launched. This state applies only to task instances. Routine instance is always on Launched state when it is waiting for execution.
Launched		The instance has been launched, but it is waiting for the FRENDS Iron service to start the execution
Starting		FRENDS Iron service is about to start new task execution and is initializing task execution.
Started		Instance is currently executing, and is running the task.

Finished instance states:

State	Icon	Description
Successful		Instance was executed successfully
ActionSkipped		Instance was executed successfully, but it did not do its main action for some reason. This state is usually used for polling tasks, to signal that while the task executed without errors, it did not do anything (e.g. transfer a file).
Failed		Instance has failed. The task was executed successfully, but it returned a false return value, signaling some kind of business level failure.
Exception		An error occurred when executing or stopping the instance. This is a general error that can signal an exception. Please see the logs for more details on the error.
Canceled		The routine was stopped before any task instances were started.
TerminatedByUser		The instance was terminated by user request
FinishedAsZombie		The instance was left in a Started state, even though the task had already finished. This is usually caused by problems with updating the routine instance states to the database. The FRENDS Iron service periodically cleans up old “zombie” instances and sets their state to FinishedAsZombie.

The finished routine instance end state is calculated from the end states of its task instances as follows:

• If any task instances ended in failed state, and the error is not ignored, the routine instance end state will be the state of the failed task instance. If there are multiple failed instances, the most severe error state will be used.
• If a task's failure action is set to ignore errors and the task instance fails, the end state will not be taken into account when calculating the routine instance end state. If there are only task instances in ignored error states, the routine instance end state will be set as Ok.
• If there are only task instances in the ActionSkipped state (or in ignored error states), the routine instance end state will be set to ActionSkipped.
• If there are only task instances in Ok or ActionSkipped states (or in ignored error states), the routine instance end state will be set to Ok, i.e. the Ok state takes precedence over the ActionSkipped state.

Tasks

FRENDS Iron tasks define the execution logic: the steps that are taken to get to the end result. Technically, tasks are parameterized Windows Workflow Foundation (WF) workflows or Simple Tasks which are plain old CLR objects (POCO). You define the logic using Visual Studio, compile the task and deploy it to FRENDS Iron.

Schedules

FRENDS Iron schedules define the times a routine should be launched by the FRENDS Iron process engine. You can re-use existing schedules with different routines.

A routine is executed once for each schedule per day (i.e. 24 hour time window). For instance, if a routine has three open schedules in season, the process engine will launch three routine instances every day. A schedule is said to be open if the all its time conditions are true: for example, the daily start time has passed, but the daily end time has not yet passed.

A schedule must define:

1. Name - Descriptive name for the schedule
2. Daily start time - Time when the FRENDS Iron process engine should launch the routine
3. Season start date - The first day the schedule is open ("in season")
4. The recurrence type - on which days the schedule is open:task
   • - Daily: the schedule is open on every given day, i.e. every day (the default), or every second day and so on.
   • - Weekly: The schedule is open on every given week and on given weekdays
   • - Monthly: The schedule is open only on specified months and either on
     1. Specified dates, like the 13^th or the last of the month, or
     2. Relative dates, like the second Friday or the last weekday of the month, for instance.

The schedule may also define:

• Daily end time -Time after which no further routines should be launched. This value is usually important for repeating routines.

NOTE: Once a routine has been launched, it is allowed to complete even if it goes past its end time

• Season end date - The last day the schedule is open
• Repeats - Repeat the task after the given delay starting from the daily start time, and ending at the daily end time. If no end time is set, the routine will repeat the entire 24 hours in a day, i.e. all the time.
• Excluded dates - The dates when the normal scheduling will not take effect and the schedule will not be open (these dates could be for example holidays). Dates can be added and deleted manually or imported from iCalendar .ics files (see more on import).

For details on how the schedules are used with routines, see section Scheduling routines

Connection points

Connection point contains the parameter information, such as IP address and user name for an FTP connection, on individual endpoints. The parameter data is stored as an XML structure to the FRENDS Iron control database. The data structure adheres to a XML schema, defined when creating the connection point.

You can use connection points in many ways, for instance keeping track of remote endpoint states or storing messages sent to BizTalk. You can access the data either via direct database queries to the FRENDS Iron control database or using connection point references (see Task parameter references.

Connection point data schemas

The connection point data structures are typed using XML schemas. Only very basic XML schema functionalities are supported:

• Simple element definitions
  

  <xs:element name="name" type="xs:string" />
  

• Complex element definitions with a <sequence> of sub-elements:

  <xs:element name="Data"> 
      <xs:complexType> 
          <xs:sequence> 
              <xs:element name="Name" type="xs:string" /> 
              <xs:element name="Id" type="xs:int" /> 
              <xs:element name="Location"> 
                  <xs:complexType> 
                      <xs:sequence> 
                          <xs:element name="Latitude" type="xs:double" /> 
                          <xs:element name="Longitude" type="xs:double" /> 
                      </xs:sequence> 
                  </xs:complexType> 
              </xs:element> 
          </xs:sequence> 
      </xs:complexType> 
  </xs:element>
  

• Simple global type definition:

  <xs:element name="Data"> 
      <xs:complexType> 
          <xs:sequence> 
              <xs:element name="Name" type="xs:string" /> 
              <xs:element name="Id" type="xs:int" /> 
              <xs:element name="Location" type="LocationType" /> 
              <xs:element name="OtherAddress" type="LocationType" /> 
          </xs:sequence> 
      </xs:complexType> 
  </xs:element> 
  <xs:complexType name="LocationType"> 
      <xs:sequence> 
          <xs:element name="Latitude" type="xs:double" /> 
          <xs:element name="Longitude" type="xs:double" /> 
      </xs:sequence> 
  </xs:complexType> 
  

• <all> list, as well as enumeration restrictions
  

  <xs:schema elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema"> 
      <xs:element name="Data"> 
          <xs:complexType> 
              <xs:sequence> 
                  <xs:element name="All"> 
                      <xs:complexType> 
                          <xs:all> 
                              <xs:element name="One" /> 
                              <xs:element name="Two" /> 
                              <xs:element name="Three" type="CarEnum" /> 
                          </xs:all> 
                      </xs:complexType> 
                  </xs:element> 
              <xs:element name="Enum" type="CarEnum" /> 
          </xs:sequence> 
      </xs:complexType> 
  </xs:element> 
  <xs:simpleType name="CarEnum"> 
      <xs:restriction base="xs:string"> 
          <xs:enumeration value="Audi" /> 
          <xs:enumeration value="Golf" /> 
          <xs:enumeration value="BMW" /> 
      </xs:restriction> 
  </xs:simpleType> 
  </xs:schema>
  

Most of the more notable functionalities that are not supported are:

• Element attributes
• Schema imports
• Choice-elements

Schemas that use these features will not be shown correctly in the view: for instance, even if you define minoccurs="3", only a single element will be shown. However, as the schema is only used for initializing the editor, the data can still adhere to the schema - it just needs to be edited externally, and written directly to the database.

If you need to maintain multiple versions of CP schemas, you can use the XML Schema version attribute for distinguishing between the different versions. If the schema has a version, it will be shown e.g. in the Choose Connection Point Schema dialog.

Note: If you want to change target namespace for your connection point schema, you have to create a new connection point. Otherwise old target namespace is used.

Connection points and BizTalk Parties

In BizTalk, role links and parties can be used to model individual training partners - somewhat similar to FRENDS Iron connection points. You can store trading party specific details to them, especially

• Send ports, that store the connection details, e.g. addresses and protocol for that trading partner
• Certificates used for resolving the party at runtime

For more details on the parties, please see the BizTalk documentation.

FRENDS Iron connection points are loosely associated with BizTalk parties: You can give a party name to a special field in the connection point. This value can be used in routine parameters using the {CP:PartyName} reference. You can use this especially in the ready-made BizTalk tasks, which have the PartyName parameter. The value you give to this parameter will be passed on to the FRENDS Iron adapter, which in turn sets it the PartyName message property in the message sent to BizTalk. This property can then be used in orchestrations to associate the process with specific parties. For an example of this, see the FRENDS Tutorial 2.

Note: In FRENDS 2.2 or later, BizTalk parties are no longer created with connection points by default, as they were in 2.0 and 2.1. This was done for performance reasons. Therefore you need to create the parties and set the Party Name field in the connection point properties manually.

Connection point groups

Sometimes you need to send the same message to many different endpoints at once. You can do this easily in FRENDS Iron by using connection point groups. Using them, you can group connection points together using criteria like CP name or tags, and give them to a single routine's task. When the routine is executed, the connection point group will be expanded so that a task instance is launched for each connection point in the group. This allows you to maintain these kinds of processes efficiently.

Connection point group filter

Connection points are grouped by creating a filter that defines conditions for connection points. These conditions are then evaluated for the entire list of connection points, and the ones that fulfill all the conditions will be returned.

The conditions are written in a simple language: each condition has a condition type, an operator and a value. There are two types of conditions:

• CP Name conditions that do string matching on the CP name.
• CP Tag conditions that do string matching on the CP tag values

NOTE: "This Group" conditions from 2.0 and 2.1 are no longer supported, because they cannot be reliably imported between environments - they reference specific CP IDs, which are likely not the correct ones on other environments. Furthermore, unique tags or names can be used instead. Therefore, please change filters using "This Group" conditions to use unique names or tags instead: e.g. "This Group Includes TestCP" -> "CP Name Equals TestCP"

You can group filter conditions together using basic Boolean operators AND, OR, and their negations NOT AND and NOT OR.

Actions only supported in the FRENDS MMC UI

There are a number of features that are currently only supported on the now deprecated FRENDS Microsoft Management Console (MMC) UI.

Editing task parameters and tracking profiles

You can edit task parameters and setup tracking profiles currently only from the FRENDS Iron Management Console as follows:

1. Open the Tasks list, right-click on the row of the task to edit and choose Properties. The Properties dialog is opened.
   
2. Activate the Task parameters tab. Here you can choose, which task properties you want to expose from the task, by either double-clicking on the parameters or choosing the parameter and clicking on the Add >> button.

   NOTE: Only the parameters you choose here can be given in the routine task arguments view

3.  Click OK. The task is updated.

You can also choose to enable the default tracking profile by checking the "Enable tracking profile" checkbox. If set, it will generate a tracking profile for tracking the values of each property of each value, allowing the values to be viewed from the Workflow Monitor.

Please note that enabling the tracking profile is only recommended for development and debugging, because when on, tracking can generate large amounts of data, especially for large workflows with large parameters, such as FRENDS Cobalt.

Viewing workflow execution details

From the Routine instances view you can see if the task execution was successful: if it was, the view will show a green icon and the state "Ok".

You can view the details of the execution by right-clicking on the task instance row and choosing View execution (only workflow tasks supported). This brings out the workflow monitor view that shows the flow of execution in the task and the parameter values of all executed activities (if the tracking profile was enabled).

Changing or adding a connection point schema

Currently you can only change the schema of an existing connection point from the FRENDS MMC UI:

1. Connection Points list, right-click on the row of the task to edit and choose Properties. The Properties dialog is opened.
   
2. Change the connection point schema:
   1. Click on Load Schema.
   2. In the Choose Connection Point Schema dialog, you can choose from already loaded schemas or load a new schema by clicking on the Import button in the upper-right corner to open a XML schema file.
   3. Click Ok.
   
   
3. The Connection point Parameters grid tries to keep the existing data: if the new schema structure and names of the elements are the same as the previous one (e.g. you only added a new field to the schema), the data should remain the same as before.
4. Click Ok. The schema and data is changed.

Currently you can only add new connection point schemas either by importing them with connection points, or from the "Choose Connection Point Schema" dialog in the FRENDS MMC UI (see image above). Clicking on the Import button will open a file selection dialog. The selected xsd file will be imported as a new schema in the database.

Editing encrypted connection point fields

You define and edit new encrypted connection point fields and their values in the CP property dialogs in the FRENDS Iron Management Console: Just define the values in the Encrypted Fields tab.

Once you click OK, the data is encrypted to the FRENDS database, and only the FRENDS Iron Service can read and use the data - even the system administrator cannot see the encrypted field values in plain text in the Management Console or in the database.

Technical details

Task parameter references

You can use the details of the routine or information stored in a connection point to parameterize the execution of routine tasks. You do this with task parameter references. They are parameter strings enclosed in curly braces and starting with the type name: "CP", "ECP", "ROUTINE", "INSTANCE" or "RESULT". The references are expanded when the task instance is launched: the scheduler expands the value and writes it to the serialized XML parameters for the task, which are then used when starting the actual execution.

Currently there are four CP references you can use:

• CP Name Reference: "{CP:Name}" - The name of the Connection Point
• CP ID Reference: "{CP:ID}" - The ID of the Connection Point
• CP Party Name Reference: "{CP:PartyName}" - The BizTalk party name associated with the Connection Point
• CP Data Reference: "{CP:<XPath>}" (where XPath refers to an XPath query string) - A reference to the data stored in the Connection Point. When using this reference, the XPath query is executed on the XML data structure associated with the Connection Point of the routine.
• Encrypted CP Field Reference: "{ECP:<FieldName>}", where FieldName refers to the encrypted CP field name - for more on this, see the next section, Encrypted connection point fields

In addition, there are two routine references and one task instance reference:

• Routine Name Reference: "{ROUTINE:Name}" - the name of the Routine
• Routine ID Reference: "{ROUTINE:ID}" - the ID of the Routine
• Routine Execution ID: "{ROUTINE:RoutineExecutionId}" - the persistent ID for a single routine execution
• Task instance execution ID: "{INSTANCE:ExecutionID}" - the execution ID (GUID) for the individual task execution. The database column is WORKFLOW_INSTANCE_ID, and this value is e.g. set as Cobalt's BatchId.

There are some things to note when using the CP Data References:

• If the XPath query returns an XML data structure, the structure is transformed to a string (using XmlElement.OuterXml()).
• If the XPath query returns more than one node, the first one is used.
• If the XML data defines namespaces, you need to provide the namespace information in the query.

For example uses of the references, see the table below.

CP reference examples:

Run-time references

In addition to the task parameter references that are expanded when the routine is launched, there are some references that are expanded only during run-time. The most important of these are:

• "{RESULT}" (or "{PREVIOUSTASK}") - the reference for sharing information between tasks, and

Creating your own tasks

FRENDS Iron tasks can be created as a Simple task (plain old CLR object (POCO)) or a .NET 3.5 Windows Workflow Foundation workflow (Simple task support was added in the FRENDS Iron v.3.3). Note that .NET 4.0 Windows Workflow Foundation workflows are not supported. You can create your own tasks easily using Visual Studio. The workflow assembly needs to be compiled with the target framework set to .NET 3.5 but a Simple task can also use .NET 4.0.

For information on creating a new workflow using Visual Studio, please see the WF documentation, now part of the latest Microsoft Windows SDK documentation.

You can use almost any kind of WF workflow with FRENDS Iron. As the workflows are executed internally by the FRENDS Iron execution engine, you cannot expose any Web Service interfaces from the workflow, but otherwise there are no limitations: you can even provide your own WF runtime service (see Providing your own WF runtime service) for more complex interactions with external systems.

There are some properties which are handled specially by FRENDS Iron:

• Success - boolean value indicating the success of the task
• UserResultMessage - string value which is shown in the user interface
• Output - string value which can be accessed by consecutive task instances
• ActionSkipped - boolean value indicating that the task did not really do anything
• ExecutingTaskContext - dictionary of additional information of the executing task context

Task return value

Often you would need to handle a situation where a task execution had a logical failure: the execution went as it was supposed to and there were no timeouts or other technical failures, but the task failed, e.g., a called service declined the request. In these situations you would not want to throw an exception, because the situation was not exceptional, but still would like to communicate that the task did not go through as normal to the end-user.

You can communicate a return value from the task to the FRENDS Iron process engine by providing a Boolean output parameter with the name "Success", and setting the value to true if the task execution was deemed successful and false otherwise. The FRENDS Iron process engine reads the value of this property when the task has been completed, and sets the result state of the routine to "Ok" if the property was true and "Failed" if false.

NOTE: If the "Success" property is not found on the task class, the execution is considered to be successful, and the routine is set to "Ok" state.

You can also provide a custom string message that is displayed in the routine instance's task instance list once the task has completed. To do this, you only need to define a string property named "UserResultMessage" with the message in your task. The process engine will read this value and store it in the history database, from which it is then shown by the user interface.

Sharing information between tasks

The task instances can receive information about the previously executed task instances with the custom run-time reference "{RESULT}", the reference is expanded as the PreviousTaskResult. Task instances which were launched in the same group are not included in the previous task result set, even though they might have finished before starting this task.

You can share data between consecutive tasks with the "Output" String property. The Output element of the PreviousTaskData contains the contents of the property. It can be used to share information, e.g. by serializing objects in xml or json.

The {RESULT} custom reference is replaced with an XML-structure with the following structure:

• PreviousTaskResult - root node
  • PreviousTaskData - element for each previously executed task instance.
    • Name - The name of the routine task
    • State - The end state of the task instance
    • Id - Task instance's execution time id
    • Task - The name of the task
    • Routine - The name of the routine
    • StartTime - The execution start time of the task in UTC
    • EndTime - The execution start time of the task in UTC
    • ConnectionPoint - The name of the connection point used by the task if any
    • ConnectionPointGroupId - The GUID of the connection point group used by the task, the id is same for each task instance in the same group
    • UserResultMessage - The user result message provided by the task
    • Output - The output provided by the task in the string property "Output".
    • StepId - The GUID of the process step for connection references
    • PreviousStepIds - The list of GUIDs of the steps that preceded the step
    • Distance - The number of connections between this step and the current step

Example:

<?xml version="1.0" encoding="utf-16"?>  
<PreviousTaskResult xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 
  <PreviousTaskData> 
    <Name>First task</State> 
    <State>Ok</State> 
    <Id>1</Id> 
    <Task>Sleep</Task> 
    <Routine>Demo</Routine> 
    <StartTime>2013-02-27 12:23:39Z</StartTime> 
    <EndTime>2013-02-27 12:23:39Z</EndTime> 
    <ConnectionPoint /> 
    <ConnectionPointGroupId /> 
    <UserResultMessage />
    <Output /> 
    <StepId>2ab53c46-bfdb-4729-8228-9f2092d411ab</StepId>
    <PreviousStepIds />
    <Distance>2</Distance>
  </PreviousTaskData> 
  <PreviousTaskData> 
    <Name>Second task</State> 
    <State>Ok</State> 
    <Id>2</Id> 
    <Task>Send file</Task> 
    <Routine>Demo</Routine> 
    <StartTime>2013-02-27 12:23:40Z</StartTime> 
    <EndTime>2013-02-27 12:23:41Z</EndTime> 
    <ConnectionPoint /> 
    <ConnectionPointGroupId /> 
    <UserResultMessage />
    <Output>1932978865</Output> 
    <StepId>985d9482-f00c-462c-bcc6-386069f23f6d</StepId>
    <PreviousStepIds>
      <guid>2ab53c46-bfdb-4729-8228-9f2092d411ab</guid>
    </PreviousStepIds>
    <Distance>1</Distance>
  </PreviousTaskData>
</PreviousTaskResult> 
		

XPath can be used for fetching specified element, value or XML structure from PreviousTaskResult. Syntax for using XPath is as follows: {RESULT:XPath}.

For example, {RESULT:/PreviousTaskResult/PreviousTaskData[Distance='1']/Output} for the example above, would return 1932978865 which could be used as parameter for a task. XPath like {RESULT:/PreviousTaskResult/PreviousTaskData[Distance='1']/PreviousStepIds} would return an XML structure like:

<PreviousStepIds>
    <guid>2ab53c46-bfdb-4729-8228-9f2092d411ab</guid>
</PreviousStepIds>

The same restrictions apply as with the CP reference XPaths.

You may also use the shorthand version {PREVIOUSTASK}, which will only return information for the previous task. When used without any parameter, it is executed as {RESULT:/PreviousTaskResult/PreviousTaskData[Distance='1']}. There are three possible parameters:

• {PREVIOUSTASK:state} - Returns the State element contents of the previous task
• {PREVIOUSTASK:userresultmessage} - Returns the UserResultMessage element contents of the previous task
• {PREVIOUSTASK:output} - Returns the Output element contents of the previous task

ActionSkipped return value

If your task has a logical precondition, e.g. it checks if a directory has any files before trying to copy them, you may want to show a different end result if the precondition failed and the actual action was skipped. Since version 3.1, FRENDS Iron has a separate end state for task instances like this: ActionSkipped. It signals that the execution was successful, but did not do anything.

To show the task instance end state as ActionSkipped, you need to provide two Boolean output parameters and set their value to true:

• Success -the property already discussed above that signals that the task executed successfully
• ActionSkipped - the new property signaling the task failed some precondition and did not execute its main action

ExecutingTaskContext input property

ExecutingTaskContext property is a key value dictionary that includes parameters of the executing task context. For example parameters include TaskId, RoutineId, ConnectionPointId (if available), etc. These parameters can be used freely by the task if needed.

Providing your own WF runtime service

For activities that would require more than a simple calculation, e.g., communication with external systems or other potentially long running processes, you should create a local WF service for handling the actual work, so the workflow execution does not block unnecessarily. For details on creating a local WF service, see the WF documentation.

Once created and compiled, the service can be added to the runtime as follows:

• The assembly containing the service should be copied to the FRENDS Iron installation directory.
• In the FRENDSAccService.exe.config file, you can add the service to the set of available runtime services simply by adding a new entry under the \configuration\workflowRuntime\services node. For details on the configuration formats, see the MSDN for Workflow configuration files.

Workflow versioning

If you change a workflow that has already been deployed and is in use, you must change the assembly version. Otherwise, you will run into problems when using tracking profiles and when dehydrating persisted workflows. If, for instance, a workflow is suspended and persisted in the database, waiting for a message, updating the workflow assembly without changing the version will make the runtime to try to rehydrate the stored state using the new assembly. If there are any changes in the workflow, e.g. new activities or variables, this dehydration is likely to fail.

Encrypted connection point fields

Encrypted Connection point fields allow you to store login information and other sensitive data securely.

In order to secure the field data but at the same time allow changing it in the UI, the field data is encrypted in two phases: First the UI encrypts the data using AES with a 256-bit symmetric key. Then the symmetric key is encrypted using the public key from an asymmetric 2048-bit key pair (generated during initial configuration - see the configuration tool documentation for more on this), and stored to the database with the encrypted data. The UI knows the public key because it is stored to the UI configuration file.

Only the FRENDS Iron service can decrypt the symmetric key, because only it knows the private asymmetric key, which is stored to the service configuration file. To secure this private key, it is encrypted using Windows' DPAPI so that only the FRENDS Iron service user account can decrypt and use the private key.

The encrypted value can be used by a routine by passing it as a Task Argument. This is done by setting the argument field {ECP:<Encrypted field name>}. Example: {ECP:password}

NOTE: If the key pair or FRENDS Iron user account is changed, the FRENDS Iron service cannot decrypt the private key anymore. Therefore it is recommended to backup the key pair by exporting it to a file. This is described in the Configuration Tool documentation.

Web Service interfaces

FRENDS Iron service publishes three web service interfaces for routine executions as well as querying for

• Routine REST API – For synchronously starting routine instances and waiting for them to finish, as well as querying for the status of individual executions.
• WCF Control service – For sending one-way action messages to e.g. start a new routine execution or activate a routine.
• WCF CP service – For querying and updating data from connection points

Routine REST API

The Routine Web API allows you to start new routine instances and wait for them to finish, as well as check the status of a specific routine instance.

The API uses simple REST -style calls. The service will be hosted from the FRENDS Iron service (FRENDSAccService.exe) at port 9998 by default, with the base address scheme of /api/routineinstances/.

POST to start a new routine instance

In order to start a routine instance and wait for it to finish you just POST a message to the URL http://[server name]:9998/api/routineinstances/[routine name]

This will start a new instance of the routine with a matching name. Note that the routine name needs to be URL encoded, e.g. spaces need to be encoded as '%20'

You can give the timeout parameter as an URL parameter to the query. It is the time in seconds to wait for the started routine instance to finish, e.g./api/routineinstances/Test%20Routine?timeout=30. The timeoutis 10 seconds by default.

You can also pass a JSON structure for in the message body. If the name of any field in the structure matches the name of a task parameter, the value in the JSON will override the value given in the routine task.

For instance, the following JSON fill any task argument named EmployeeId with the value 22:

  {
    "EmployeeId": "22"
  }
    

Please note that the matching parameters will be replaced for all tasks, not just the first one. This means that if the routine has many same tasks, or tasks with same parameter names, they will all be overwritten.

If no routine matches the given name, HTTP error 404 is returned. If there are multiple routines with the same name (there should not be, as elements with duplicate names are not supported by FRENDS user interfaces), an error is returned.

The service will wait for 10 seconds by default for the routine instance to finish. If the instances finishes in time, the HTTP result will be 200, with the results of the routine task executions serialized into a JSON structure. The successful result contains the execution details of all executed task instances (Note: the list will contain only the task instances that have been executed, in the order of execution - any branches not taken will not show in the list)

{
"instance": {
        "State": "Ok",
        "RoutineStartTime": "2013-12-03T09:22:30.473Z",
        "RoutineEndTime": "2013-12-03T09:22:36.513Z",
        "RoutineExecutionId": "df304fa0-0932-4522-9ff5-3a8ed51204db",
        "RoutineName": "Test process",
        "ScheduleName": "Immediate execution",
        "TaskInstances": [
            {
                "Name": "Process input to file",      
                "State": "Ok",
                "TaskName": "Insert to DB",
                "StartTime": "2013-12-03T09:22:30.517Z",
                "EndTime": "2013-12-03T09:22:32.533Z",
                "ConnectionPointName": null,
                "ConnectionPointGroupId": null,
                "UserResultMessage": "",
                "Output": ""
            },
            {
                "Name": "Send acknowledgement",            
                "State": "Ok",
                "TaskName": "Send XML",
                "StartTime": "2013-12-03T09:22:33.51Z",
                "EndTime": "2013-12-03T09:22:36.513Z",
                "ConnectionPointName": null,
                "ConnectionPointGroupId": null,
                "UserResultMessage": "",
                "Output": ""
            }
        ]
    },
    "url": "/api/RoutineInstances/df304fa0-0932-4522-9ff5-3a8ed51204db"
}
    

If the instance does not finish in time, the HTTP result will be 202, and the result will not contain the instance details. Instead, the link in url field can be polled for the result. Once the routine instance has finished, the instance details will also be returned.

{
    "message": "Instance is still being processed",
    "url": "/api/RoutineInstances/30e7fb8c-7146-4577-b1e8-9147ed14661a"
}
    

The scheduler may also be paused at the time when the POST message is received. In this case, the new routine instance will be launched only, not started. The execution will start once the scheduler is resumed.

GET to get routine instance status

You can do a GET to the /api/routineinstances/[routine instance execution ID] URL to get the status of the given routine instance. The result is the same kind of JSON structure as defined above for the POST action, i.e. it will return the state of the routine instance and the statuses of any already executed task instances.

Security

In order to call the API, the client needs to authenticate using Windows authentication, i.e. the user doing the call must be a domain user and authenticate.

Unfortunately the Iron 3.4 version of the REST API does not support HTTPS connections with authentication.

WCF Control service

The FRENDS process engine has a number of commands that can be used to control routine status from outside the main engine process.

The commands are provided as a Web Service interface, available at the port and location specified during installation (see Installing FRENDS Iron for details). The WSDL description of the service can be accessed by navigating to the Web Service endpoint by default.

StartRoutine

StartRoutine creates a new routine instance of the given routine and starts it up immediately. Possible schedules are ignored.

StartRoutineWithParameters

StartRoutineWithParameters also starts a new routine instance, but also accepts parameters, given as name-value pairs. These values will be used instead of the values given in the routine parameters. If no value is given in the call, the value given in the routine parameters will be used as the default.

StartRoutineByNameAndWithParameters

StartRoutineByNameAndWithParameters works the same way as StartRoutineWithParameters, but the routine is defined by name instead of routine ID. This makes deploying routines on other hosts easier: when you import a routine, the ID will be initialized to a value unique in the database at that time - therefore the IDs will most likely not be equal to the original ID.

StopRoutine

StopRoutine terminates all instances of a routine that are currently executing. See StopRoutineInstance below for details on what happens when a routine instance is terminated.

Stop RoutineInstance

StopRoutineInstance terminates a single instance of a routine workflow. The WorkflowInstance.Terminate() call is used for this, so the execution cannot be resumed anymore after it has been terminated.
It is recommended a workflow terminates as soon as possible when a stop command is issued, but the workflow may want to do some cleanup before terminating. For an example of this, see the provided BizTalkTwoWayActivity, which cancels the wait for a response from BizTalk.

ActivateRoutine

By default, when a routine is created, it is left in the 'Deactivated' state, meaning it will not be triggered according to a schedule, but it must be started manually. To change this behavior, you need to activate the routine, which is done with the ActivateRoutine command. The command changes the state of the routine to 'Waiting'.

DeactivateRoutine

The DeactivateRoutine command in turn changes the state of a routine to 'Deactivated'.

RescheduleRoutine

Routines are scheduled only once for each time window in their schedule, meaning that a routine with the default schedule 'Always' will execute once a day, at midnight. If you change the schedule of a routine or otherwise want to make the FRENDS Iron task process engine re-evaluate the schedule, you can use the RescheduleRoutine command.

WCF CP service

You can query and update connection points through a Web Service interface. The URL for the Web Service is defined in the FRENDS Iron Configuration tool . Three different operations are provided: QueryCP, QueryCPValue and UpdateCP.

QueryCP

QueryCP operation sends a CPQuery message and returns a list of CPInfo messages. The CPQuery message defines the query fields. All of the fields are optional, but you need to define at least one field. If you give multiple fields, all of them must be true for a connection point for it to be returned.

• CpId, CpName and PartyName refer to the corresponding CP properties - NOTE: name fields are expected to be an exact match!
• CpGroupId and CpGroupName refer to the connection point group properties. As with CP properties, these are also expected to be exact match.
• CpDataXQuery allows you to query for only those CPs with a specific value stored in the CPData. You give an XQuery that is matched to the CPData structures, and if there is a match, the CP is returned. An example XQuery: "//Id/text()[contains(.,"1")]". It selects all CPs that contain the number 1.

QueryCPValue

QueryCPValue is like QueryCP, but instead of returning a CPInfo it returns a string which contains the data from CPData selected with an XPath query.

UpdateCp

UpdateCP modifies an existing Connection Points fields by sending a CPInfo which contains the Id of the CP to modify and the new values. It is meant to be used in conjunction with QueryCP: First, fetch CPInfo structure for the connection point by QueryCP. Then modify the structure and send it to UpdateCP. An example of this can be found in the FRENDS Iron installations Help in the Example-ConnectionPointWebService-directory.

NOTE: Connection point's PartyName or CP Schema cannot be updated via the Web Service interface.

Workflow persistence

FRENDS Iron uses Windows Workflow Foundation for defining and executing individual tasks. To allow for long-running processes and error recovery, the workflows are automatically persisted to a database. This allows the execution of a workflow to be resumed after long periods of inactivity or server restarts. FRENDS Iron uses the standard SqlWorkflowPersistenceService to persist the workflows to the WorkflowStore SQL Server database.

The execution will be continued from the latest persistence point. Workflows are persisted on following occasions ( http://msdn2.microsoft.com/en-us/library/aa663362.aspx ):

• After executing a transaction scope or another activity marked with the PersistOnCloseAttribute.
• Before workflow instance completion or termination
• When the workflow instance goes idle (for five seconds) or is unloaded

These rules apply to any FRENDS Iron task workflows: if there are no delays or activities that demand persistence, the workflow will get persisted only when it completes. Therefore, if the server shuts down unexpectedly, the task workflows may not be resumed: the next execution will begin from the start of the task. You should usually consider tasks as stateless, transient.

If you need to make sure the tasks will continue from a specific location, you should explicitly persist the workflows. You can do this either with a custom activity decorated with the PersistOnCloseAttribute, or use a transaction scope.

Tracking activities

Workflow event logging

By default FRENDS Iron logs all workflow runtime error messages to the event log. You can change the logging settings from the FrendsAccService.exe.config file by modifying the value parameter on any of the switches in the "system.diagnostics" section. Allowed values can be found in MSDN.

    </system.diagnostics> 
    ... 
    <switches> 
        <add name="System.Workflow.Runtime" value="Error" /> 
        <add name="System.Workflow.Runtime.Hosting" value="Error" /> 
        <add name="System.Workflow.Runtime.Tracking" value="Error" /> 
        <add name="System.Workflow.Activities" value="Error" /> 
        <add name="System.Workflow.Activities.Rules" value="Error" /> 
    </switches> 
    </system.diagnostics> 

Workflow tracking

FRENDS Iron uses the default WF SqlTrackingService to store workflow execution details to the SQL database. For the workflows that have been run from FRENDS Iron, you can access this data simply by right clicking on the executed task instance and selecting View execution in the Routine Instance list view. All public properties of the workflow are tracked and shown in the Workflow Monitor by default. If you don't want to show the property values in the Workflow Monitor, you can assign the PasswordPropertyTextAttribute to the property. This needs to be done during design time in the Visual Studio workflow development environment.

Windows Workflow Foundation provides a tracking service that stores the execution details of an executed workflow. By default, only the details on what activities were executed, are stored. If you wish to store other information from the execution, you need to define and store a tracking profile.

With FRENDS Iron, there are two ways of creating a WF tracking profile:

1. You can create a default tracking profile when creating a new task by selecting the checkbox "Enable Tracking Profile". This will create a tracking profile that tracks all workflow events, activities and extracts all string or value type data.
2. You can create a tracking profile by hand, using, e.g., the Workflow Tracking Profile Designer, distributed as a sample in the .NET framework 3.5 documentation. For details on using the designer, please refer to the framework sample documentation or FRENDS Tutorial 3.

You may consider creating a task without a tracking profile (or removing an existing profile) especially if a task is used in a repeating routine - large amounts of executions with the default tracking profile may cause the tracking database to fill up. The tracking profile of a task can be deleted by unchecking the "Enable Tracking Profile" checkbox in the task properties dialog and clicking OK. This removes the tracking profile, and after restarting the FRENDS Iron service, no extra execution details like parameter values are stored to the tracking database. You can create a default new tracking profile by checking the "Enable Tracking Profile" checkbox again.

LOG table

The LOG table in the FRENDS Iron control database is meant for simple activity logging: you can use it to store key-value pairs with timestamp information. You can store e.g. the latest status of an order process, or keep track of the total order amounts.

The LOG table has the following structure:

The data stored to the table could be as follows:

To add entries to the LOG table, it is recommended to use the provided stored procedure, [dbo]. [AddLogEntry] It inserts the current timestamp to the new row automatically. Reports can then be generated by querying rows using the key columns, taking aggregates as needed.

The LOG table is not a full-blown business activity monitoring (BAM) implementation. It is meant to be used when you only need to track a small number of activities (< 100 updates per minute), and the data you need to track can be defined at design time. As said, you also need to create your own user interfaces for the data. In many cases it may be wise to use a ready BAM solution or create your own log system.

Ready-made WF elements

FRENDS Iron uses Windows Workflow Foundation for defining and executing its processes. FRENDS Iron comes with a set of ready-made WF activities and workflows that can be used for basic BizTalk integration.

WF Activities for FRENDS Iron

FRENDS Iron comes with a set of ready-made Windows Workflow Foundation Activities, mainly for integration with BizTalk Server. The activities are installed in the Frends.Acc.WF.BizTalk.dll assembly. You can use the activities in your own workflows created with Visual Studio 2008 and hosted in FRENDS Iron.

BizTalk Two-Way Activity

The BizTalkTwoWayActivity is a sequence activity that sends a single message to the specified BizTalk Server receive location and waits for a response. The activity also has a timer for specifying a timeout for the wait.

This activity is used in the BizTalkTwoWayWorkflow. It has three mandatory parameters:

• Uri - the URI of the BizTalk receive location to send the message to.
• TimeOutSeconds - the timeout in seconds. If no timeout is specified, the default is 10 minutes.
• InMsg - the message to send to BizTalk.

In addition, you can provide the PartyName as a parameter to the activity. This name will be assigned to the DestinationParty property in the message sent to BizTalk.

There are two parameters for activity output:

• MessageId - the Id of the actual message sent to BizTalk. This parameter is set when the message has been sent so you can track the process also when it is executing in BizTalk, using HAT, for instance.
• OutMsg - the response message from BizTalk.

BizTalk One-Way Activity

The BizTalkOneWayActivity is a one-way activity that sends a single message to the given BizTalk Server receive location and returns without checking for the response.

This activity is used in the BizTalkOneWayWorkflow. It has two mandatory parameters:

• Uri - the URI of the BizTalk receive location to send the message to
• InMsg - the message to send to BizTalk.

As with the Two-Way activity, you can also define the PartyName property.

There is also one output parameter:

• MessageId - the Id of the actual message sent to BizTalk. This parameter is set when the message has been sent so you can track the process also when it is executing in BizTalk, using HAT, for instance.

BizTalk Receive Location Activity

The BizTalkReceiveLocationActivity is meant for enabling and disabling BizTalk receive locations. This can be useful if you want to take advantage of FRENDS Iron scheduling with existing receive locations.

BizTalk Send Port Activity

The BizTalkSendPortActivity is meant for starting and stopping BizTalk send ports. This can be useful if you want to take advantage of FRENDS Iron scheduling with existing send ports.

WF Workflows for FRENDS Iron

In addition to a set of ready-made WF Activities, the FRENDS Iron installation includes a set of WF Sequential Workflows that make it possible to use the ready-made Activities out-of-the-box, without the need for creating a new workflow using Visual Studio 2008.    

The BizTalk workflows can be found in the assembly Frends.Acc.WF.BizTalk.dll:

• BizTalk Two-Way Workflow - meant for sending a single message to BizTalk, and waiting for the response. The single activity in it is the BizTalkTwoWayActivity.
• BizTalk One-Way Workflow - meant for sending a message to BizTalk Server without waiting for a response. The single activity in it is the BizTalkOneWayActivity.

There are also three ready-made WF workflows meant for simple preconditions (in assembly Frends.Acc.WF.Conditions.dll):

• Always Condition - a sample condition workflow used for creating the system conditions 'Always True' and 'Always False'. It merely sets the 'Success' parameter to a value according to its rule definition.
• File Exists Condition - simple workflow with a single policy activity and a "FilePath" parameter.
• Ping Condition - a sample condition workflow that pings the host (given in the "Host" parameter) and sets the "Success" output parameter to true if the host responded.

BizTalk Server integration

FRENDS Iron is meant to work on top of BizTalk Server. The idea is that BizTalk is used for all those tasks it does well, like message transformations, connections to backend systems, long-running transacted processes, processing large amounts of messages etc.

FRENDS Iron is used for triggering and parameterizing these BizTalk solutions, either according to a schedule or manually. FRENDS Iron process engine registers itself as an isolated adapter to BizTalk, and sends the trigger messages to BizTalk using the adapter interface. A configured receive location in BizTalk is all that is needed for receiving the messages from FRENDS Iron.

Submit Adapter

The Submit adapter used by FRENDS Iron uses the SubmitDirect sample programmatic adapter from BizTalk Server SDK. For more information on this adapter, see the SDK documentation.

Registering the FRENDS Iron submit adapter to BizTalk

Before you can create a receive location for FRENDS Iron, the Submit adapter must be registered to the BizTalk group. The adapter is installed by the setup program and added to the group's list of available adapters. If you need to install FRENDS Iron on multiple hosts in a group, you don't need to install the submit adapter separately.

If the adapter is not correctly installed on the server, you may need to install it yourself later on as described below:

1. Open the BizTalk Server Administration Console and expand BizTalk Server Administration > BizTalk Group [Name] > Platform settings > Adapters.
2. Right-click on the Adapters node and choose New > Adapter from the menu.
3. In the Adapter properties dialog, give the adapter a name, like 'FRENDS' and choose 'Submit' from the Adapter combo box.
4. Click Ok. The new adapter should show on the list.

Receive location configuration

In order to pass messages from FRENDS Iron to BizTalk Server, you need to specify a Receive Location and a Port, using the Submit Adapter:

1. Open the BizTalk Server Administration Console and expand BizTalk Server Administration > BizTalk Group [Name] > Applications > BizTalk Application 1 > Receive Ports .
2. Right-click on the Receive Ports node and choose New > Request Response Receive Port... from the menu.
3. In the Receive Port Properties dialog, give the Receive Port a name, like 'FRENDS In'.
4. Choose Receive Locations from the list on the left side of the dialog, and click New... to create a new Receive Location on the Receive Port.
5. In the Receive Location Properties dialog, give the Receive Location a name, like 'FRENDS In'.
6. Select Submit as the Transport Type from the combo box, and click Configure....
   

   NOTE: If you are using BizTalk Server 2006/2009, then you need to configure this feature from the FRENDS Iron Management Console BizTalk Snap-in. This is because all dll's starting from Iron 3.3 are compiled for .Net 4.0.

7. In the Submit Transport Properties dialog, give the Uri property a unique value, like 'Tutorial1'.
   

   NOTE: This value is used for passing the message from FRENDS Iron to the correct endpoint, and is needed for configuring the FRENDS Iron task later on.

8. Click Ok when done.
9. In the Receive Location Properties dialog, set the Receive Pipeline to "XmlReceive", so the message in FRENDS Iron will be parsed and the properties promoted.
10.  Click Ok. The receive port and location should be available.

Logging to database with FileTable

FRENDS Iron can be configured to use Microsoft SQL Server FileTable to save log files to the database. By saving the logs to the database, you can leverage SQL Full-Text Search capabilities.

NOTE: FileTable requires Microsoft SQL Server 2012.

Configuration guide

• Enable SQL Server FILESTREAM feature:
  1. Open SQL Server Configuration Manager
  2. Select SQL Server Services
  3. Right click SQL Server instance which contains FRENDS Iron databases and click Properties
  4. On the FILESTREAM-tab: Check Enable FILESTREAM for TRANSACT-SQL access and Enable FILESTREAM for file I/O access
• Run the following SQL-script which
  • enables the filestream for the FRENDS database
  • creates the filegroup and file in the same directory as the FRENDS database
  • creates the 'LogFiles' FileTable

  -- ENABLE filestream access on DB
  EXEC sp_configure filestream_access_level, 2
  RECONFIGURE
  GO
  -- Add FILESTREAM File group to FRENDS DB
  ALTER DATABASE FRENDS
  ADD FILEGROUP LogFilesGroup CONTAINS FILESTREAM 
  GO 
  ALTER DATABASE FRENDS
      SET FILESTREAM ( NON_TRANSACTED_ACCESS = FULL, DIRECTORY_NAME = N'FrendsFileStream' );
  GO
  DECLARE @frendsDbDir varchar(255);
  -- GET FRENDS DB Location
  SET @frendsDbDir = (SELECT TOP(1) SUBSTRING(physical_name, 1, CHARINDEX(N'frends.mdf', LOWER(physical_name)) - 1) DataFileLocation
  FROM master.sys.master_files
  WHERE name = 'FRENDS' AND type_desc = 'ROWS')+'FrendsFileStreams'
  -- ADD Filestream file to FRENDS DB Location
  EXEC( 'ALTER DATABASE FRENDS ADD FILE
  (
  	NAME = LogFiles
  	,FILENAME=['+@frendsDbDir+']
  ) TO FILEGROUP LogFilesGroup')
  GO
  -- CREATE LogFiles FileTable
  CREATE TABLE LogFiles AS FILETABLE
    WITH
    (
      FILETABLE_DIRECTORY = 'LogFiles',
      FILETABLE_COLLATE_FILENAME = Finnish_Swedish_CI_AS
    )
  GO
  			

• Open Logging Settings from the FRENDS Iron Management Console by right clicking the FRENDS Iron service node and selecting Configure Logging. Set the Log directory to the FileTable share, e.g. \\[SqlServerMachineName]\[FilestreamWindowsShareName]\FrendsFileStream\LogFiles
• From now on the log files are saved to the LogFiles FileTable and can be queried. NOTE: The log file currently being written to is locked while the service is running. It may be prudent to use a smaller Maximum file size for the log.

StreamInsight supported

FRENDS Iron is Microsoft StreamInsight ready, you can install StreamInsight alongside FRENDS Iron and develop a StreamInsight project that leverages data from FRENDS Iron.

Support for custom columns in Web UI

Web UI installation folder includes Autofac.config. This file can be used to configure custom columns to be shown in Web UI views. Custom modules need to be placed in Web UI/bin folder.

Troubleshooting

While developing FRENDS Iron solutions is rather easy, there are many things that can go wrong in the process. This section gives some guidance on what to do in case of errors while executing FRENDS Iron processes.

For troubleshooting concerning FRENDS Iron configuration see FRENDS Iron Configuration Tool Documentation.

Finding the reason for the error: viewing logs

The first step in fixing an error is to find out what caused it. The FRENDS Iron user interface shows the user simple error messages when it encounters an error, but the technical details of the errors are written to a log file. For the FRENDS Iron process engine, no user interface is available, so all error and informational messages are written to a log.

All in all, the log entries are written in two locations.

• Critical errors from the FRENDS Iron process engine service are logged to the Windows Event log.
• Informational and debug events are logged to files located under the directory Logs\ in the FRENDS Iron installation folder:
  • FRENDSAccService.log for FRENDS Iron process engine events, and
  • FRENDSAccManagementConsole.log for FRENDS Iron UI events.
  • In debug builds, the general trace log listener is also configured on by default for the FRENDS Iron process engine service. It logs the following information to the file TraceOutput.log:
    • All informational, debug and trace messages of the process engine
    • All messages from the WF runtime

Checking the event log for critical process engine errors

FRENDS Iron process engine exceptions are logged to the event log, under the event source "FRENDS.Acc.Service". The log entries contain the exception message along with execution trace.

Checking the log files for execution details

Both the FRENDS Iron process engine service and the FRENDS Iron Management Console write their informational messages to a log file. These files are located in the Logs\ directory under the FRENDS Iron installation directory.

Just open the log files using a text editor. For diagnosing errors, it is usually best to navigate to the end of the file to see the most recent messages, and look especially for WARN and ERROR entries.

In some situations, checking the trace output log can also be beneficial, at least in diagnosing where the actual error happened.

Contacting FRENDS Support

If the following common solutions do not provide the necessary assistance, contact our support: support@frends.com. They can provide you with additional support and debugging symbols if necessary.

Common errors and fixes

BizTalk reports a routing failure when sending a message from FRENDS Iron

You try to use the "BizTalk One-Way call" or "BizTalk Two-Way call" task to send a message to BizTalk, and the message seems to have been sent, but BizTalk reports the message could not be routed because no subscriptions were found.

BizTalk routing failures can be caused by multiple reasons. The following causes, with simple fixes, are probably the most common:

The BizTalk receive location uses the PassThru Receive pipeline

If the BizTalk receive location uses the PassThruReceive pipeline, the schema of the message is not recognized and no subscriptions that depend on it will match the message.

Set the FRENDS Iron receive location to use the XmlReceive pipeline.

The message sent is empty

The message sent to BizTalk is defined by the InMsg property in the routine parameters. Check the property value from the routine properties window. If the property is empty, fill a valid value, such as a connection point data reference to it. For instance, for sending the entire CP data structure to BizTalk, as is usually done, set this parameter to "{CP:/}".

Timeout when running BizTalk Two-Way Call

You try to use the "BizTalk Two-Way call" task to send a message to BizTalk and wait for a response. BizTalk does not respond within the specified timeout and the call is terminated.

There can be many reasons for the call to BizTalk not returning within the specified timeout. If the processing of the message simply takes long, you should adjust the timeout to a large enough value.

If the timeout is reasonably long (e.g. 60 seconds) but the process still times out, then it is likely there was some error when processing the message instance within BizTalk. Check the Event log, BizTalk Administration Console or Health and Activity Tracking tools for possible error messages. If the reason was a routing failure, see above for possible solutions. However, if the error was internal to the BizTalk process, please refer to the BizTalk documentation for troubleshooting guidance.

WF Tracking service throws an exception when a workflow has been changed

You have made a modification to a workflow used by a task or condition, and replaced the assembly with the new version. When executing the task or condition, an exception is thrown from the System.Workflow.Runtime.Tracking namespace (see the stack trace).

If you have made a modification to a workflow that has been used in FRENDS Iron, and tracked to the tracking database, you need to clear the database from previous instances of the same workflow instance. The easiest way of achieving this is to recreate the tracking database. For details on this, please see the .NET Framework 3.0 SDK documentation.

FRENDS Iron process engine does not start

FRENDS Iron process engine service does not start.

There are many reasons why the FRENDS Iron process engine service may refuse to start correctly. Common reasons are disabled FrendsAccService, database connection problems, invalid configuration file, or expired license. See the event log for more detailed information and contact support if the issue cannot be resolved.

If the service is disabled, please enable it from the Services applet by finding the Frends Acc Service and changing its Startup type to something else than Disabled from the service properties.

If the problem is with the database connection, please check that the database server is not disabled, is running and not being blocked by a firewall. The error message in the event log indicating a database connection problem will likely look like this:

Service cannot be started. System.ArgumentException: Unable to open database connection
Parameter name: connectionString ---> System.Data.SqlClient.SqlException: 
	A network-related or instance-specific error occurred while establishing a connection to SQL Server. 
	The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server 
	is configured to allow remote connections. 
	(provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)...

If the configuration file is invalid, please backup the old configurations mmc.exe.config and FRENDSAccService.exe.config and rerun the configuration tool FRENDS Iron Configuration.

With license issues please contact our support.

NOTE: Starting the FRENDS Iron process engine service may take over a minute, especially on hosts with little processing resources. This may cause the service applet or the "net start" commands to return a timeout error, even though the service is started.
Before diagnosing startup problems further, please make sure the service really could not be started, and the reason for that from the event log or service log files.
See the following section for a possible fix.

Error 1053: The service did not respond to the start or control request in a timely fashion

Sometimes the service startup takes an unusual long time. A workaround for this is to increase the timeout duration:

1. Open the registry editor: regedit.exe
2. Open registry location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control
3. Create or update the value for 'ServicesPipeTimeout' (DWORD) to '300000' (decimal) which is a five minute timeout.
4. Restart the computer

HTTP Endpoint could not be registered

If the FRENDS Iron Service is not running as the built-in Administrator, it can throw a System.ServiceModel.AddressAccessDeniedException, caused by the security settings of the HTTP.SYS component. This error basically means that the Windows Communication Foundation service hosted by FRENDS Iron Service could not open an HTTP port, because the account the service is running on does not have the necessary rights.

The FRENDS Iron installer should set these rights during the install; if the rights do not seem to be set correctly, you could either:

• Add the account running the service to the Administrators group.
• Explicitly give the necessary rights to the account, using HTTPCFG.EXE. For details on how to do this, see the documentation on TechNet or the blog post by Dominic Baier.

Database connection error

A common reason for startup failures is that the connection to the FRENDS Iron control database could not be established.

In the configuration file FrendsAccService.exe.config, in directory \Program Files\FrendsTechnology\FRENDS Iron, there are two places where the database connection information is given:

• Under the section /configuration/nhibernate/, the FRENDS database connection string is stored in the key hibernate.connection.connection_string

  <nhibernate> 
      … 
      <add key="hibernate.connection.connection_string" 
      value="Data Source=(local); Initial Catalog=FRENDS; Integrated 
      Security=True;"/> 
      … 
      </nhibernate> 
  

• The other needed connection details, to the tracking database and the BizTalk database, are stored under the section /configuration/connectionStrings

  <connectionStrings> 
      <add name="Frends.Acc.Service.Properties.Settings.BizTalkMgmtDbConnectionString" 
          connectionString="Server=(local);Initial Catalog=BizTalkMgmtDb; 
      Integrated Security=SSPI" providerName="System.Data.SqlClient" /> 
      <add name="Frends.Acc.Service.Properties.Settings.TrackingConnectionString" 
          connectionString="Server=(local);initial catalog=WorkflowStore; 
      Integrated Security=SSPI" providerName="System.Data.SqlClient" /> 
      </connectionStrings>
  

Make sure these entries point to the right database locations.

Other Configuration errors

Errors in configuration can cause a number of other issues, too. Check the logs for the actual errors and see all configuration entries are ok.

FRENDS Iron process engine crashes after starting

The FRENDS Iron process engine service starts up, but crashes after starting up

If the FRENDS Iron service has started up correctly, but crashes after a while, the most probable reason is a database connection error. Check the Event log and service log files for details, and the sections above for possible solutions.

Cannot connect to the FRENDS Iron web services

The FRENDS Iron web services do not answer to web service calls.

The web services are hosted in the FRENDS Iron process engine service. First, make sure the service is running. If it is, check that the web service addresses are correct: check the FrendsAccService.exe.config file in \Program Files\Frends Technology\FRENDS Iron, and look for following kind of entry:

<system.serviceModel> 
    <services> 
        <service Configuration="returnFaults" name="Frends.Acc.Service.AccSvc"> 
        … 
            <host> 
                <baseAddresses> 
                    <add baseAddress="http://localhost:9999/FRENDSAccService/ControlWSOneWay"/> 
                </baseAddresses> 
            </host> 
         </service> 
    <service behaviorConfiguration="returnFaults" name="Frends.Acc.Service.AccSvcCP"> 
    … 
    <host> 
        <baseAddresses> 
            <add baseAddress="http://localhost:9999/FRENDSAccService/CPWS" /> 
        </baseAddresses> 
    </host> 
  </service> 

If the address has been changed, use the correct address.

If you are not calling the service from the local machine, also make sure there are no firewalls blocking the connection and that you are using the correct machine name.

WorkflowStore database grows at an alarming rate

If you are using custom workflows with polling routines, your WorkflowStore probably is growing to many gigabytes in size. This is probably caused by a tracking profile that is enabled for the task. This profile will track all properties of all activities in the workflow. Therefore each execution of the workflow results in > 10 rows added to the WorkflowStore database tables (even more, if you have many properties or activities in the workflow).

If the database is growing too much because of this, it is recommended to disable the tracking profile for the workflow. To do this, simply uncheck the Enable tracking profile checkbox from the Task properties dialog and save the changes.

The other option would be to reduce the length of the tracking partition interval as described in http://msdn.microsoft.com/en-us/library/aa347868%28VS.85%29.aspx . The current default is monthly, but it could be changed to weekly or daily, if needed. Then the 'FRENDS Purge History' cleanup job will delete all instances later than this.

Warning messages in event log from AvailableTaskClassLoaderProxy

If the Iron service cannot load a DLL file to check for tasks for some reason, a warning is logged to the event log. There are a few DLLs included by default in the Iron installation which cause this warning. The Service can be instructed to ignore these files with the DllsIgnoredByAvailableTaskScan application setting, with each file separated with a semicolon.

If this setting is not defined, the following DLLs are ignored by default:

• Frends.Acc.SnapIn.dll
• Microsoft.Samples.BizTalk.Adapters.TransportProxyUtils.dll
• Microsoft.Samples.BizTalk.Adapters.TransportProxyUtilsMgmt.dll
• Ninject.Extensions.Conventions.dll

Example setting with the default values:

<applicationSettings>
    <Frends.Acc.Service.Properties.Settings>
    … 
        <setting name="DllsIgnoredByAvailableTaskScan" serializeAs="String">
            <value>
                Frends.Acc.SnapIn.dll;Microsoft.Samples.BizTalk.Adapters.TransportProxyUtils.dll;Microsoft.Samples.BizTalk.Adapters.TransportProxyUtilsMgmt.dll;Ninject.Extensions.Conventions.dll
            </value>
        </setting>
    </Frends.Acc.Service.Properties.Settings>
</applicationSettings>

FRENDS Iron Management

FRENDS architecture

Technically FRENDS Iron consists of two functional modules, the FRENDS Iron process engine and the FRENDS Iron Management Console, i.e., the user interface. In addition to these, there are several installed resources, like the databases and the assemblies containing ready-made workflows for FRENDS Iron tasks (see Ready-made WF elements for details).

FRENDS Iron process engine

The FRENDS Iron process engine is a windows service (FRENDS Acc Service) that handles the actual work in FRENDS Iron:

• Hosts the WF workflow runtime and services
• Launches new routine executions according to schedule
• Creates and starts routine task executions, and expands execution groups when launching routine tasks using connection point groups
• Provides a Web Service interface for process control commands

NOTE: If you are wondering what "Acc" in the service name means, FRENDS Iron was previously called FRENDS Acc. To preserve backwards compatibility, the name of the service or assemblies used was not changed.

The service provides a BizTalk adapter implementation that is registered to BizTalk Server as an isolated adapter

NOTE: For the isolated adapter to communicate with BizTalk, the service needs to be installed on the same host as BizTalk.

The user account and startup type of the service is defined during installation. By default, the service is set to start automatically, which is also the recommended option as the service needs to be running for the scheduling and triggering to work. The service user account is automatically granted access to the FRENDS Iron databases.

The configuration of the service is defined in the FRENDSAccService.exe.config file.

Controlling the Iron process engine service

You can start, stop, pause, resume and restart the FRENDS Iron process service from the Windows Services applet. You can also control it from the FRENDS Iron Management Console: Right-click on the FRENDS Iron node, and choose e.g. Start FRENDS Iron service to start the service.

Shutting down the FRENDS Iron process engine will cancel all task instances currently running on the server. The service will wait at most 10 seconds for the task instances to finish. After that, the process is terminated.

Pausing the FRENDS Iron process engine service pauses the scheduler, so no new executions are started. Already running executions will be allowed to finish normally. You can use the pause command e.g. to shut down the service in an orderly fashion: first pause the service and wait until all running executions have finished, and shut down the service after that.

NOTE: You can only control the service installed on the same computer.

FRENDS Iron management console

The FRENDS Iron Management Console is meant to be the tool for operators for managing FRENDS Iron operations. It allows for:

• Creating, viewing, updating, deleting and parameterizing all FRENDS Iron elements
• Manually starting and stopping(terminating) routines
• Monitoring the status of executed processes

The UI is a Microsoft Management Console 3.0 snap-in, which makes it possible to integrate it to the same console view with other Management Console snap-ins, such as the BizTalk Server Administration console (2006, 2006 R2 or 2009) or the event log viewer.

NOTE: Currently, as with BizTalk orchestrations, creating or editing workflow processes is meant to be handled by developers. This means the Management Console UI does not support editing entire workflow processes.

FRENDS Iron databases

FRENDS Iron stores all the information on its elements and their parameters to its own control database, named 'FRENDS'.

All routine execution data (execution times, parameters, results) is stored in a single database table in the FRENDS control database (ROUTINE_INSTANCE). This makes generating reports from the data very easy, e.g. by utilizing SQL Server Reporting Services. A sample of this is provided in Tutorial 2.

As FRENDS Iron also utilizes the Windows Workflow Foundation SqlTrackingService and SqlWorkflowPersistenceService, a separate database instance for these services is also installed by default to the same database server as the FRENDS Iron control database.

Data maintenance

When running FRENDS Iron, routine history data (parameters, start and stop times, results), as well as workflow tracking data is stored to the database. Eventually, this data will fill up the database, unless you clean it up periodically.

For cleaning up the data, parameterized stored procedures are installed with the databases (corresponding scheduled jobs in parentheses):

• BackupFullDB (FRENDS Full Backup) for backing up the FRENDS database.
• BackupLog (FRENDS Transaction Log Backup) for backing up the FRENDS database transaction log.
• PurgeRoutineHistory (FRENDS Purge History) for removing routine history and dropping the inactive workflow tracking data partitions.

The purge history job by default removes:

• Successful routine instances older than 7 days
• Failed routine instances older than 30 days
• Workflow tracking data older than the latest complete month

You can also define custom history preservation time for individual routines using the Use custom cleanup settings for routine setting for a routine. If it is set, the purge history job will delete all routine instances older than the given number of days, no matter what the end state of the routine instance is.

Changing the custom cleanup settings will affect only future routine instances. Existing routine instances will be cleaned up based on the configuration during the time of routine instance's execution.

Setting the history preservation value to 0 means that all routine instances will be cleaned up from the database every time the FRENDS Purge History job is run. The job is run once per hour by default.

Location for database backup files are provided during FRENDS Iron configuration sequence and database jobs are enabled by default. If jobs are not enabled automatically for some reason or you need to disable or adjust them, see Readme.

For more details on cleaning up old WF tracking data, see the .NET Framework 3.0 documentation

Providing high availability for FRENDS Iron host

You can increase the availability of an FRENDS Iron system in two major ways: Firstly, you can create an SQL Server cluster. Secondly, you can run FRENDS Iron services concurrently on multiple hosts, i.e. in a server farm.

SQL Server Clustering

All FRENDS Iron schedule and task information is stored in the FRENDS database on an SQL Server. Also all task workflow tracking and persistence data is stored in the WorkflowStore database. This means that the SQL Server availability is critical for FRENDS Iron to function.

The recommended way to increase the availability of the SQL Server is to create a server cluster. FRENDS Iron will resume operations after a cluster failover: The scheduling loop as well as the persistence and tracking services cannot work while the connection is down, but will retry automatically, and continue after the connection to the database is resumed (more on MSDN ).

Farming

For additional redundancy and better performance, FRENDS Iron can be installed on many servers, all using the same control database. This allows scheduled tasks to run when one of the servers is down as well as distributing task executions to multiple hosts.

You can also limit routines to execute only on specified hosts. For more details, see Running routines on multiple hosts on a farm

For details on how to install FRENDS Iron on multiple hosts, see the installation documentation in Readme

You also need to enable locks on the workflow persistence store, so the workflows are executed on a single host at a time. To do this, you need to define a value to the WFPersistenceInstanceOwnershipDuration setting in the FRENDS Iron service configuration file, FRENDSAccService.exe.config - see the section Configuration files for more details. The persistence instance ownership duration essentially defines how long a workflow runtime will maintain a lock on the persisted and loaded workflow so no other workflow runtimes will try to run it. The period must be longer than the longest time any persisted workflow in the system will run. Otherwise, there will be errors when the other workflow runtime in the farm may try to resume a workflow already executing, or when the workflow runtime tries to persist an instance that has been running for longer than the timeout, as the lock has expired.

There are some things you need to consider when running FRENDS Iron on multiple hosts:

• The load between the different servers is spread randomly: all services are polling for the common database, and the first one to notice a task instance that is ready to launch, starts executing it. This means:
  • The load may be uneven for the different servers at any point in time.
  • All hosts should have identical deployments, i.e. have the same versions of assemblies and have the same parameters for routines etc. You cannot be sure which host will execute the individual task instance - unless the routine is explicitly configured to run only on one host, in which case all task instances are executed on the same host.
    Note: This means that also all references to files must be accessible from every node, e.g. a file used by multiple tasks in a routine must be stored in a file share accessible from each node, not in the local file system.
• Task instances will be started in batches. A single batch will run on one FRENDS Iron host. The content of the batches is random, i.e. there may be task instances from different routines / routine instances.
• Once a task instance has been started on one host, it will execute on that host as long as the workflow runtime has ownership on it (see this blog post for more). If the server goes down while executing the task, the other servers will not be able to resume the execution of that task, until the ownership duration has ended.
• If you want to use the BizTalk integration features of FRENDS Iron, like the ready-made BizTalk tasks or the party integration, BizTalk needs to be installed on the same host. This is because the features require BizTalk assemblies to work.
  If you don't want to install FRENDS Iron on the same host as BizTalk, you can use e.g. custom WCF tasks for communication: expose a WCF endpoint on BizTalk and create a custom workflow and task that uses the WCF send activity.
• The FRENDS Iron management console can only be configured to manage a single FRENDS Iron service instance (the local instance by default). This means:
  • You cannot terminate routines running on other service instances.
  • You cannot restart the FRENDS Iron service instances on other hosts.
  • You can only start routines on the managed service instance

Configuring FRENDS Iron

Configuration files

Most of the operational FRENDS Iron configuration is defined in XML configuration files:

• FRENDSAccService.exe.config for the FRENDS Iron service
• mmc.exe.config for the user interface

These files contain the database connection strings, WCF endpoint configuration, log level configuration and other settings. It is recommended to use the configuration tool for changing most of the settings. The functionality of the configuration tool is described in the next section.

However, some settings can currently only be changed by directly modifying the values in the files. The defaults are usually good enough, but you may want to update the settings by hand in some cases, e.g. when installing FRENDS Iron in a high availability environment on a server farm. These settings are defined in the <applicationSettings> section of the configuration file.

For the FRENDS Iron service, the most important settings change how the workflow engine works (in FRENDSAccService.exe.config):

• WFMaxSimultaneousWorkflows - The maximum number of workflows executed at one time by the workflow runtime. If the value is defined, it is given directly as a parameter to the DefaultWorkflowSchedulerService constructor , where it sets the MaxSimultaneousWorkflows setting. If the value is not given, the default constructor is used, which will mean that roughly 4 workflows at a time per processor core will be executed.
• WFMaxTotalRunningInstances - The maximum number of task instances that can be in the running state simultaneously, by default 100. This setting limits the queue, reducing the possibility that a huge queue of instances is created to memory. Routine instances may be started and left to wait for execution as soon as there are free slots. In a FRENDS Iron farm, this setting is node-specific. If increased, change also TaskInstanceCheckBatchSize which should to be ten times bigger than WFMaxTotalRunningInstances so all possible task instances to process can be fetched from the database.
• WFMaxRunningInstancesPerRoutine - The maximum number of task instances under a single routine that can be in the running state simultaneously, by default 50. As the WFMaxTotalRunningInstances, this setting is also meant for limiting excessive load on the workflow runtime. In a FRENDS Iron farm, this setting is farm-wide.
• WFPersistenceInstanceOwnershipDuration - The length of time that the workflow runtime maintains locks on persisted workflows. The period must be longer than the longest time any persisted workflow in the system will run.

  You should only define this value if you are running multiple FRENDS Iron services using the same database, i.e. in a server farm situation. See the section Farming for more on this.
  

  If the value is defined, it is given directly as a parameter to the SqlWorkflowPersistenceService constructor . If no value is given, the default constructor is used, which will not set a lock on idle workflows.

• WFPersistenceLoadingInterval - The frequency at which the persistence service will poll the database for workflows with expired timers. The period essentially defines the shortest time a workflow can be idle, e.g. waiting for a Delay activity to trigger.

  If the value is defined, it is given directly as a parameter to the SqlWorkflowPersistenceService constructor . If no value is given, the default constructor is used, which will use a 2 minute delay between database polls.

• WFBatchSize - (Deprecated) The amount of routine instances that are processed in a single batch. By default 50.
  In FRENDS Iron 3.0, the processing is done at the task instance level, and this limit is set by the TaskInstanceBatchSize setting.
• TaskInstanceCheckBatchSize - The maximum number of task instances processed at a time. By default 1000.
  If you have many routines scheduled to run at the same time that also use large CP groups, you may want to increase this limit: if there are over 1000 waiting or launched task instances at a time, only the first 1000 of them will be processed. This may leave some task instances waiting long to be launched or started. However, increasing this limit too high may cause the scheduler loop to take long to execute. This value should be ten times bigger than WFMaxTotalRunningInstances so all possible task instances to process can be fetched from the database.
• PollDelay - The delay between processing loops, in milliseconds, by default 1000 ms. Note that there only a single batch can be processed at once, so making the loop shorter may not improve performance.
• LicensePath - The path to the folder where the license file, FrendsAcc.lic is located.
• LicenseWarningPeriod - The time (in days) a warning is written to the event log before the license ends.
• ScheduleFullCheckIntervalSeconds - The interval for doing schedule checks for all schedules, in seconds. By default 60, i.e. one minute.
  Once the interval has passed, the schedule of each active routine will be checked. This makes the possible schedule changes to take effect.
  This limit also limits the minimum polling interval for polling schedules: if you have a schedule that should execute every 10 seconds, you should also set the ScheduleFullCheckIntervalSeconds to less than or equal 10 seconds. Also note that if you have many routines, reducing the interval too low may cause the schedule loop take long to execute.
• CheckTaskInstancesForZombiesRunningLongerThan - FRENDS Iron service will periodically check whether there are task instances running whose workflow or task is no longer running and mark those executions as zombies. This time limit (in minutes) controls the time for how long the task instance with no corresponding workflow can run before marking it as zombie.

  By default, the value is 30 minutes. If set to 0, health check is disabled. The terminator will also check all executions right after the service has started.

  Note: From 3.3, the health check will only check those instances that are supposedly running on the same node. This means that if one node is stopped, the instances on that node will not be marked as zombies until the node comes online.

For the user interface (in mmc.exe.config), the settings are mostly related to the license handling or the BizTalk integration:

• LicensePath - The path to the folder where the license file, FrendsAcc.lic is located.
• LicenseWarningPeriod - The time (in days) a warning is written to the event log before the license ends.
• EnableBizTalkIntegration - If true, the features related to BizTalk, like party support are enabled.
• HATQueryTemplateFile - the file used for running custom queries on HAT for a specific routine. By default ServiceQueryTemplate.trq.
• FrontPageUri - the URI of the page to show in the front page.
• IsFrontPageUriReportingServerUri - If true, the query parameters needed for showing a SQL Server reporting server report in the front page are added to the URI before showing the page.

Configuration tool

You configure FRENDS Iron with the FRENDS Iron Configuration wizard tool. You need to run it to initialize the configuration before you can use FRENDS Iron.

The tool:

• Installs the FRENDS Iron process service
• Registers the FRENDS Iron Management Console
• Installs the FRENDS Iron control and tracking databases
• Configures web service endpoint settings.
• Configures automatic error reporting settings.
• Exports and imports configuration settings.

For more details on all these actions, see the FRENDS Iron Configuration Tool Documentation. The next sections give a quick view on how to configure FRENDS Iron for the first time as well as how to configure logging

First time configuration

For the basic configuration with the default settings, you can just go through the dialogs, giving the basic information:

• The license file - You need to obtain a license to run FRENDS Iron.
  • You can get an evaluation license from the Frends web site at http://www.frends.com/. There you can also find the contacts for purchasing a production license.
• The user account and password to use for running the service – This account is used for connecting to the database and BizTalk. In order to send messages from FRENDS Iron to BizTalk, the user account needs to belong to the BizTalk Isolated Host Users group.
• The name of the FRENDS and BizTalk database servers – The FRENDS databases will be created or updated on the server given. The BizTalk database will be used for registering the adapter. 
  • If you are using remove database servers, you need to take into account some special considerations. See section Installing on a remote database server in Readme.htm for details.  

See the configuration tool documentation for details on all the options available.

The actual configuration settings are stored in text files in the FRENDS Iron installation directory:

• mmc.exe.config for the user interface
• FRENDSAccService.exe.config for the process engine

It is recommended that you do not edit these files yourself.

Logs

FRENDS Iron writes log entries to a file during its operation, and these files can be used to check for possible problems or issues. The log files are written to the /Logs subdirectory of the installation directory.

The default log files are:

• FRENDSAccService.log - Information messages from the process engine service
• FRENDSAccManagementConsole.log - Information messages from the FRENDS Iron Management Console

FRENDS Iron internally uses the log4net logging library. You can control the level of logging and where the log messages will be written to in the management console:

1. In the FRENDS Iron Management Console, right-click on the FRENDS node and choose Configure logging in the menu.
   
2. You can change the logging levels for the FRENDS Iron internal logging ("FRENDS messages") and database diagnostic logging ("NHibernate messages"). The defaults are usually fine, but if you want to diagnose errors, it may help to e.g. set FRENDS Iron messages level to "DEBUG".
3. By checking the Log everything to trace stream checkbox all messages are written to the system trace stream in real time. This allows you to see what is happening at runtime using a tool like DebugView
4. The Log file settings allow you to define a folder where the log files will be written to.
   NOTE: the FRENDS Iron service user needs to have write rights to this directory.
5. Click OK to apply the changes. The changes will be written to the configuration files and taken into use immediately - you don't need to restart the service or UI.

Batch element import with deployment tool

FRENDS Iron comes with a command line tool that can be used for importing elements from multiple files at once, or for scripted import as well. Frends.Acc.DeploymentTool.exe can be found from the FRENDS Iron installation directory. The tool has three parameters that are all required. Parameters are case-insensitive.

• --importSource File or directory from which to import routine elements.
• --duplicateAction Action to perform if elements with same name already exists in data store. Allowed options are:
  • overwrite Existing elements are overwritten with the data of the imported elements.
  • addWithNewName Elements are added with a new copy number.
  • useExisting Existing elements in data store are used, no new elements are imported when duplicate elements are found.
  • error When duplicate is found the import process is aborted.
• --activeRoutineAction Action to perform if routines with active schedules are found within an import file. Allowed options are:
  • keepActive Routines are imported as active. Depending on the routine's schedules, importing routine as active may trigger routine execution immediately after import process.
  • deactivate Routine's schedules are deactivated upon import. Routine's schedules have to activated manually.
  • error When a routine with active schedules is found the import process is aborted.
• --connectionString Connection string to data store to which import the elements into.

NOTE: If no user account information is given in connection string, make sure that user that runs the Frends.Acc.DeploymentTool.exe has necessary rights to database server and its databases.

Command line help for tool can be acquired by typing Frends.Acc.DeploymentTool.exe --help.

Example usage:

Frends.Acc.DeploymentTool.exe --importSource "D:\Import\ConnectionPoints.xml" --duplicateAction addWithNewName --activeRoutineAction keepActive --connectionString "Data Source=(local);Initial Catalog=FRENDS;Integrated Security=True"

The example:

• Imports FRENDS Iron routine elements from a single file named ConnectionPoints.xml.
• If elements with same name as the imported elements already exists, imported elements get a new copy number.
• Elements are imported to locally configured datastore and into FRENDS database.