You want to fetch a file with HTTP from a specific URL or POST files to a web page.
This example has been designed to work on an environment which has IIS 7 set up. A few tests have been written for the HttpEndPoint. The example solution location can be found here.
The unit tests for the end point can be found in the Frends.Cobalt.Extensions.Tests project's HttpEndPointTests.cs file and the integration tests are located in IntegrationTests/HttpEndPointWorkflowIntegrationTest.cs file. The integration tests test the EndPoint with the Cobalt workflow. The requirements for the test environment can be found from the beginning of the test files.
public interface IEndPointDetails
{
/// <summary>
/// The name of the file (or file mask) that this endpoint is configured to use. Meant for reporting purposes.
/// </summary>
string FileName { get; }
/// <summary>
/// The name of the directory this endpoint is configured to use. Meant for reporting purposes.
/// </summary>
string Dir { get; }
/// <summary>
/// The possible server address this endpoint is configured to use. Meant for reporting purposes.
/// </summary>
string Address { get; }
/// <summary>
/// The string representation of the type of transfer for the endpoint. Meant for reporting purposes.
/// </summary>
string TransferType { get; }
}
///<summary>
/// The common interface for all Cobalt endpoints.
/// This interface defines the methods needed for both source and destination endpoints
///</summary>
public abstract class EndPointBase : IEndPointDetails, IDisposable
{
/// <summary>
/// Returns the name or description of EndPoint
/// </summary>
/// <returns></returns>
public virtual string GetEndPointName();
///<summary>
/// Deletes the given remote file.
///</summary>
///<param name="remoteFile">The name of the remote file to remove.
/// If this is a relative path ("foo/file.txt") or just a file name ("file.txt"), it will be based on the current directory.
/// If you give an absolute path (that has a root, i.e. '/' or e.g. 'C:\'), it will be used as is
/// </param>
public abstract void Delete(string remoteFile);
///<summary>
/// Renames the remote file.
/// <remarks>This method is called when:
/// <list type="bullet">
/// <item>Starting the transfer of a file: renames the source file with a unique name in order to lock the file during transfer</item>
/// <item>Executing the source file operation: Nothing renames the source temp file back to the original file name, Rename renames the source temp file to the given name</item>
/// <item>Rolling back the transfer (due to an error): The temporary source file is renamed back to the original file name to release it.</item>
/// <item>Ending the transfer: renames the temporary destination file (again, done to lock the file during transfer) to the final file name</item>
/// <item></item>
/// </list>
/// </remarks>
///</summary>
///<param name="remoteFile">
/// The name of the remote file to rename.
/// If this is a relative path ("foo/file.txt") or just a file name ("file.txt"), it will be based on the current directory.
/// If you give an absolute path (that has a root, i.e. '/' or e.g. 'C:\'), it will be used as is.
/// </param>
///<param name="toFile">
/// The new name of the file.
/// As with <see cref="remoteFile"/>, this can be either a relative or absolute path.
/// </param>
///<returns>The full path to the renamed file (so it can be used in when executing the SourceOperation, Get etc.) </returns>
public abstract string Rename(string remoteFile, string toFile);
/// <summary>
/// Moves the file to another directory
/// <remarks>
/// Used when executing the source file operation: Move moves the file to the given directory
/// </remarks>
/// </summary>
/// <param name="remoteFile">Source file name</param>
/// <param name="toPath">Full path, with filename, where to move source file after transfer</param>
public abstract string Move(string remoteFile, string toPath);
/// <summary>
/// Opens the endpoint connection.
/// This method must be called before using the endpoint, and
/// after this method has been called, the user should be able to call other methods of the endpoint
/// This can mean different things depending on the endpoint type, e.g.
/// opening a connection, logging in and setting the default directory
/// </summary>
public abstract void Open();
/// <summary>
/// Closes the endpoint connection.
/// When this method has been called, all internal connections should be disconnected and resources released.
/// </summary>
public abstract void Close();
/// <summary>
/// Gets a file from the endpoint.
/// Called for each file returned by <see cref="ListFiles"/>
/// </summary>
/// <param name="remoteFile">
/// The name of the file to fetch from the source.
/// Relative path ("foo/file.txt") or just a file name ("file.txt") will be based on the current directory.
/// Absolute path (that has a root, i.e. '/' or e.g. 'C:\') will be used as is.
/// </param>
/// <param name="localFilePath">The full path to the local temporary file to store the file to</param>
public abstract void Get(string remoteFile, string localFilePath);
/// <summary>
/// Creates and uploads a file to the endpoint
/// </summary>
/// <param name="sourceFile">
/// The full path to the local temporary file to upload (the result of possible transforms etc.)
/// </param>
/// <param name="remoteFile">
/// The name of the file to create on the remote endpoint.
/// Relative path ("foo/file.txt") or just a file name ("file.txt") will be based on the current directory.
/// Absolute path (that has a root, i.e. '/' or e.g. 'C:\') will be used as is.
/// </param>
public abstract void Put(string sourceFile, string remoteFile);
/// <summary>
/// Appends the content of the local source file to the destination file
/// </summary>
/// <param name="sourceFile">The full path to the local temporary file</param>
/// <param name="remoteFile">The name of the file to append to on the remote endpoint.
/// Relative path ("foo/file.txt") or just a file name ("file.txt") will be based on the current directory.
/// Absolute path (that has a root, i.e. '/' or e.g. 'C:\') will be used as is.
/// </param>
public abstract void Append(string sourceFile, string remoteFile);
/// <summary>
/// Gets the list of files from the source endpoint that match the <see cref="FileName"/> mask.
/// The method skips all entries for the routine
/// </summary>
/// <returns>The list of file details</returns>
public abstract IList<FileItem> ListFiles();
/// <summary>
/// Checks if the given file exists on the remote endpoint
/// </summary>
/// <param name="remoteFilePath">Full path to the file to check</param>
/// <returns>True if the file exists</returns>
public abstract bool FileExists(string remoteFilePath);
}
For this HTTP example, the most important methods to implement are:
Put functionality also needs help to function and we have provided a simple
example ASP .NET-page which can receive a file through a POST-request. This page can be found in the Frends.Cobalt.Extensions project.
WebClient in our own wrapper WebClientWrapper which implements the interface IWebClientWrapper to make it mockable.
[assembly: InternalsVisibleTo("Frends.Cobalt.Extensions.Tests")]
namespace Frends.Cobalt.Extensions
{
public class WebClientWrapper : IWebClientWrapper
{
private WebClient _client;
public WebClientWrapper(WebClient client)
{
_client = client;
}
public void Dispose()
{
if(_client != null)
{
_client.Dispose();
_client = null;
}
}
public ICredentials Credentials
{
get { return _client.Credentials; }
set { _client.Credentials = value; }
}
public byte[] DownloadData(Uri uri)
{
return _client.DownloadData(uri);
}
public byte[] UploadFile(Uri uri, string method, string file)
{
return _client.UploadFile(uri, method, file);
}
}
public interface IWebClientWrapper
{
void Dispose();
ICredentials Credentials { get; set; }
byte[] DownloadData(Uri uri);
byte[] UploadFile(Uri uri, string method, string file);
}
IDisposable we have to write a Dispose() method which frees the resources reserved by the class. This means we should dispose of all disposable classes we have initialized. In our case this means the WebClient.
public override void Dispose()
{
Dispose(true);
}
private void Dispose(bool disposing)
{
if (!disposing)
return;
if (_webClient != null)
{
_webClient.Dispose();
_webClient = null;
}
}
EndPointBase class, HTTP endpoint has some other data stored in private fields as well. We get all of these from the TransferEndPointConfig, which is initialized from the config-xml, except for the WebClient.
private readonly string _username;
private readonly string _password;
private readonly int _port;
private IWebClientWrapper _webClient;
WebClientWrapper and another one to implement the EndPointBase, which initializes a new WebClient. TransferEndPointConfig contains the basic connection information for the enpoint. If there were some extra parameters that we needed, we would parse them from the parameters string in the constructor, which is defined in the xml-configuration.
internal HttpEndPoint(TransferEndPointConfig endPointConfig, IWebClientWrapper webClient) : base(endPointConfig)
{
_webClient = webClient;
TransferType = "Http";
_port = endPointConfig.ServerPort <= 0 ? 80 : endPointConfig.ServerPort;
_username = endPointConfig.ServerUsername ?? "";
_password = endPointConfig.ServerPassword ?? "";
//If password or username defined, add them to the request
SetPossibleCredentials();
}
public HttpEndPoint(TransferEndPointConfig endPointConfig) : this(endPointConfig, new WebClientWrapper(new WebClient())) { }
public string GetEndPointName()
{
return GetUrl(FileName).ToString();
}
Source operation: Delete and Destination File Exists operation: Overwrite. The delete operation is not possible with the HTTP-endpoint, so an exception is thrown if it is called.
public void Delete(string remoteFile)
{
//deleting is not possible
throw new NotSupportedException("Deletion is not possible with HTTP Endpoint");
}
Paramaters section with the RenameSourceFileBeforeTransfer and RenameDestinationFileDuringTransfer -settings. Rename is not supported by HTTP, so an exception is thrown if it is called.
public string Rename(string remoteFile, string toFile)
{
//rename is not possible
throw new NotSupportedException("Renaming is not possible with HTTP Endpoint");
}
Source Operation: Move to move the file that was transferred to another directory. This is not supported by the HTTP -endpoint so an exception is thrown when called.
public string Move(string remoteFile, string toPath)
{
//Move is not possible
throw new NotSupportedException("Moving files is not possible with HTTP Endpoint");
}
public void Open()
{
//Transfers are done in their own connections
}
public void Close()
{
//Transfers are done in their own connections
}
WebClient and with the GetEndPointName-method.
internal Uri GetUrl(string remoteFile)
{
UriBuilder builder = new UriBuilder();
builder.Host = Address;
builder.Port = _port;
builder.Scheme = "http";
builder.Path = Path.Combine(Dir, remoteFile);
return builder.Uri;
}
WebClient class to download the data from the URI.
public void Get(string remoteFile, string localFilePath)
{
Uri uri = GetUrl(remoteFile);
var fileBytes = _webClient.DownloadData(uri);
File.WriteAllBytes(localFilePath, fileBytes);
}
WebClient if either username or password is provided.
private void SetPossibleCredentials()
{
if (!String.IsNullOrEmpty(_username) || !String.IsNullOrEmpty(_password))
_webClient.Credentials = new NetworkCredential(_username, _password);
}
Put method contains the code used for sending the files to destination endpoint. The HTTP -endpoint simply uses the .NET WebClient class to POST the file the URI. The response is discarded, but it could be used to detect errors.
public void Put(string sourceFile, string remoteFile)
{
Uri uri = GetUrl(remoteFile);
var response = _webClient.UploadFile(uri, "POST", sourceFile); //TODO: Handle response, errors such as 404 are thrown out by the _webClient as exceptions
}
Destination Exists operation: Append is selected and the destination file exists(FileExists returns true). The method should contain the code to append the content of the sourcefile to the destination file. The sourcefile is the path to the temporary local file and remoteFile is the path to the file on the remote machine. The HTTP -endpoint does not support append operations so it throws an exception if called.
public void Append(string sourceFile, string remoteFile)
{
throw new NotSupportedException("Append is not supported by http");
}
Filemask and only return an IList of FileItems which will be transferred. The HTTP -endpoint as a source endpoint supports only the retrieval of a single file, so it returns the filename that was provided in the config.
public IList<FileItem> ListFiles()
{
//HttpEndpoint can only transfer single files so we return the original FileName,
//which normally is used for reporting only.
//This example only supports getting a single file because implementing a directory
//listing parser would be too tedious for the scope of this example.
//TODO: Check if the FileName is a filemask and throw an exception if this is the case
return new List<FileItem>() { new HttpFileItem(FileName) };
}
FileExists is called when transferring files to the destination endpoint before each transfer. If it returns true the workflow acts accordingly to the DestinationFileExists Operation setting in the config. The HTTP -endpoint always returns false because the destination page should always exist because it handles receiving the file data.
public bool FileExists(string remoteFilePath)
{
//Destination filename is the page which is receiving the data with the POST
//This method is used when checking if the remote file exists so we won't overwrite it
//but remoteFilePath is a page which should always exist, and we cannot overwrite it
//therefore we always return false so the Cobalt workflow will function correctly
return false;
}
HttpFileItem class that inherits the FileItem class. We don't use the base class because most of the data defined is unavailable for us with the HTTP -endpoint. This is why we instantiate the FileItem object with some default values.
public class HttpFileItem : FileItem
{
public HttpFileItem(string fileName)
{
//Using static values because we cannot get the file information without getting the whole file.
this.Modified = DateTime.MaxValue;
this.Name = fileName;
this.Size = -1;
}
}
}
In order to use the code with Cobalt you first need to compile it to an assembly and deploy the assembly somewhere FRENDS Iron can find the code.
This means either the \Program Files\Frends Technology\FRENDS Iron -directory or the GAC
Once this is done, you need to configure Cobalt to use the new custom endpoint. To do this, just create a new Cobalt connection point as described in the manual and:
Source endpoint type to "Custom"index.htm, text.txt, order.xml, but not *.txtDestination endpoint type to "Custom"Directory to the location of the page that will receive the filesCustom elements under Source and Destination:
AssemblyName to the name of the assembly containing the custom class, e.g.
Frends.Cobalt.Extensions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null and ClassName to the name of the endpoint class, e.g. Frends.Cobalt.Extensions.HttpEndPointParameters empty, as our custom endpoint does not take any parameters.RenameSourceFileBeforeTransfer to false, because our source end point is a HttpEndPoint, which does not support renaming.RenameDestinationFileDuringTransfer to false, because our destination end point is a HttpEndPoint, which does not support renaming.