Scripting Methods

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Overview

Describes the types of methods and features available for use during export script development.

Details

You have the entire array of C# .NET at your disposal. The only caveat is that the runtime complier object for .NET Framework 4.5.2 does not yet support C# 6.0 features.

Your C# script will consist of a typical class code file with .cs extension which includes:

A list of one or more using statements (must include using IWU.ExportManager;)
Optionally, additional DLL reference(s)
The IWU.ExportManager.Exports namespace declaration
Export class declaration which must inherit Export or one of its subtypes (the only subtype currently is FtpExport)
- Class constructor which takes string[] args and calls base( args )
  - Options and setup should be set in the constructor.
- Override of the void Run() method
  - Method calls and actions should be in the Run() method.
- Optional override of the void Transform() method to manipulate data before writing to disk.
- Additional methods and properties you create for your own purposes.

Typical Script Flow

Constructor
- Set up configuration and options
Run method
- Copy files in to working set from external sources
- Query SQL Server and write results to disk
- Export files to another network server
- Upload files to a secure FTP server
- Archive / clean up files

Basic Example:

Script

using IWU.ExportManager;
using System.IO;

namespace IWU.ExportManager.Exports
{
    public sealed class BasicSample : Export
    {
        public BasicSample( string[] args ) : base( args )
        {
            // Try to keep setup & options in the constructor
            DbConnection.DataSource         = "ERPdb";
            DbConnection.InitialCatalog     = ColleagueConnection.DatabaseName;
            DbConnection.IntegratedSecurity = true;
            SqlScript = "use coll18_live; SELECT TOP 100 * FROM dbo.PERSON";
            AddOutputFileName( "data.csv" );
        }

        public override void Run()
        {
            // Try to keep this method limited to other method calls rather than setup & options
            LoadFromDB();
            Transform();
            WriteToDisk();
            ExportFilesTo( @"N:\" );
            ArchiveFiles( daysToKeep: 30, filesToKeep: 0 );
        }

        public override void Transform()
        {
            // Optional, do something with ExportData.Tables
        }
    }
}

Log output

Note that much more detail appears in the logs for the Dev and Test instances of ExportManager. The examples shown below include this extra detail.

Expected log output in ExportManager.log:

2016-09-13 11:58:31.0595 INFO -------------------------------------------------------------------------------
2016-09-13 11:58:31.1125 INFO IWU Export Manager launched
2016-09-13 11:58:31.1275 INFO -------------------------------------------------------------------------------
2016-09-13 11:58:31.1385 TRACE OS=Microsoft Windows NT 6.1.7601 Service Pack 1  Cores=4  64bit=True
    Machine=IWU71563  Domain=IWUNET  User=konrad.willmert
    Command=ExportManager  BasicSample
-------------------------------------------------------------------------------
2016-09-13 11:58:31.1705 DEBUG Export "BasicSample" found
2016-09-13 11:58:31.1895 DEBUG Export "ColleagueTest" found
2016-09-13 11:58:31.2035 DEBUG Export "SampleExport" found
2016-09-13 11:58:32.6045 INFO Executing "BasicSample"
2016-09-13 11:58:35.0795 DEBUG "BasicSample" export process executed in 3,840ms
2016-09-13 11:58:35.0795 DEBUG IWU.ExportManager.Exports.BasicSample with data connection: "Data Source=ERPdb;Initial Catalog=coll18_dev;Integrated Security=True;Application Name="ExportManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null"" FileManager@C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\data\BasicSample: 1 data files, 0 other files ColleagueConnection mocha.task@DEV with database "coll18_dev" and HOLD dir "\\ERPapp\d$\datatel\ftp\coll18_dev\data\_HOLD_"
2016-09-13 11:58:35.0965 DEBUG ExportManager total run time: 4,038ms
2016-09-13 11:58:35.0965 INFO Exiting.

Expected log output in BasicSample.log:

2016-09-13 11:58:32.6415 INFO -------------------------------------------------------------------------------
2016-09-13 11:58:32.6415 INFO IWU.ExportManager.Exports.BasicSample initializing.
2016-09-13 11:58:32.6645 INFO -------------------------------------------------------------------------------
2016-09-13 11:58:33.4485 WARN TEST MODE is enabled. No actual data files will be exported/uploaded. Instead, write access will be tested. Override with "notest".
2016-09-13 11:58:33.4705 DEBUG Data directory: C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\data\BasicSample
2016-09-13 11:58:33.4705 DEBUG Archive directory: C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\archive\BasicSample
2016-09-13 11:58:33.4855 TRACE Added output file: data.csv
2016-09-13 11:58:33.4995 INFO Executing SQL script.
2016-09-13 11:58:33.4995 TRACE Connecting to database with connection string: Data Source=ERPdb;Initial Catalog=coll18_dev;Integrated Security=True;Application Name="ExportManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null"
2016-09-13 11:58:34.7875 INFO Received 100 total rows in 1 result set(s)
2016-09-13 11:58:34.7955 DEBUG FileManager@C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\data\BasicSample: 1 data files, 0 other files, SQL data loaded
2016-09-13 11:58:34.7955 DEBUG SQL script executed in 1,288ms
2016-09-13 11:58:34.8135 INFO Writing SQL results to disk.
2016-09-13 11:58:34.8135 DEBUG Number of output files matches number of SQL result sets received.
2016-09-13 11:58:34.8505 DEBUG Successfully wrote 36,864 bytes from result set 0 to "data.csv" in 25ms
2016-09-13 11:58:34.8575 INFO Successfully wrote result set to disk.
2016-09-13 11:58:34.8575 DEBUG FileManager@C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\data\BasicSample: 1 data files, 0 other files, SQL data loaded, SQL data written to disk
2016-09-13 11:58:34.8745 INFO Exporting working set to: N:\
2016-09-13 11:58:34.8745 DEBUG Created directory: N:\
2016-09-13 11:58:34.9055 DEBUG Test file copied successfully.
2016-09-13 11:58:34.9465 INFO Archiving working set.
2016-09-13 11:58:35.0525 DEBUG Archive created: C:\Users\konrad.willmert\Source\Repos\IWUExportManager\IWUExportManager\bin\Release\archive\BasicSample\BasicSample 2016-09-13 11.58.34.zip
2016-09-13 11:58:35.0635 DEBUG Working set deleted.
2016-09-13 11:58:35.0635 INFO Expiring old archives.

Considerations

Many classes and methods have been provided for your benefit when developing export scripts:

For a full example of most of these features, see the ExportSample.cs file located in the exports directory under the ExportManager application instance.