Importing XML files on-demand using the Plugin Scheduler

Samples and Examples of using the Jiwa 7 framework - including plugins, stand-alone applications and scripts.

Importing XML files on-demand using the Plugin Scheduler

Postby Mike.Sheen » Wed May 21, 2014 5:32 pm

FileWatcher.rar
Plugin XML for FileWatcher sample
(3.48 KiB) Downloaded 478 times

One of the utilities we provided in version 6 and included in version 7 also is the Jiwa File Watcher services.

This windows service allowed any executable to be invoked when a file appeared in a nominated folder. When files did appear, the Jiwa File Watcher Service invoked the configured executable with the filename as a command-line argument.

This typically was used together with the Jiwa XML Import or Jiwa CSV Import to import data from an external source, on-demand (ie: not polled) - this was useful for importing things like sales orders from external web sites.

It is, however a little clunky and there isn't that much flexibility or control over what you can do.

In Version 7 we have a much more powerful way to monitor folders and import files - the Jiwa 7 Plugin Scheduler server can be used to perform actions based on a schedule - but you can also hook into the file system to be notified of a file system change when the service starts. You don't even need to define a schedule - just add the appropriate handlers in the OnServiceStart method of the ScheduledExecutionPlugin class.

The code below is a sample of how to do this - it will monitor a folder named C:\Import\Watch for any files with an .XML extension. When one does appear, it will create an instance of the Jiwa Import Queue manager and pass that XML to be imported. Any failures are moved into the C:\Import\Failed folder and an error entry placed in the event log. Successful imports are moved to the C:\Import\Succeeded folder.

The XML for the plugin is also attached to this post - but I've posted the code it contains below anyway for convenience.

Code: Select all
Public Class ScheduledExecutionPlugin
    Inherits System.MarshalByRefObject
    Implements JiwaApplication.IJiwaScheduledExecutionPlugin

   Private _plugin As JiwaApplication.Plugin.Plugin
   Private watcher As FileSystemWatcher
   Private watchFolder As String = "C:\Import\Watch"
   Private pendingFolder As String = "C:\Import\Pending"
   Private successFolder As String = "C:\Import\Succeeded"
   Private failedFolder As String = "C:\Import\Failed"
      
    Public Sub Execute(ByVal Plugin As JiwaApplication.Plugin.Plugin, ByVal Schedule As JiwaApplication.Schedule.Schedule) Implements JiwaApplication.IJiwaScheduledExecutionPlugin.Execute

    End Sub

    Public Sub OnServiceStart(ByVal Plugin As JiwaApplication.Plugin.Plugin) Implements JiwaApplication.IJiwaScheduledExecutionPlugin.OnServiceStart
      _plugin = Plugin
      ' Start watching a folder
      watcher = New FileSystemWatcher
      watcher.Path = watchFolder
      watcher.NotifyFilter = (NotifyFilters.LastWrite Or NotifyFilters.FileName)
      watcher.Filter = "*.xml"
         
      AddHandler watcher.Changed, AddressOf OnChanged
        AddHandler watcher.Created, AddressOf OnChanged
         
      watcher.EnableRaisingEvents = True
    End Sub

    Public Sub OnServiceStopping(ByVal Plugin As JiwaApplication.Plugin.Plugin) Implements JiwaApplication.IJiwaScheduledExecutionPlugin.OnServiceStopping
      RemoveHandler watcher.Changed, AddressOf OnChanged
        RemoveHandler watcher.Created, AddressOf OnChanged
    End Sub
      
   Private Sub OnChanged(source As Object, e As FileSystemEventArgs)
      If System.IO.File.Exists(e.FullPath) Then
           ' First Move the file to the pending folder      
         Dim pendingFileNameAndPath As String = Path.Combine(pendingFolder, System.IO.Path.GetFileName(e.FullPath)) & "." & now.ToString("yyyy-MM-ddTHmmss.FFF")
                     
         File.Move(e.FullPath, pendingFileNameAndPath)
         
         Try
            ' Now import the file using the import queue manager
            Dim importQueueManager As JiwaImportQManager.ImportQueueItemCollection = JiwaApplication.Manager.Instance.BusinessLogicFactory.CreateBusinessLogic(Of JiwaImportQManager.ImportQueueItemCollection)
            
            Dim queueItem As New JiwaImportQManager.ImportQueueItem
              queueItem.TransformedXML = File.ReadAllText(pendingFileNameAndPath)
              queueItem.Status = JiwaImportQManager.ImportQueueItem.ImportQueueItemStatuses.ReadyForImport
              importQueueManager.Add(queueItem)
              queueItem.Process()
               
            ' Move to success folder
            Dim successFileNameAndPath As String = Path.Combine(successFolder, System.IO.Path.GetFileName(pendingFileNameAndPath))
            File.Move(pendingFileNameAndPath, successFileNameAndPath)
               
            Dim Log As New System.Diagnostics.EventLog("Application")
             Log.Source = String.Format("Jiwa Plugin : {0}", _plugin.Name )
             Log.WriteEntry(String.Format("Successfully imported file '{0}'", pendingFileNameAndPath), System.Diagnostics.EventLogEntryType.SuccessAudit)
             Log.Close()   
               
         Catch ex As System.Exception
            ' Log the exception to the event log
            Dim Log As New System.Diagnostics.EventLog("Application")
             Log.Source = String.Format("Jiwa Plugin : {0}", _plugin.Name )            
             Log.WriteEntry(String.Format("Failed to import file '{0}' - Error : '{1}'", pendingFileNameAndPath, ex.Message), System.Diagnostics.EventLogEntryType.Error)
             Log.Close()   
            
            ' Move to failed folder
            Dim failedFileNameAndPath As String = Path.Combine(failedFolder, System.IO.Path.GetFileName(pendingFileNameAndPath))
            File.Move(pendingFileNameAndPath, failedFileNameAndPath)
               
         End Try
      End If
         
    End Sub         
End Class
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2583
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 807

Re: Importing XML files on-demand using the Plugin Scheduler

Postby Mike.Sheen » Thu Jun 12, 2014 10:39 pm

FYI - As of build 07.00.64.00 of Jiwa, the File Watcher Service is retired and is now replaced with a standard plugin (installed on all databases upgraded to 07.00.64 or later) which is identical to the above.

If you have been using the File Watcher Service for things other than XML importing, then you will need to create a plugin similar to the above. If you have need for guidance on how to do that, please create a new thread.

EDIT: As per Bug 10639 the standard plugin to replace the File Watcher service had been expanded to replace the JiwaCSVImport.exe (as well as replacing the JiwaXMLImport.exe and JiwaXMLQueueImport.exe).

If you have a need to import CSV or XML as not part of a file watching service (ie: on-demand) then a powershell script can fill that need.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2583
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 807


Return to Samples and Examples

Who is online

Users browsing this forum: No registered users and 0 guests

cron