Visual Studio/Git integration.

[pricerc]

In reference to Mike's ticket https://service.jiwa.com.au/browse/DEV-7226.

There have been a couple of discussions around ways of developing plugins, and managing source/change control.

It occurred to me that those of us actively doing plugin development should probably share what we're doing and how, to help give Mike & the team some idea of what would help us plug-in developers.

So as promised; this is that thread

I'll post my ideas in separate comments on this thread, rather than just putting everything in here, mostly because my ideas are all over the place, and some will probably conflict with each other when I put them down in black and white.

[pricerc]

The first question that I think needs answering is:

Do we want Visual Studio (VS) integration, Git integration, or both? The answer of course, is both. But Jiwa's resources are finite, and whichever one they tackle first will render the other much less important, because diminishing returns. So perhaps a better question would be, if you had to choose, which one would you prefer to go first?

There's also the question of the goals. Is it about development, is it about version control and distribution or is it a combination of those?

1) good VS integration would allow VS to handle the source control integration, and make using VS as a plugin development platform much more attractive.
2) good Git integration on its own would allow VS and Jiwa to share a Git repository, which could probably simplify VS use, if not as much as 1).
3) good Git integration would ideally allow Jiwa to use remote repositories as a source for plugin updates. i.e. you and your customer could share an online git repo; you push changes from your development system, and then pull them onto their test and/or production systems.

[pricerc]

Because this background will (hopefully) inform the reader of where my thought processes are coming from.

I've been working with Jiwa since about 6.5.8, although there was quite a lull between all the work I did on 6.5.13 stabilising and our first Jiwa 7 upgrade, which I was involved in early 2017.

Those early forays into converting breakouts into plugins were 'entertaining', and a I went through a few revisions before I got to the system I'm using now.

Style-wise, I ended up working with many small plugins rather than one or two large ones. I found that there was a lot of 'Business Logic' overlap with the various UI components. For example, in the Sales Order Maintenance screen, I might need business logic from Sales, Debtors, Inventory, Purchasing and Creditors; and so, instead of duplicating code, I chose to maximise code re-use, and to use the 'Plugin Reference' feature extensively. This means that many of my plugins only use one of the "ApplicationManagerPlugin", "BusinessLogicPlugin" or "FormPlugin" sections. I also have ended up putting *most* custom fields and system settings into a single "Ryan's Custom Fields" plugin (as I do more customised display logic, that is likely to change, but since I'm currently only working with upgrades where they all end up in one plugin anyway, it's a bit easier to work with for now).

Most of my "business logic" plugins are comprised of a VB "Utilities Module" (initialised from an ApplicationManagerPlugin) and, where needed, a BusinessLogicPlugin.
The 'Utilities' module holds things like Extension methods that make doing stuff elsewhere easier. e.g.

Code: Select all

   <Extension()>
   Public Function DebtorInGroup(debtor As JiwaApplication.Entities.Debtor.Debtor, GroupId As String) As Boolean
      Return DebtorInGroup(debtor.RecID, GroupId)
   End Function


Allows me to do this:

Code: Select all

   Public Const PartnerGroupID As String = "CB05438A39EC46F5BC1A"
...
   If debtor.DebtorInGroup(PartnerGroupID) Then
       ' Do something Partner-y
   End If


UI modules are mostly comprised of (surprise!) just a FormPlugin.

Structurally, to help me keep track of the reference hierarchy, all my plugin names start with a 3-digit number, with the first digit representing a 'layer': '0' = business logic, '2' = EDI-related items, '8' = UI, '9' = scheduled tasks. The second digit is about the 'module', so e.g. '0' = 'module neutral', '1' = Debtors/Creditors, '2' = Inventory, '3' = Sales, '4' = Purchasing. The last digit is just a number.

I also give all my plugins their own namespace, mostly so that they can all co-exist in the same VS project (more on that later).

The project I'm working on at the moment (~25 plugins all up) looks something like this (names have been changed to protect the guilty innocent).

Plugin Name	Namespace	Modules	Jiwa classes	Dependencies	Notes
000 - Custom Fields	<None>	<None>	CustomFieldPlugin		(almost) all my custom fields.
001 - Shared Library	Ryan	RyansSharedLibrary	ApplicationManagerPlugin, SystemSettingPlugin		this module contains general-purpose helper functions. Things like error-logging, common custom field handling and system settings.
010 - Debtor Library	Ryan.DebtorLibrary	RyansDebtorLibrary	ApplicationManagerPlugin	001
015 - Creditor Library	Ryan.CreditorLibrary	RyansCreditorLibrary	ApplicationManagerPlugin, BusinessLogicPlugin	001
020 - Inventory Library	Ryan.InventoryLibrary	RyansInventoryLibrary	ApplicationManagerPlugin, BusinessLogicPlugin	001
030 - Sales Order Library	Ryan.SalesOrderLibrary	RyansSalesOrderLibrary	ApplicationManagerPlugin, BusinessLogicPlugin	001, 010
040 - Purchase Order Library	Ryan.PurchaseOrderLibrary	RyansPurchaseOrderLibrary	ApplicationManagerPlugin, BusinessLogicPlugin	001, 015, 030
831 - Sales Order UI	Ryan.SalesOrderUI		FormPlugin	001, 010, 020, 030, 040
841 - Purchase Order UI	Ryan.PurchaseOrderUI		FormPlugin	001, 020, 030, 040

My plugin development workflow looks a bit like this:

1. For a customer's Jiwa 7 implementation, I create a Visual Studio solution, with a project for each language that I'm using (VB and/or C#). I use an actual project file, but just 'opening a folder' might work just as well.
2. I have some 'standard' utility plugins that I import into Jiwa, and then I add new plugins via the Jiwa UI
3. I have a Microsoft Access database that queries the plugin table (using ODBC-linked tables), and exports the plugin code for all plugins with me as the author, to an 'export' folder (not the VS project folder); setting the file timestamp to match the 'LastSaved' time of the plugin.
4. I use WinMerge (which is back under maintenance, which is good news), to compare the exported plugins to the VS plugins. Initially, this is just copying from the export folder to the project folder, and 'adding' the files to the project. But later, it can be merging in both directions, because sometimes I do minor editing within the Jiwa plugin editor while debugging, and then feed it back, while also working on files within VS.
5. I maintain the plugins in Visual Studio.
6. I use WinMerge again to move any changes back to the 'Export' folder
7. The 'Export' folder now becomes an 'Import' folder, and I use the Access database to copy the plugin code back up to Jiwa. This updates the version number field, as well as the 'LastSaved' timestamp.
8. Load Jiwa and fix any dependencies; whether they be plugin, assembly or embedded; restart Jiwa if needed.
9. Test & debug in Jiwa
10. Rinse and repeat

Notes
for the solution, an earlier approach I tried, which is probably technically closer to the way plugins work within Jiwa, was to create a project (and folder) for each plugin. This worked well, until I wanted to move code in-and-out of Jiwa in 'bulk' and compare Jiwa's code with my VS code. So I changed to one folder per language. I suspect this will change if we get some kind of Jiwa integration going with either VS or Git.

[pricerc]

In an ideal world, I should be able to:

1. Open up Visual Studio
2. Create a new 'Jiwa Plugin Project' (C# or VB)
3. Add further plugin projects to the solution.
4. A new project would automatically have the standard 'Jiwa' plugin classes (FormPlugin, BusinessLogicPlugin, etc).
5. Edit the project(s) properties to specify my development Jiwa database, and the location of my Jiwa executable.
6. Develop the plugin(s)
7. 'Build' the project, which will automatically populate the correct Jiwa tables.
8. Click 'Debug', and have Visual Studio start Jiwa, and attach to it for debugging.

A separate, and related discussion, is what structure should a plugin project have: a) within VS, and b) inside the Jiwa database. I'll come back to that after I've had my dinner

[pricerc]

In its simplest form, I imagine this would involve a mechanism for saving a git repository URL and credentials, and having either the XML or SQL for a plugin sync with a folder in the repo.

I've already got a plugin that will extract all plugins to a folder in XML or SQL format; so extending that with some git commands is probably not difficult; it could probably just spawn git commands to do so, or otherwise integrate using libgit2sharp https://github.com/libgit2/libgit2sharp/. I could then extend it to pull from the repository as well and load up updated versions.

A more sophisticated option would basically require the work involved in a VS integration exercise, where the plugin components are exported as a set of projects in a folder hierarchy, and the whole hierarchy is linked to a git repository.

[pricerc]

At the moment, I put all my source files into one big project, because that's convenient for the way I work. However, an effective integration with Jiwa would have to work differently.

Without needing any changes (that I can think of) to the way Jiwa currently handles plugins, I think the following could work.

Each plugin compiles to its own DLL, and the source code is put into its own folder under C:\ProgramData\... to get compiled.

So, to me, the natural thing would be for each plugin to get its own folder and its own VS project (VB or CS, as the case may be), within a solution folder.

Besides the source code, some additional information would be required. Version numbers, references, documents, images, etc.

• Metadata, like version and author, can be saved within the VS project file. A project file even has a ProjectGuid, which could just be the Plugin's RecID.
• Plugin references would be project references (i.e. references to other projects in the same solution)
• Embedded references would be saved in a local folder and added as a 'Local Copy' reference.
• Assembly references should be just normal references.
• Business Logic/Form links would need be saved in a special file, because they only have meaning in Jiwa.
• Related documents and images should probably go into appropriate 'content' folders.
• Custom fields and system settings should probably go into discrete JSON or XML files
• Metadata that doesn't fit into the project file could go into a custom 'meta' file.
• Schedules are possibly out-of-scope, but could be stored in a schedules folder, in JSON or XML format.

Using my current project for inspiration, something like:

Code: Select all

C:\Projects\Jiwa\CustomerX\Plugins\
   - CustomerXPlugins.sln

C:\Projects\Jiwa\CustomerX\Plugins\000 - Shared Library\
   - 000 - Shared Library.vbproj
   - 000 - Shared Library.vb

C:\Projects\Jiwa\CustomerX\Plugins\001 - Card Processing\
   - 001 - Card Processing.vbproj
   - 001 - Card Processing.vb
   - Documents\Certificate.PFX
   - EmbeddedReferences\CardPaymentLibrary.DLL
   - EmbeddedReferences\Log4Net.DLL
   - SystemSettings.XML

C:\Projects\Jiwa\CustomerX\Plugins\010 - Debtor Library\
   - 010 - Debtor Library.vbproj
   - 010 - Debtor Library.vb
   - BusinessLogicLinks.XML {JiwaDebtor}
   - CustomFields.XML

C:\Projects\Jiwa\CustomerX\Plugins\831 - Sales Order UI\
   - 831 - Sales Order UI.vbproj
   - 831 - Sales Order UI.vb
   - FormLinks.XML {SalesOrderEntryForm, BackToBackSalesOrderEntryForm}
   


Given that the structure of project files is well defined, it shouldn't be too onerous (but nonetheless, probably a couple of weeks' work) to achieve a Plugin that can export something like this, and then re-import changes - with some caveats around the inherent limitations with the plugin system. I don't think it needs to be made completely 'idiot proof', because the target market would be Jiwa resellers or consultants like me, not end-users; i.e. it can just use standard VB/CS projects, it doesn't need special handling to make sure that the project doesn't get broken.

I'm not sure how this structure would affect namespaces, but I suspect they'll still be necessary if you're going to have multiple plugins talking to each other. I've tried to think of a way to automate namespace naming, but I've been unsuccessful.

If I could make one change to the way Jiwa does things that would make this work even better, it would be if we could have multiple source files per plugin, so that, for example, the different plugin classes (BusinesLogicPlugin, FormPlugin, etc) could each be in their own file, and that we could add additional files for things like custom forms and data structures. I think that the additional DB structure required would be simple - a new table that contains optional additional code files.

Code: Select all

SY_Plugin_Code(RecID, SY_Plugin_RecID, Name, Code, FileHash, CreateDateTime, LastWriteDateTime, RowHash)

[pricerc]

Ok. Those are my "initial" thoughts on the topic

For those subscribing to the forums: sorry about the spam, but I thought multiple posts would be easier to digest than one big one.

[Mike.Sheen]

Thanks for the comprehensive write-up and suggestions!

I'd be also interested as what people think about integrating VSCode into Jiwa (it's an Electron app - which is just a desktop application hosting a browser - so it can be relatively easily embedded in a form within Jiwa) as an alternative to the current plugin maintenance form.

As VSCode already has integration with AzDo (Azure Devops nee VSTS), TFS as well as Git, has a flourishing and rich extension ecosystem and is open source it may be a compelling technology to leverage for a more pleasant plugin development and management experience.

[pricerc]

Thanks for the comprehensive write-up and suggestions!

You're welcome.

I should probably review them myself (and clean them up a bit where necessary). I know they're a bit rambling, but it there was a lot to get through, because it's a broad topic.

For me, I guess VSCode vs VS would be situation-dependent.

The project I'm working on right now is substantial, because of the sheer volume of JIWA 6 customisations the customer has, and I think VS has a more complete 'project' experience (most importantly intellisense and refactoring tools).

On the other hand, we're expecting to do an upgrade soon for customer with next-to-no customisations, in which case just an enhanced environment within JIWA would be good.

[SBarnes]

Mike wrote:
I'd be also interested as what people think about integrating VSCode into Jiwa (it's an Electron app - which is just a desktop application hosting a browser - so it can be relatively easily embedded in a form within Jiwa) as an alternative to the current plugin maintenance form.

As VSCode already has integration with AzDo (Azure Devops nee VSTS), TFS as well as Git, has a flourishing and rich extension ecosystem and is open source it may be a compelling technology to leverage for a more pleasant plugin development and management experience.

To be honest I am not familiar with the VS Code environment but have used other web based IDEs I've got to say I am not a fan given a web browser in a lot of cases represents the lowest common denominator to any user.

The biggest issues as I see it with the current Jiwa Plugin maintenance are

Only one source file
No integration of other assets, images, web pages etc
No source control integration
No Nuget / 3rd Party Library integration

Visual Studio is an environment that is easily accessible and the comes with the most features that I would assume most professional developers would have access to and be the most familiar with so my vote would be for integration with it.

Jiwa is an ERP not an IDE, and whilst a full blown IDE in Jiwa would be wonderful, I doubt its going to be a reality but better integration with visual studio is something I would like to see.

It would also be helpful it there was a way to drag and drop controls to not only extend custom forms but also Jiwa's standard forms.