This blog has a new home! You can now find it at: https://blog.steveendow.com/. Dynamics GP Land will no longer host new posts.
Thank you!
By Steve Endow
If you’ve built a Dynamics GP integration with eConnect and .NET, you’re likely familiar with the eConnect Serialization process. This process uses the eConnect Serialization .NET assembly to transform your transaction objects and data into an XML document.
Let’s look at an example of serializing a GL transaction.
You’ll notice this code uses a single GL transaction object, glTrans, with its properties populated by glTrxHeader and glTransLines.
This single GL transaction is then placed as a lone element within the glType array, which is subsequently converted to XML.
But wait… why is the GLTransactionType object, named glType in this instance, an array?
The array format exists because eConnect Serialization can handle converting multiple transactions into a single XML document.
What?
This begs the question: Why would you choose to serialize multiple transactions together instead of processing them one by one?
That’s a great question!
This video delves into this feature and provides a performance demonstration:
I had the same question over a decade ago, back when I was working with eConnect on Dynamics GP 7.5 using Visual Basic 6 and the eConnect COM components.
The presence of transaction array support in eConnect serialization suggested an underlying reason for its existence, so I designed a test to compare import performance.
I vaguely recall achieving an average import rate of 20 AP invoice transactions per second with VB 6 when serializing and sending transactions individually. This was considerably faster than what I observe with .NET today. Interestingly, my tests with serializing and sending multiple transactions simultaneously didn’t yield any performance gains, leaving me perplexed about the functionality.
Although there were no performance benefits, I discovered a drawback to sending multiple transactions to eConnect at once. If eConnect encountered an error during the import of any transaction within the ‘batch’, the entire batch would be rolled back, resulting in no transactions being saved. For instance, if you sent 100 transactions and one failed, none would be imported.
Adding to the frustration, if your batch of 100 transactions contained five erroneous transactions, eConnect would only return one error. It would stop processing as soon as the first error was encountered, roll back all changes, and return a single error. Resubmitting after fixing the first error would lead to eConnect stopping at the second error. This forces you to address errors sequentially, preventing any transactions from being imported until all errors are resolved. Not an ideal error handling process.
Fast forward to 2018, when I was asked if eConnect could import multiple transactions in a single call. My initial response was that it could, but there was no performance benefit in doing so.
Then, for reasons unknown, I felt compelled to test and verify my long-held ‘belief’ by comparing single versus multiple transaction submissions.
I dusted off my eConnect load testing tool and modified it to accommodate the serialization of multiple transactions alongside single transaction submissions.
When I saw the results of sending multiple transactions to eConnect in a single submission…
My…
Jaw…
Dropped.
100 GL JE transactions, when sent individually to eConnect, took 17 seconds to import.
However, when I submitted a single XML document containing 100 journal entries, eConnect imported them in a mere 3 seconds.
WHAT???
Intrigued, I then imported 700 transactions in a single import request to eConnect. They were imported in 17 seconds – the same time it took to import only 100 transactions individually.
This indicates a remarkable 7-fold performance enhancement achieved through a minor tweak in the import code. The difference is astounding.
Multiple test repetitions confirmed that this was not a fluke.
I then conducted tests with AP Invoices and observed similar results. In one test, I managed to import 750 AP Invoices in a single submission within the same timeframe as importing 100 AP Invoices individually. Absolutely incredible!
These tests with GL JEs and AP Invoices clearly demonstrate that submitting multiple transactions to eConnect simultaneously using the eConnect .NET assemblies can lead to a significant performance boost.
It’s possible that the conversion of eConnect assemblies to .NET introduced changes that slowed down individual transaction imports but improved the performance of multiple transaction imports.
However…
There’s a catch.
It’s the same catch I encountered while testing with VB 6 and eConnect COM objects. A single import error will cause the entire batch to fail, and no transactions will be saved. If you attempt to import 500 transactions and one encounters an error, all 500 will be rolled back, requiring you to fix the error before resubmitting. Subsequent errors will result in the entire batch failing again. Essentially, 495 transactions will be held hostage by 5 errors – a frustrating situation.
Furthermore, if you’ve assigned 500 journal entry numbers and the batch fails to import, you’ve wasted 500 JEs. Auditors reviewing the JEs in Dynamics GP will notice a gap of 500 JE numbers, prompting questions. Frequent errors like this can quickly deplete thousands of transaction numbers. Not a catastrophic issue, but not ideal either.
So, considering these drawbacks, what’s the verdict?
My recommendation is that most Dynamics GP users should stick to submitting single transactions to eConnect. For most, eConnect imports aren’t a business bottleneck, and it’s uncommon for customers to experience delays waiting for eConnect integrations to finish. Usually, customers can import hundreds or thousands of transactions fairly quickly. The bottleneck typically arises during transaction posting in Dynamics GP, which is significantly slower than importing.
However, some high-volume customers import tens or hundreds of thousands of transactions daily, with imports or postings running continuously. If you have Dynamics GP integrations that take hours to complete, and this technique can save you hours of import time, then it’s worth considering. However, this approach is only advisable if you: 1) can implement pre-validation routines for all transactions to minimize errors, 2) are equipped to handle errors and reimport failed transactions, and 3) can develop a system to manage transaction numbers to avoid wastage during failed imports.
I hope this information proves valuable!
Steve Endow is a Microsoft MVP in Los Angeles. He is the owner of Precipio Services, which provides Dynamics GP integrations, customizations, and automation solutions.
You can also find him on Twitter, YouTube, and Google+
http://www.precipioservices.com
