Recently, I’ve been getting a lot of questions about how to build real-time integrations between Dynamics GP and other systems. It seems like everyone wants their data synced as quickly as possible!
While I’ve successfully designed several of these integrations, I’ve also noticed some common misunderstandings about what “real-time” actually means. My goal here is to share some insights based on my experience. This isn’t about being “right” or dictating a specific approach, but rather offering a practical perspective.
To illustrate my points, let’s imagine a company called Acme Sales. They use Dynamics GP for inventory management and a separate point-of-sale (POS) system for their retail operations. Acme wants any new inventory items or item/site records added in Dynamics GP to be instantly reflected in their POS system.
During an initial meeting, the sales manager emphasizes the need for immediate data transfer to enable sales as soon as entries are made in Dynamics GP. The IT developer, eager to deliver, sets out to create a “real-time” integration.
The developer’s first instinct is to write code that triggers whenever a new item is saved in Dynamics GP. This seems straightforward – capture the data when the save button is clicked and instantly send it to the POS. They then attempt to apply similar logic to the Item Quantities Maintenance window to track new site assignments for items.
However, challenges arise quickly. Acme Sales receives bulk updates of items from distributors monthly, which are imported directly into GP using eConnect. The developer realizes that monitoring the save button won’t capture these changes, throwing a wrench in their initial plan.
Additionally, working with the Item Quantities Maintenance window presents unforeseen technical hurdles. The Site ID field behaves differently than expected, and detecting item/site combinations becomes more complex than anticipated.
As the developer tests the prototype, they encounter unreliable network connectivity between the GP workstations and the POS system. Sometimes, the code execution causes delays or hangs for GP users. In other cases, the POS update fails, resulting in data discrepancies and confusing errors for the users.
To compensate for potential failures, the developer now has to create a separate synchronization process to ensure data consistency between GP and the POS. This sends them down a slippery slope of trying to achieve true real-time integration, which becomes increasingly complex and potentially unsustainable.
This scenario highlights three common misconceptions surrounding “real-time” integrations:
True “real-time” integration is often unnecessary. From a business perspective, is a delay of even a few milliseconds truly impactful? If an item takes 60 seconds to enter into GP, would a 1-second, 30-second, or even 5-minute delay in POS availability be noticeable? Shifting our mindset from “real-time” to “acceptable latency” helps us define more practical requirements. For most businesses, a delay of 2-5 minutes, or even longer, is perfectly acceptable. This is what we often refer to as “near real-time” integration.
User interfaces are unreliable triggers for integration logic. Attempting to capture data at the point of entry within the user interface might seem intuitive, but it leads to significant complications. Different save mechanisms, variations in user behavior, and the possibility of data entry bypassing the UI entirely make this approach unreliable. Instead, focus on the database as the single source of truth. All data eventually flows into SQL Server tables, making it a more stable and comprehensive point for monitoring changes.
Synchronous integrations are inherently fragile. Relying on a seamless, uninterrupted sequence of events is unrealistic in complex integrations. Network issues, server downtime, authentication problems, or even slight delays can disrupt the process. Asynchronous communication, often involving queues to temporarily store data until the destination system is ready, is crucial for robust integrations. This approach handles interruptions gracefully and prevents data loss. Message queues or even simple database tables can act as effective buffers in these scenarios.
By understanding and avoiding these common pitfalls, you’ll save yourself significant time and frustration. This will pave the way for designing fast, reliable, and maintainable integrations that meet your actual business requirements.