Why .NET Integration Projects Fail: A Pattern Across Multiple Engagements

Integration projects that fail have one thing in common, and it shows up before development starts. Nobody on the project can describe precisely what the two systems disagree about.

They can describe what each system does and what the end state should be. But ask what happens when a client exists in System A and not System B, or which side wins when both hold a value and the values differ, and the answer turns general. It'll be handled during implementation. We'll map the fields in development. That's a data question for the business.

That vagueness is not a scoping oversight. It's the project's central risk, deferred to the phase where addressing it costs the most.

The disagreement is the project

Data movement is the trivial part. The actual work is reconciling two systems that have described overlapping reality differently for years, each internally consistent and mutually incompatible.

System A thinks a client is one row. System B thinks a client is a hierarchy with a parent organization and locations. System A allows a record with no assigned owner; System B requires one. System A stores status as one of four values; System B has eleven, three deprecated but still present. Neither is wrong. They were built for different purposes at different times.

Every mismatch requires a decision, and almost none are technical. Whether an unowned record is rejected, assigned a default owner, or synced with a null is a business rule. Getting it wrong is invisible in testing and expensive in production.

Projects that succeed do this during scoping, write the rules down, and get someone with authority to agree. Projects that fail defer it, and developers make the decisions implicitly — not out of arrogance, but because a developer facing an unspecified case has to write something. Fifty reasonable choices later, the system's behavior is the sum of decisions nobody made deliberately and nobody can now enumerate.

"We'll handle it in the integration"

The second pattern announces itself in the phrase people use for it.

It comes up when profiling reveals a quality problem in a source system: duplicate client records, inconsistent naming, a field mandatory in policy and empty in eight percent of rows. Fixing it upstream means a cleanup project with an owner and a timeline. Handling it in the integration means normalization logic, and it can start tomorrow.

It is almost always the wrong choice, and it's almost always chosen, because the cost lands later and on someone else.

Two things follow. The integration accumulates correction logic — special cases for known-bad records, fuzzy matching for inconsistent naming, defaults for fields that should have been populated. That layer becomes the most complex part of the system, and nobody outside it can predict its behavior.

More damaging, the upstream problem becomes invisible. It was producing friction, and friction was generating pressure to fix it. The integration absorbs the friction, the pressure disappears, and the bad data keeps being created, because nothing about the process that creates it changed.

What works is fixing upstream what can be fixed, and where it can't, having the integration reject bad records loudly rather than silently repairing them. Rejection is uncomfortable and keeps the problem where it belongs.

The error handling gap

Integration specs describe the happy path in detail and treat the rest as an implementation concern. Twenty pages on field mapping, one paragraph saying errors will be logged and retried.

Six months later that produces a recognizable situation. The integration has been failing intermittently for weeks, logging into a file nobody reads because no alert was defined. Retries are running and they aren't idempotent, so they created duplicates, and the duplicates flowed downstream. Nobody knows how many records were affected, because failed items were never recorded in a queryable form.

Remediation costs more than the original integration, and most of that is reconstructing what happened — possible only if the logs reach back far enough, which they often don't.

What durable integrations include

A short list, and the one that gets cut first when a scope has to fit a number.

A written reconciliation of how the two systems disagree, with the decision for each case, agreed before development.

Explicit behavior for records that don't match — surfaced in a report someone owns, never silently skipped.

Idempotent operations, so a replay after failure doesn't duplicate anything. That's a design decision, not a feature; retrofitting it means reworking the data model.

Alerting tied to the absence of expected activity, not only to thrown exceptions. The dangerous failure is the job that runs, processes nothing, and reports success.

And a reconciliation check that can answer whether the two systems agree right now. Without one, the most anyone can say is that nobody has complained lately.