Power BI Incremental Refresh for Corrected Production Records

A production report can refresh successfully and still contain old errors. The latest sewing figures may be correct. A revised order from last month may still be wrong. A deleted transaction may survive in a historical partition. A moved transaction may appear twice.

Power BI incremental refresh reloads part of a large table on a routine basis. It leaves older parts stored in the model. That is useful only when the refresh policy matches how the factory changes its records. Plan for late entries and backdated corrections. Include cancellations and dates that move.

This guide develops an original four-record example to expose those risks. It explains the source fields and filter tests. It also shows how to choose a correction window and check a historical repair. The example quantities are hypothetical. Configuration details should be checked against the deployed Power BI environment and connector.

Understand what the policy keeps unchanged

Incremental refresh divides a table into date-based partitions. The policy refreshes recent partitions. Older ones can stay stored without routine reloading. The service manages those partitions after the policy is published and applied. Microsoft incremental refresh overview

That design creates a clear contract with the source. Older records must stay stable or have another route into the model. The routine window will not cover every change.

An apparel ERP often needs valid corrections after a shift or order closes. A receipt may be assigned to the wrong lot. An inspection count may be corrected. A cancelled issue may require reversal. Ask how the source stores those changes. Then choose a refresh period that can handle them.

Do not equate “historical” with “cannot change.” The term describes the refresh policy. It does not prove that the business record is final.

ApparelDiary's ERP factory checklist explains why stock events and corrections need traceable records. Incremental refresh must carry those valid changes into the reporting model.

Give the source three separate identities

The source view needs three fields. Provide a stable record identifier, a partitioning date, and a change timestamp. Each answers a different question.

The first three fields are not substitutes for each other. A correction made today can still belong to an old production date. A new interface receipt timestamp does not necessarily mean the business record changed.

Define which edits update LastModifiedUtc. The proposed rule should cover quantity and status changes. Include relevant dimensions and deletion markers too. Confirm that batch imports and administrative repairs follow the same rule.

Keep source timestamps on a documented time basis. The change column may look precise. It can still mislead if one interface writes local time and another writes UTC.

Test the date filter before enabling the policy

Power BI uses the reserved, case-sensitive parameter names RangeStart and RangeEnd. Their values define the date range used to retrieve a partition. The service replaces the development values when applying the policy. Parameter and partition behaviour

An illustrative Power Query filter is:

Table.SelectRows(
SourceRows,
each [ProductionTimeUtc] >= RangeStart
and [ProductionTimeUtc] < RangeEnd
)

Use compatible date/time types and a consistent time basis. The example includes the start and excludes the end.

Microsoft warns against equality at both boundaries. A boundary row could then match two adjacent partitions. Incremental refresh configuration

Build a small test set. Place one row before the start and one exactly at the start. Add one inside the range and one exactly at the end. The middle two should appear. The end row belongs to the next interval.

Keep the test values in the handover record. A screenshot may show the filter dialog. The test records must show that its boundaries work.

Verify that the source does the intended filtering

A date filter can appear in Power Query while the source still returns far more data than intended. Check the actual query path.

Query folding translates supported transformations into operations performed by the data source. Unsupported transformations may run in the Power Query engine. The connector, source, and transformation sequence all affect the result. Query folding basics

For a relational source, inspect the generated query or use authorised source-side tracing. Confirm that the partition's start and end constraints reach the source in a useful form.

A small development window should retrieve the intended subset. A window may contain just a few thousand rows. If it causes a very large source read, check the filter and transformation steps.

Converting the partition date to text may seem convenient. Check its effect before using it. An upstream reporting view may help. Give its date field the correct type.

Microsoft describes folding failures in its troubleshooting guidance. Such faults can cause repeated data scans for multiple partitions. Incremental refresh troubleshooting

Calculate the expected window size carefully

Suppose a hypothetical plant produces 100,000 reporting rows per day. A 365-day history contains:

Stored rows = 100,000 rows/day × 365 days = 36,500,000 rows.

A seven-day refresh window contains:

Window rows = 100,000 rows/day × 7 days = 700,000 rows.

The window contains about 1.92% of the year's rows:

Window share = 700,000 ÷ 36,500,000 × 100 = 1.92%.

That is a row-distribution calculation under a uniform-volume assumption. It is not a measured refresh-time saving, a source-read count, or a capacity guarantee.

Actual work depends on several things. Query plans, partition grouping, compression, and transformations matter. So do source load and the periods that changed. Measure the real operation after confirming correctness.

Also test peak volume. A month-end packing upload may be much larger than an ordinary day. A uniform average can hide the period most likely to strain the refresh.

Follow four corrections through the model

For this example, set the routine window to start at 00:00 on 22 October. It ends at 00:00 on 29 October. Include the start and exclude the end. A record's partition date decides whether it falls in that window.

Before the changes, it contains these records:

The initial total is 1,050 pieces.

Now the source changes A to 590 pieces and B to 280 pieces. It removes C through a hard delete. It also corrects D's production date from 20 October to 27 October without changing its 50 pieces.

The current source total is:

Current source pieces = 590 + 280 + 50 = 920 pieces.

Record A is inside the routine window. Record B's partition date remains outside it. C has vanished from an old source period. D now belongs to a recent period, while its earlier model copy may still remain in history.

These are four different refresh cases. One successful recent refresh does not resolve them all.

Reconcile the misleading successful result

In this example, refreshing only the recent interval replaces A with 590 pieces. It also loads D's current 50-piece row into the recent period.

The historical model still contains B at 300, C at 100, and D's old 50-piece row. The resulting total is:

Model pieces = 590 + 300 + 100 + 50 + 50 = 1,090 pieces.

The difference from the current source is:

Unreconciled difference = 1,090 − 920 = 170 pieces.

The difference has three causes:

A refresh status of “Completed” says that the requested refresh operation completed. It does not prove that every period needed by the business correction was included.

A general warning says the dashboard is wrong. This record-level check explains why. It tells the team which periods and records require repair.

Know what change detection can and cannot find

Power BI's Detect data changes option uses a date/time audit column. Microsoft describes evaluating the maximum value for each period in the incremental range. The audit column should differ from the date column used to partition the table. Change detection configuration

The important limit is the evaluated range. An old production record may have a recent LastModifiedUtc value. That does not automatically bring its old partition into the routine window.

Suppose B's change timestamp is updated on 28 October. That alone does not guarantee a refresh of the 20 October partition. The policy still needs a route to that period.

Test the behaviour with an actual backdated source change. Check the row in the model after refresh, not just the source audit column.

Treat change detection as a way to avoid unnecessary work within a suitable policy. It is not a universal substitute for understanding historical corrections.

Prefer an explicit deletion contract

A hard-deleted row leaves no current row for a change-detection scan to inspect. Microsoft contrasts this with a soft deletion. The row remains, but it is marked deleted. Its change timestamp is updated too. Deletion limits

For a reporting source, define how cancellation and removal are represented. One option is a retained deletion marker. Another is a reversal event. A separate change log could also drive historical repair.

Do not invent a source record to compensate for a missing one. Use the ERP's approved transaction semantics.

Test the extraction path as well as the source table. An upstream view may remove deletion markers. The detection logic could then miss the intended signal.

A soft deletion still needs the affected period to be refreshed. It does not make a very short window sufficient for unlimited historical changes.

Keep the deletion count and affected identifiers in the comparison record. They help explain a smaller model total.

Avoid changing the partition date casually

A record can move between partition periods. It may leave a stale copy behind. It can also disappear from the model's effective history. Microsoft identifies changed transaction dates as a problem to check. They can cause duplicate or inconsistent results. Changed-date troubleshooting

In the worked case, D needs attention in both its old and new periods. Refreshing the new period alone adds its current row without removing the old copy.

One option is a partitioning field that cannot change. Use it only if it fits the data

contract. Keep the corrected business date in another field. This choice is not a universal answer. It changes the link between the refresh window and the business process.

The business may need a changeable production date as the partition key. If so, log both its old and new periods. Use it to identify all affected partitions.

Test a movement in each direction. Move a record from old history into the current window. Then test the reverse move. The two cases can fail in different ways.

Choose the window from correction evidence

Ask for the age of actual corrections over a representative period. For this review, age starts at the partitioning event. It ends at that record's final relevant change.

Do not use only the average. A small number of old changes can matter greatly to stock or shipment decisions.

Separate common corrections from exceptional restatements. A seven-day window may fit one process. Rare older changes would then need a controlled repair. Another process may need a much wider window.

Record the operating agreement. State who can request an older refresh. Define the evidence needed and how to check the repaired result.

A longer window increases routine work. A shorter one increases dependence on the exception process. Compare those costs using real correction patterns and tested source capacity.

The policy should keep the report in step with authorised source changes. Fast completion alone does not prove that it works.

Give historical repairs an approved route

The XMLA endpoint supports advanced partition management and selective historical refresh. The deployed capacity and permissions must support that use. Microsoft's guidance describes targeted backfills and metadata-only deployment options. Advanced incremental refresh

For the example, the repair needs the periods containing B, C, and D's old copy, as well as a current recent refresh. The actual partition may span a month or another managed period. It need not cover just one calendar day.

Identify the real partition boundaries before issuing the operation. Do not assume a name based only on the transaction date.

Some environments cannot run selective partition operations. Use a supported alternative agreed by the model owner. The team might change the policy or perform a broader rebuild. Assess its resource impact first.

Keep the repair procedure limited to authorised reporting administration. It should not require changing the ERP record again merely to force a refresh.

Reconcile a repair before reopening the report

After the repair, compare source and model at the same observation cutoff. Check the original transaction identifiers and the relevant quantities.

For the four-record example, the expected live record set is A, B, and D, with quantities 590, 280, and 50. C should no longer contribute. D should contribute once.

Confirm that the model total is 920 pieces. Also check distinct TransactionID count and the row count. A total alone can conceal offsetting errors.

Review affected dimensions, such as plant, order, buyer, or material. A correction can move quantity between groups without changing the grand total.

Retain before-and-after results and the refresh operation record. Report users may be making decisions from old periods. Tell them when those periods change.

Do not describe the data as final unless the business has a separate finalisation rule. A reconciled snapshot can still receive a later authorised correction.

Coordinate the source cutoff

Several partitions may be read at different times. If the source changes during extraction, a comparison can mix states.

Choose a source consistency approach that fits the platform. A reporting snapshot or another controlled extract boundary can make reconciliation easier. Verify what the source and connector actually support.

Record a source snapshot identifier or cutoff where available. Match the control totals to that same state.

A model may have refreshed at 08:00. A source query at 09:30 can include later changes. Those differences are not all refresh defects. Separate new business activity from missed updates.

An exceptional correction may need a coordinated extract. Use the approved operational process to arrange or pause it. The reporting team should not interrupt production transactions casually.

Record the boundaries so another analyst can reproduce the comparison.

Treat initial loading as a separate exercise

The first service refresh creates the historical structure. It then loads that history as defined by the policy. It can have a much larger workload than routine refreshes. Initial refresh behaviour

Plan that operation with source owners. Check the history range, credentials, gateway route, and expected volume before launch.

Use a small controlled model to test logic, then assess the full initial load with realistic data. A five-minute development refresh does not predict the first multi-year load.

Advanced partition management can support staged loading in suitable environments. Confirm the deployed capacity and permissions before choosing that method. Staged historical loading

Retain a restart procedure. Find the cause when an initial load fails. Check source filtering, resource limits, credentials, and individual partitions. Do not repeat the same large job without finding the cause. That can burden the factory's systems without fixing the fault.

Do not confuse real-time access with historical repair

Some configurations support a real-time DirectQuery partition. It can cover recent data after the import refresh boundary. It does not automatically revisit all old imported periods. Hybrid partition scope

Return to the worked example. Showing newer sewing events in near real time would not remove C from August history.

Document which part of the report is imported and which part queries the source. Test totals across that boundary, including an event exactly at the handoff.

Check source availability before enabling a live path. Assess the effect of users querying it at the same time. A dashboard may open quickly in a quiet test. Check how it behaves during a busy production review too.

Keep the business promise precise. Recent visibility, historical accuracy, and report availability are linked needs. Each still needs its own evidence.

Monitor the result as well as the refresh status

Show when the model last refreshed successfully and which source cutoff it used. Also show the age of each unresolved historical correction request.

Check row counts and quantities by selected periods. Count duplicate transaction identifiers. Also count source changes that still need a backfill.

A refresh failure needs an owner, but a reconciliation failure does too. The model can be technically available while its business totals are under review.

Microsoft distinguishes data refresh, partial refresh, and other refresh-related behaviour. Use those terms carefully when writing the support procedure. Power BI data refresh

After a model deployment, repeat a small correction test. A source view, parameter type, or query step can change. Any of these may affect the refresh while leaving the report layout unchanged.

Keep the evidence compact enough that the support team will use it during an incident.

Keep a mutation test beside the model

A useful release test makes several changes to a controlled source dataset. Check the model after each supported refresh path. This is more informative than repeatedly refreshing an unchanged table.

Save a baseline. Record its row identifiers, quantities, and partition dates. Insert a recent row and update a recent quantity. Cancel a recent row through the source's approved method. Confirm the expected result after the normal refresh.

Next, repeat those actions against an older period. The expected outcome should reflect the documented policy. The test may require a backfill. Raise that request. Show that the older model is unresolved until the backfill completes. It should not report a false pass because the routine job finished.

Add a date-movement test like record D. Check the old and new locations. Then search for the identifier across the whole model. This catches a duplicate that a current-period total alone can miss.

Finally, change a dimension value without changing the quantity. For example, change the order on a test record. Use an approved correction to make that change. Confirm both order totals. A quantity-only reconciliation would not detect a stale order assignment.

Keep the expected outputs in a small machine-readable file or controlled worksheet. Record which tests were executed, by whom, against which model version. These are proposed acceptance cases. Extend them to the organisation's own transaction types and source rules.

Questions that often cause confusion

Does LastModifiedUtc make the entire history self-correcting?

No. The refresh policy and detection range still matter. An old period needs an explicit supported refresh route.

Can I fix duplicate records by removing duplicates in Power Query?

That may hide the symptom while retaining the wrong version or period. Find why the same identifier appears across partitions and correct the loading contract.

Should every table use incremental refresh?

No. Evaluate volume, change behaviour, source filtering, and maintenance effort. A small reference table may be easier to refresh in full.

Does a completed refresh prove current stock is correct?

It proves completion of the requested operation. Stock accuracy still depends on correct source transactions, scope, and reconciliation.

Make every correction reach its intended period

Incremental refresh needs consistent rules. Its date filter, source change rules, and repair process must agree. Keep identifiers stable and test exact boundaries. A recent change timestamp is different from an old partitioning date.

The four-record example shows why success must be checked at record level. A model can refresh recently and still overstate the source by 170 pieces. Document how to repair history. Keep a comparison that another person can repeat. Those steps give the reporting team a way to manage the risk.

Refresh design supports dashboards, so see factory KPI dashboard design, the operations dashboard, the manufacturing analytics stack and ERP for the apparel industry.

References

• Microsoft Learn — Incremental refresh overview.

• Microsoft Learn — Configure incremental refresh.

• Microsoft Learn — Troubleshoot incremental refresh.

• Microsoft Learn — Advanced incremental refresh with XMLA.

• Microsoft Learn — Query folding basics.

• Microsoft Learn — Data refresh in Power BI.

• ApparelDiary — ERP for the apparel industry.