Why Are There Gaps in Your Historian Data?
Historian data may be missing because the source stopped updating, collection failed, storage rejected records, or the query hid records that still exist. An empty chart alone does not establish which happened. Start with one affected tag and a precise time window, then compare the source, collector, stored records and display. Check timestamps and quality alongside values before calling the interval a data gap.
Is the data missing, or is the process value flat?
First define what you expected to see: a stored sample at every collection interval, a record whenever the value changed, or a continuous trend.
Some collection and storage configurations retain changes rather than every repeated value. A deadband defines a change threshold; compression can reduce stored points according to configured rules. A steady process can therefore have sparse records without a collection failure. The exact behaviour depends on the historian and its settings.
Do not accept a flat line as proof of healthy collection either. Establish whether the display is showing fresh samples, carrying forward an old value, or drawing between stored points.
Ask for the underlying timestamps, values and quality indicators. Compare them with the tag's expected recording behaviour. The useful question is whether the available records meet that expectation.
Where should you start tracing the gap?
Choose one affected tag, one nearby unaffected tag and a shared time window. Record the timezone explicitly. Keep this scope consistent while moving through the data path.
Use this table as an investigation checklist.
| Stage | Evidence to request | Question to resolve |
|---|---|---|
| Source signal | Source value, timestamp and diagnostic status | Was a valid reading available? |
| Collection | Connection history, tag status and buffer records | Did the collector receive and retain it? |
| Storage | Write results and raw archived records | Did storage accept the record? |
| Query | Tag identity, time boundaries and filters | Did the request include the record? |
| Display | Returned data and chart settings | Did the screen represent it correctly? |
Find the first boundary where evidence of the expected record disappears. Avoid restarting equipment or changing settings before preserving the available diagnostics.
Was the source signal actually updating?
Inspect the source through an authorised diagnostic view. Compare its timestamp and status with the affected interval; today's healthy reading does not explain an earlier gap.
Investigate whether the source was available, the tag remained mapped to the intended signal, and collection was expected during that operating state. Check recorded changes to equipment, tag definitions and operating modes.
Separate a process question from a communications question. “The temperature stayed steady” and “the temperature reading stopped updating” require different evidence.
If the source never produced a valid reading, downstream troubleshooting cannot establish the missing process value. Keep any later estimate clearly separate from measured records.
What do quality flags tell you?
Read the quality information attached to the records, including any detailed status code. A numeric value alone is insufficient evidence that the reading was usable.
Quality categories can distinguish usable readings from uncertain or failed readings, but their definitions and propagation depend on the source, interface and historian. A status may describe a communications problem, a source problem or another condition; interpret it using the documentation for that data path.
Check whether the query excludes particular quality states. A useful diagnostic comparison is the same raw-record request with and without its quality filter.
Do not relabel poor-quality readings as good to fill a chart. Preserve the status and explain the limitation wherever those records inform operational decisions.
Did collection stop, and was anything buffered?
Compare the affected interval with collector connection history, service events and tag-level diagnostics. Investigate shared timing: did the gap affect one tag, one source connection or everything handled by that collector?
Request evidence of whether collection stopped entirely or continued while the storage destination was unavailable.
Some collectors support local buffering and later forwarding. Recovery depends on the configured buffering behaviour, available capacity and whether records were actually collected. An outage does not, by itself, establish that backfill is possible.
Before attempting recovery, establish:
- Whether buffered records cover the missing interval.
- Whether forwarding is progressing or reporting errors.
- How duplicate and out-of-order records are handled.
- Whether recovery could affect ongoing collection.
Preserve the available buffer and diagnostics before making changes that might remove them.
Did storage accept the records?
If collection evidence shows records leaving the collector, inspect the destination's write results and raw archive.
Check for rejected writes, incorrect tag mapping, timestamp validation failures and storage availability. Review retention settings when the missing interval is older than the surrounding records being investigated.
Timestamp acceptance, duplicate handling and retention are product-specific. Confirm the applicable rules before resending data or changing archive settings.
Treat “sent” and “stored” as separate checkpoints. Close this part of the investigation only when the expected records can be retrieved from storage, with their timestamps and quality intact.
Could the query or dashboard be hiding records?
Compare the dashboard request with a raw-record query for the same tag and interval.
Inspect the tag identity, timezone conversion, start and end boundaries, quality filters and any batch or equipment filters. Then check whether the request asks for recorded values, calculated summaries or interpolated values.
Aggregation and interpolation can produce a different representation from the underlying stored records. Boundary inclusion and timezone handling also depend on the query interface.
If raw records exist but the chart omits them, focus on retrieval and presentation. Record the exact request that reproduces the discrepancy.
This distinction matters when historian values feed production reports. Our guide to what an OEE number hides explains why the meaning behind a reported figure deserves attention.
What does a worked investigation look like?
Illustrative example: A pressure trend appears empty between 10:00 and 10:20. The operator reports that the process remained running.
Start by asking what “empty” means. In this example, a raw query returns pressure records throughout the interval, but each carries a quality status excluded by the dashboard.
That establishes a display omission. It does not establish that the pressure readings were valid.
Next, inspect the detailed quality code and collector diagnostics. Suppose those records identify a source connection problem during the interval. The investigation then has two separate findings: the source connection lost usable readings, and the dashboard concealed the associated quality information.
The corrective work should address both findings. Restoring the connection addresses collection; making unavailable or uncertain data visible helps readers interpret the chart.
What common mistakes should you avoid?
Changing several settings together. Preserve the original configuration and test one explanation at a time, so the result remains interpretable.
Filling gaps before finding the cause. Keep estimated or substituted values distinguishable from measured records. Agree how reports should treat them.
Treating reconnection as proof of recovery. Verify both current collection and the historical interval in storage.
Assuming every tag should record at the same frequency. Check each tag's intended collection and storage rules before comparing record counts.
Leaving failure behaviour undefined. Specify how missing, stale and uncertain data should appear, who investigates it, and what recovery evidence is required. Put these decisions in the functional design specification.
What should you record before closing the issue?
Keep a short evidence record:
- Affected tags, interval and timezone.
- Expected recording behaviour.
- First boundary where the expected data was absent or unusable.
- Relevant quality codes and diagnostic evidence.
- Recovery performed and records still unavailable.
- Verification that current collection and display work as intended.
Define success before changing anything: the cause is evidenced, recoverable records are verified, and remaining gaps are visible and explained.
Related reading: setting historian deadbands without hiding events.
Frequently asked questions
Can missing historian data always be recovered?
Do not promise recovery until you have identified retained source records, collector buffers or another suitable record. Confirm that any recovered data preserves the required timestamps and quality information.
Does a flat trend prove the process was steady?
Treat it as a prompt to inspect the records. Check whether fresh readings support the line and whether the display has carried forward or interpolated a value.
Should missing intervals be filled for reporting?
Agree the reporting rule first. Show which values are measured, estimated or unavailable, and explain how each affects the result.
When should we ask for help?
Seek support when you cannot establish the failing boundary, recovery may overwrite records, or missing data affects operational decisions. Our plant data services can help define the investigation and its acceptance criteria.