Back to all posts
Plant Data7 min read·November 8, 2026

How to Clean Plant Data Before Building Dashboards

TB
ThynkBlox Automation Team
Automation Engineering

To clean plant data before building a dashboard, check six things in order: tag names, engineering units, timestamps, missing values, outliers and quality flags. Write down every rule you apply, and apply it to a copy so the raw data stays as it was. A dashboard built on unchecked data puts errors in front of more people, and they look more official on a chart. If every cleaning step is written down and the original values are kept, any figure on the screen can be traced back to its source.

Why clean the data before you build charts?

A chart can't tell you whether the number behind it means what its label says. If a tag was renamed, a unit changed or a clock drifted, the chart will still draw a neat line, and people will make decisions from it.

Cleaning first also changes the dashboard design. You may find that a measurement you planned to show has long gaps, or that two lines record the same thing in different units. It is much cheaper to find that out before screens, calculations and reports depend on the data.

Think of cleaning as a review of what each value means. It isn't about making trends look smooth. The aim is data whose meaning, time and reliability you can state.

What should you check first: tag names or values?

Start with tag names, because every later check depends on knowing what each tag measures.

Build a tag register with one row per measurement:

  • The tag name in each source system (controller, SCADA, historian).
  • A plain-language description and the physical location of the measurement.
  • The equipment and line it belongs to.
  • The engineering unit and expected range.
  • Who owns the measurement and who approves changes.

Look for duplicates (one measurement under two names), reused names (one name that pointed at different instruments at different times) and tags without a clear owner. Where nobody can confirm what a tag measures, leave it off the dashboard until someone can.

If tags come from an existing controller through a gateway or collector, confirm the addressing and scaling at the source. The guide to collecting data from an existing PLC covers that path.

How do you check units and timestamps?

Units. Confirm the unit at each stage: instrument, controller scaling, historian configuration and dashboard label. Typical problems include a value stored in one unit and labelled in another, raw counts that were never scaled, and similar lines configured differently. Agree one display unit for each type of measurement and record any conversions. See standardising engineering units in plant data for a fuller method.

Timestamps. Find out where each timestamp is created (at the device, the controller, the collector or the historian) and which time zone or offset it uses. Check daylight-saving behaviour, clock synchronisation and whether buffered data arrives late with its original time. Events from different systems can only be compared once their clocks agree. The article on aligning timestamps across plant data systems explains how to check this.

How should missing values be handled?

First work out why the values are missing, then decide what to do about it. A gap can mean a network outage, a collector restart, a stopped machine, or a deadband that is working as intended and only records changes.

Record a treatment rule for each kind of gap:

  • Machine stopped: show it as a stop, not as zero production or a flat line.
  • Communication lost: mark the interval as no data. Don't draw a line across it.
  • Change-based recording: carry the last value forward only if the historian is configured to record on change and the rule is documented.

Don't fill gaps with averages or interpolated values on a dashboard unless an approved method allows it, and label any filled values. If gaps keep coming back, look into the cause as well. Diagnosing gaps in historian data is a good place to start.

What counts as an outlier, and should you remove it?

An outlier is a value outside what the process and instrument can plausibly produce. An unusual value is not automatically an outlier. A sharp spike may be a real event that operators need to see.

Use physical limits to separate the two: the instrument range, the process limits and known rates of change. A reading above the instrument's range, or a tank level that rises faster than the inlet could fill it, points to a measurement or data problem. A value that is high but possible needs to be checked with the process owner. Don't remove it just because it spoils the chart.

Flag outliers rather than deleting them. Keep the original value, record the rule that flagged it, and state whether calculations include or exclude it. If a reading slowly moves away from reality without ever going out of range, outlier rules won't catch it. See detecting sensor drift in plant data.

What do bad quality flags mean for a dashboard?

Many controllers, servers and historians store a quality status alongside each value, such as good, uncertain or bad. The exact codes depend on the system and protocol. Check the vendor documentation for the codes your systems use.

Make sure the quality status survives each step of the data path. A common problem is a collector or export that keeps the value but drops the flag, so a bad reading arrives looking normal. Decide how each status appears on the dashboard. For example, bad values could be hidden behind a text label and uncertain values marked as such, with both rules written down.

Data cleaning checklist

CheckWhat to look forTypical treatmentRecord
Tag namesDuplicates, reused names, unclear meaningMap to one register entry; hold unconfirmed tagsTag register with owner
UnitsMismatched labels, unscaled counts, mixed unitsAgree a display unit; document conversionsConversion rules
TimestampsTime zone, daylight saving, clock offset, late arrivalsNormalise to one time basis; flag unsynchronised sourcesTime basis per source
Missing valuesOutages, restarts, stops, change-based recordingClassify the gap; show no data rather than invented valuesGap treatment rules
OutliersValues beyond instrument or physical limitsFlag; keep the original; state calculation treatmentLimits and their approver
Quality flagsBad or uncertain status, flags dropped in transitCarry the status through; label on screenStatus display rules
Raw dataOverwritten or edited originalsClean a copy; keep the raw data read-onlyLocation of raw and cleaned sets

What does this look like in practice?

Illustrative example: a plant wants a dashboard showing hourly energy use per line. Every name and number below is invented to show the method.

The team pulls a sample week and works through the checklist:

FindingCheckAction taken
Line 2 meter appears under two tag names after a panel changeTag namesBoth mapped to one register entry, with the change date recorded
Line 1 reports kWh, Line 3 reports WhUnitsLine 3 converted to kWh; conversion documented
Line 3 collector timestamps are one hour ahead after a clock changeTimestampsOffset corrected in the cleaned copy; clock synchronisation raised with IT
A six-hour gap on Line 2 during a network faultMissing valuesShown as no data, not as zero use
One reading of 9,999 kWh in a single hourOutliersFlagged as beyond the meter's possible range; excluded from totals with a note
Several Line 1 values marked uncertain during a meter faultQuality flagsLabelled uncertain on the chart; kept in totals with a visible note

The raw week stays unchanged in its own location. The cleaned copy, the register and the rule list go to the dashboard builder, so each figure can be traced back to its source.

What mistakes should you avoid?

Cleaning inside the dashboard tool only. Rules hidden in chart settings are hard to review and easy to lose. Keep them in a written rule list.

Overwriting raw data. Once the original is gone, you can't check a rule or reverse it.

Treating zero and no data as the same. A stopped meter and a failed connection should look different on screen.

Smoothing away real events. Filtering spikes before checking them with operators can hide the problems the dashboard exists to show.

Cleaning once and stopping. New tags, instrument replacements and clock changes bring new problems. Re-run the checklist whenever the data source changes.

When does this not apply in full?

A quick one-off chart for an engineer's own investigation may not need a full register. Keep the raw data and note what you changed. Any dashboard that others will rely on, especially for shift reviews, reporting or decisions about equipment, needs the full process. Where data feeds safety-related functions, involve a qualified assessor before changing how it is handled.

Frequently asked questions

Who should own the cleaning rules?

Each measurement needs an owner who can confirm what it means and approve its limits. That is often someone in process engineering or maintenance. The dashboard builder applies the rules but shouldn't make them up.

Should cleaned data be written back to the historian?

Generally, keep the historian as the raw record and store cleaned or derived values separately, clearly labelled. Agree any exception with the historian owner.

How often should the checklist be re-run?

Re-run it whenever tags, instruments, collectors or clocks change, and review the rule list at an agreed interval set with the measurement owners.

Can cleaning be automated?

Rules that are well defined, such as unit conversions, range flags and gap labels, can be automated once they are documented and tested. Keep a person responsible for reviewing flagged values and approving new rules.

If you are weighing a live model of your plant, see what a digital twin is in manufacturing.

Next step

Want to apply this to your plant?

Every project starts with a fixed-fee assessment, and you can stop after the report.