Video Analytics Trends Part 2: AI Assisted Reporting

Graeme Woods

Global Business Analyst

Last time, I wrote about the trend towards combining structured and unstructured data and how tapping into unstructured data such as video, freeform text or audio and combining it with other data can produce new insights.

This posting is about a trend that I have identified for AI to automatically extract actionable insights in real time, from multiple data sources (including ERP and video surveillance) using causal methods and communicate this insight and the justification for it. The following diagram summarises this:

By actionable insights, I mean a supported observation that can be used to improve an organisational result. For example, the AI identifies that there is often disorderly behavior around closing time in a mall but that by putting two guards outside a certain area, that can be prevented, that is an actionable insight.

Traditionally data analysis is the domain of data scientists. Data scientists are usually busy people because there is more data, and demand for insights than there is time to build models.

There are a lot of well-proven tools and techniques for analysing structured data. However, building models from unstructured data is more specialised, and may rely on very extensive data sets. In the same way that most aircraft manufacturers buy their jet engines from Rolls Royce rather than trying to make their own from scratch, most organisations will prefer video analytics from a company with specialist skills such as Scylla rather than trying to build their own.

After an organisation selects a proven solution for unstructured data, there is still the challenge of combining structured and unstructured data types together to identify actionable insights. One approach is to use a new area of machine learning called causal methods. This identifies the relationship between cause and effect. For example, causal analysis can tell us whether an advertising campaign increased sales, or if the increase would have happened anyway.

This analysis doesn’t just use correlational pattern recognition but understands the nexus between causes and effects. In the example above of disorderly conduct in the shopping mall at closing time, the solution develops an underlying understanding of security dynamics in a mall and the relationship between staff placement and availability and disorderly activity.

Usable causal estimation must be able to operate without having the opportunity for a counterfactual – that is, what would have happened in a world with a different set of initial conditions. Causal effect estimation methods are being extended to include unstructured data such as video and text, and this is the trend that I see for video analytics.

Video analytics will combine structured and unstructured data with AI-powered insights to connect to different data sources and automatically uncover causal relationships between inputs (such as surveillance data or marketing campaigns) and outputs (such as security outcomes or sales). These tools will incorporate underlying knowledge so that they will be able to make intelligent recommendations, not just observations. The AI will include natural language generation to be able to communicate the insight and rationale in a way that humans can easily understand.

For example, in retail, the AI will analyse traffic flow in a store and correlate this with advertising campaigns, whilst also understanding how advertising may be correlated to shoplifting. It may recommend to move a display to allow free flow of customer traffic, call in more staff from another store to secure items in another part of the store.

Due to the complexity of AI and the need to incorporate more real-world experience than is available in a single organisation, these tools will be supplied by specialist companies with deep expertise in specific domains such as security, retail analytics or manufacturing. I know that Scylla will extend its leadership in video analytics to include advanced AI for automatic identification of insights using causal methods.