Advanced Data Mining and Agent-based Modeling

Many traditional techniques in data mining create “black-box” models and though they can extract patterns present in datasets they do not help explain why those patterns are present. Many techniques of complex systems (like decision tree induction and Bayesian modeling) are “white-box” and can provide us with new insights into pattern recognition and data mining with respect to large-scale datasets. Moreover, we can use agent-based models to explore those patterns.

For instance, if we extract a behavioral rule from a dataset of product-purchase data (like that product A is often bought at the same time as product B) then we can intervene in a retail design to enhance this behavior (by moving products A and B closer together). Moreover, once we start to establish these rules, we can embed them in agents representing consumers and use these agents to test out new retail designs before they ever have to be physically constructed.

Given the significant amount of resources that retailers invest in examining innovative retail situations, the creation of a virtual retail environment that could be effectively used to explore retail strategies would be extremely valuable. Explorations of how to best enhance data mining using advanced complexity techniques, and of how to use data mining results in agent-based models holds the promise for a significant avenue of research.