Demand more of the model development process

[#1]

Laura Freeman conducts research that brings together cyber-physical systems, data science, artificial intelligence and machine learning.Dr. Laura Freeman, Director of the Intelligent Systems Lab at Virginia Tech's Hume Center for National Security and Technology, explains why it's critical to figure out how to use machine learning and AI algorithms effectively in a recent event on mission-critical advanced statistics for the defense community. You can watch the full on-demand event, including a keynote talk and panel discussion.

Here's an excerpt from the panel.

"I think that’s something we need to be demanding of our model development process. So I gave a few examples in my talk earlier about how we were interrogating models after the fact to understand what they were doing, as well as how we were structuring our model development process to all the situations that we’re going to need to use them in.

"So I think we need that framework concept and we do from model development to model use. And it’s going to have to depend on the type of model that we’re using, what the goal of the problem we’re trying to solve is, the data that’s available to us and the type of decision that’s being made. That process is key to me, but being able to run experiments on our models and really interrogate them, then make predictions, then go out and actually collect the data and show that we were successful in our prediction is going to be a key element.

"When we can do that reliably, we can understand how our models are making the decision, and then we understand that we have enough quality information to support the decision-making process."

Preview her comments from the panel conversation, which is available on demand:

[video]