Can machine learning and operations research reinforce each other?

Can deep learning really do everything?

Opinions on the true potential of deep learning differ. Geoffrey Hinton, awarded for groundbreaking deep learning, isn’t entirely unbiased, but others, including Hinton’s deep learning collaborator Yoshua Bengio, want to imbue deep learning with elements of a domain still under the radar: operational research or analytical problem-solving. and decision making used in the management of organizations.

Machine learning and the variety of deep learning are practically household names now. There is a lot of hype around deep learning, as well as a growing number of applications that use it. However, its limitations are also increasingly understood. That is probably why Bengio turned his attention to operational research.

In 2020, Bengio and his collaborators examined recent attempts, from both the machine learning and operations research communities, to use machine learning to solve combinatorial optimization problems. They argue for further driving the integration of machine learning and combinatorial optimization and detail a methodology.

Until now, however, there has been no publicly visible renaissance of operations research and few commercial applications compared to machine learning.

Nikolaj van Omme and Funartech want to change that.

Operational research uses domain knowledge to optimize

While the birth of operations research (OR) is commonly identified as being during World War II, its mathematical roots can go back even further to the 1800s.

In OR, problems are broken down into basic components and then solved into defined steps through mathematical analysis. Van Omme calls himself both a mathematician and a computer scientist. After his postgraduate studies, he began to notice the similarity and complementarity between machine learning and OR. After failing to get the attention he sought to further explore this potential synergy, he launched Funartech in 2017 to realize it himself.

For Van Omme, there were several reasons why the combination of machine learning and OR seemed like a good idea. First, machine learning is data hungry and in the real world there are instances where there isn’t enough data to go by.

It is also a matter of philosophy: “If you only use data, you hope that your algorithms extract patterns from the data,” says Van Omme. ‘You hope to find some limitations, some knowledge from the data. But you’re not really sure if you’re going to do that.”

In OR, he added, knowledge can be modeled. “You can talk to the engineers and they can tell you what they do, what they think and how they work,” he explained. “You can turn this into mathematical equations so that you can have and use that knowledge. If you combine both data and domain knowledge, you will progress.”

OR is all about optimization and, according to Van Omme, using it can lead to 20% to 40% optimized results. Like Bengio, he referred to the traveling salesman problem (TSP) – a reference problem in computer science. In TSP, the goal is to find the optimal route to visit all the cities in a traveling salesman’s assigned district once.

If you approach the TSP with OR, you can create exact solutions for 100,000 cities, according to Van Omme. By using machine learning, on the other hand, it is best to solve the same problem with 100 cities for an exact solution. This is an order of magnitude of difference, so it begs the question: why isn’t OR more commonly used?

For Van Omme, the answer is multifaceted: “A few years ago, machine learning was considered a sub-field of OR, so I wouldn’t say OR isn’t applied, although people now tend to put machine learning on one side and OR on the other. place on the other side. others,” he said. “There are some areas where OK is really heavily used, for example transportation or manufacturing.”

However, machine learning was so successful in some areas that it overshadowed all other approaches, he explained.

3 ways to combine operations research and machine learning

  1. Van Omme is not out to bash machine learning. What he advocates is an approach that combines machine learning and OR to have the best of both worlds. Usually, Van Omme said, you first use machine learning so you get some estimates and then you use those estimates as input to your OK algorithm to optimize.
  2. Machine learning and OR can also be used in combination to help the other. Machine learning can be used to improve OR algorithms and OR can be used to improve machine learning algorithms. OR is primarily rule-based, and when the rules apply, it’s hard to beat, Van Omme noted.
  3. Build new algorithms. If you fundamentally understand the strengths and weaknesses of machine learning and OR, there are ways to combine the two so that the weaknesses of one are equalized by the strengths of the other. Van Omme cited neural networks as an example of this approach.

cons

OR is not without its problems and Van Omme acknowledges that. The problem, in his words, is that “usually the rules don’t apply. You don’t know exactly how to apply them. And there’s a chance that if you go one way or the other, you’ll get completely different results.”

This is aptly illustrated in one of Funartech’s most high-profile use cases: working with the Aisin Group, a major Japanese supplier of auto parts and systems and a Fortune Global 500 company. Aisin wanted to optimize the transport of parts between depots and warehouses.

This cannot be approached in “traditional” ways with one model that can solve the whole problem because it is a very complex problem on a large scale, Van Omme noted. After working on this for four months, Funartech was able to optimize by 53%. However, it turned out that they did not have the correct data for some parts of the problem.

So when Funartech tried to figure out whether their solution made sense or not, they quickly discovered that some of the estimates for the data they didn’t have were actually not very good. When the correct data was supplied, the optimization dropped to 30%.

“The thing is, our algorithms are so tuned to the instance that they stopped working when they gave us the right data,” he said. “They couldn’t produce anything. So we had to go back and simplify our approach a little bit. And because it was the end of the project, we didn’t want to invest as much time as we did.”

Scaling up operational research

Van Omme also explained that Funartech spends a lot of time with customers, aiming for a tailor-made approach for each problem. This seems like a blessing and a curse at the same time. While Van Omme said Funartech is working on developing a platform, it’s hard to imagine at this point how this service-oriented approach could scale.

Part of what has made the machine learning approach so successful is the fact that there are algorithms and platforms that people can use without having to develop everything from scratch. On the other hand, Van Omme pointed out that Funartech has a 100% success rate, while 85% of machine learning and 87% of data science projects fail.

But according to Van Omme, there is another, perhaps unexpected, obstacle that OR practitioners have to deal with: learning to get along with each other. The “no Ph.D. required to make this work” story has been integral to machine learning’s breakthrough into the mainstream. In OR, things aren’t done yet.

According to Van Omme, the fact that OR practitioners are highly skilled also means that they tend to be very stubborn. People skills, such as learning to listen and make compromises, are therefore essential.

All in all, OR – and the different ways it can be combined with machine learning – seems like a double-edged sword. It has the potential to produce highly optimized results, but at the moment it also looks brittle, labor-intensive and skill-intensive and difficult to apply.

But again, the same could probably be said about machine learning a few years ago. Perhaps the cross-fertilization of the two disciplines with techniques and lessons learned could help improve both.

The mission of VentureBeat is a digital city square for technical decision-makers to gain knowledge about transformative business technology and transactions. Learn more about membership.

Leave a Comment