DS: Please briefly introduce Neuron soundware – I know that you’re using sound for predictive maintenance of machines and prediction of machine failures. But how exactly does it work?

PK: We use artificial intelligence methods to teach neural networks to recognize standard machine sounds. We have a database of sounds and various failures that we record directly with clients. This database of records allows us to recognize some of the problems in a much more streamlined way. For example, with sounds from similar devices of the same character. Algorithms allow us to recognize ahead of time when an anomaly is going to happen – and what kind of defect it is. As computing power increases, we are able to squeeze the algorithm into a relatively small microcomputer. We have also recently developed a version that allows us to process the signal completely within the nBox, our core hardware unit. This means that going forward with all new installations, we will be able to process the entire signal on site and increase the number of channels twofold so that we do not have to move terabytes of data between the device and the cloud. It’s a reliable, sturdy solution, which is also cheaper in the long run.

DS: And how exactly does the process work between yourselves and the customer? Is there a piece of hardware that will record the sound and send it to you for processing?

PK: We use different types of microphones, the most common being the piezo contact microphone, which is connected to the nBox; the microphone signal consequently goes to the digitization platform in the box. Following that, we send the signal to the cloud for processing, or we can analyze the signal directly at our edge device.

DS: Do you use deep learning neural networks or something else?

PK: We use various algorithms, not just deep neural networks. It really depends on project to project and what requirements there are, as well as what the data situation is. There are instances where we have a lot of data, in which case we use a deep learning algorithm. When we have less data, which is more common, we use other techniques to solve the problem. What is not so common but what we in fact do very well is dealing with a severe lack of data. The technique we use to deal with these situation is what makes up the core of our technology, and we’re able to calibrate the algorithm for a particular machine very quickly. We have pre-trained models for different types of equipment and in the last step we calibrate them for the specific work of a particular machine.

DS: Detection itself runs how exactly? To compare it to a real world example, is it like listening to a hard drive and hearing it click, by which point I know it’s on its way out?

PK: That’s right, you could say that. We often mention a ‘story’ that in fact started it all in terms of our business and how we think about the work we do. My friend was driving in his car, when suddenly he heard a problem somewhere in the engine – the engine sounded different; the car was years old, and he clearly knew something is wrong. He went to the mechanic, but they couldn’t find where the real problem was. They looked at the dashboard, and said they didn’t see any obvious problem. So he left the mechanic, and kept driving. Lo and behold, a cylinder broke on the motorway 100 kilometers later and destroyed the entire engine. He told me back then, that he was very lucky to be 2 days before the end of the guarantee, otherwise this whole thing would have cost him an inordinate amount of money. We ask clients when they come up with a new case that we haven’t yet solved, if they have an experienced technician on site who can, thanks to extensive experience, detect the problem simply listening to the machines in question. Often, these engineers help us actually understand and label their data. They already know the sound of the device and know the problem: this is what often gives us the basic input for the building of the algorithm. Our basic philosophy deriving from this is, that if a person can learn it, so can a computer. In fact, there are acoustic phenomena where the algorithm is better than human – we have wider frequency sensors, which are more sensitive than the human ear and can detect even sounds we could never physically hear, which may be indicative of upcoming machine failure, for example.

DS: So examples of the applications of this technology beyond your friend’s car and the car industry are virtually any machines that make a sound?

PK: We do not install the hardware directly into the cars, even though we have also analyzed such data in the past. Instead, we are now focusing on datasets for machines that have a high added value for the client: that can for example mean various compressors in production, large diesel engines, cranes, turbines and so on. We are also performing more qualitative process-driven functions onsite, for example dealing with tasks like quality control or helping to organise workers and their shifts depending on what machines are running when. It’s a wide range of what we do and can do.

DS: So you work a lot in industrial production?

PK: Yes, it’s our main business; inside the factories, anything mechanical, really. We are now preparing to monitor warehouses, bearings, feeders that occasionally break etc. At the moment, businesses have a problem in logistics, and we can help solve that problem. In fact, our technology can be used for anything that has mechanical parts. The business can then leverage the findings strategically.

DS: What do you mean exactly re monitoring workers? Do you help organise them?

PK: We had such a project, yes. People control the function of products by hearing whether they work as they should: for example air conditioners or servomotors and so on. One stands at the end of the production line and checks to see if everything is working as it should; in other words, performing quality control – for example, with household appliances like fridges. This sometimes does not manifest itself as an explicit product malfunction (that the refrigerator would not freeze), but for example a weird noise the machine is giving off, without being broken as such. People will then return the product because it does not sound right, and they’re worried they’ve bought a broken fridge. As humans, we’re wired in a way that when something squeaks or creaks, you feel that it is broken because you have already experienced it somewhere else, and create this psychological association. Companies want to avoid these complaints and product returns; and this is where we step in. It’s an example of what we can do.

DS: So you’re actually replacing the people who fulfilled the role of your technology before you came along?

PK: Rather than replacing them, we support them. Of course, they are not able to listen to everything, they listen to a selection of samples when they’re walking next to the machine, or periodical checks. Companies are now interested in fully automating the process and really controlling everything and listening to everything. We therefore provide rather an advisory system, where we can identify a product where there is a high probability of a defect. Of course, a ‘real person’ can still check the machine. The second use case we are now testing in practice is when people are assembling something – we listen to how they mount it, whether they have done all the tasks they need to do with the product, and if they don’t, they might, for example, have to look again for the right number of screws. When a component is properly put into some machines, you can hear it, it makes a distinctive sound it  – and if it doesn’t, it won’t be properly fitted. We’re able to detect this using the algorithm, which is useful for assembly.

DS: Is the acoustic monitoring continuous or does it turn on and off sometimes?

PK: Depends on the machine. For quality control it is continuous, for some devices we collect data once a minute for a few seconds; it depends.

DS: As far as your business customers are concerned, are they the big and medium business? Or generally smaller ones?

PK: I would say generally bigger companies, car makers or work for companies like Airbus. There are a lot of things with the big producers, energy or demand at chemical factories or refineries that are interesting use cases. I think that the whole industry, which is dealing with infrastructure digitization, is interested in this area. There are places where the sensor failure has already been dealt with; in those cases, we bring better algorithms. Then there are places where, thanks to the fact that the IoT world is cheaper, our solution can be installed on a much larger number of devices.

DS: So is it possible for critical energy infrastructure, say wind power plants?

PK: Yes, we have it in the tender, there is a lot of interest in this area. Last month, a request from a technology broker came from China. They have the power of 120 gigawatts of wind and hydroelectric power; even such big customers write to and care about us. Of course, it may take years, China is far away, but the market is enormous and we have calculated how many devices out of the 20 most common types can be monitored – how many are around the world. I had a study done by a market research company, and it is potentially a worldwide 65 billion euro turnover, which requires our type of predictive monitoring. Of course, this includes machines of a smaller size than a turbine in a power plant, such as air conditioning units or machine tools, but the number is high and we are preparing standardized solutions from those first installations. So far we offer it for the simpler machines types such as electric motors, pumps, compressors and the like. There, we are able to install the solution directly and start recording and predicting failure immediately. For more complicated or custom tasks, we collect specific datasets with the client. We have discussed the need for tunneling, for example. In custom projects, we listen to how the machines work. Iron smelters have also been interested; people hear that the metals are suddenly melting differently and they want to automate the process to shorten the melting process and save time and money. We work with clients who are interested in testing the technology and preparing it for sale at the same time. We are increasing the number of people in the sales department to be able to resell solutions from the first projects we prepare with clients.

DS: Interesting! And how many people do you currently have at your company?

PK: We have over 20 people in the team, some people are installing, designing hardware, some people writing software, training models, doing visualization of results, and some people in the machine learning team that develops artificial intelligence, and the rest is marketing, account management, project management, operations and finances.

DS: Are you going to expand?

PK: We are about to pick up the hiring tempo, we have a lot of open positions, it’s on our site for Startupjobs.

DS: And, provided you want to talk about it, do you have a problem filling capacity? Are you able to find enough people?

PK: We have a lot of candidates coming in; AI attracts a lot of people, but I think some of them suffer from a bit of an illusion about how it actually works in practice and then end up disappointed that they’re just pulling data into blackboxes that someone else has built. It’s a very detailed, pernickety work. Lots of small things to constantly do and think about. We don’t have a shortage of candidates, but there are not so many people out there who have really long-term experience or are able to take a scientific article and process it into a form of AI implementation. And the second thing, which is demanding, is that a lot of people are interested in working in the fintech sector, since they studied for example statistical models or management economics at university. Regarding the hiring as such, we have a testing role during the admission process, and some of the candidates give up and don’t complete it; we don’t have the clean, labelled data they might be used to, rather an endless stream of seemingly messy data you need to orientate yourself in it. And then there are other positions that people would be interested in and apply for and where candidates generally have very high demands – graphic designers or front-end developers, for example. But this is not ideal for our company because we prefer to outsource such things to someone who has these people at their disposal. For example, our mobile application – we just bought it on the market.