By Pavel Konečný, CEO of Neuron soundware.
We are only just understanding the advantages that can come with edge computing.
Back in 2016, I visited CEBIT conference in Hannover. It was full of so called “smart” things which I did not find smart at all. This “smart” things hype included, in fact, many just “connected” devices that in most cases, delivered a single purpose, narrowly defined benefit to the user. A few examples I still remember:
- A pipe valve that allowed to monitor the position remotely (open/close)
- A gas volume measurement device that, if secretly installed into a gas tank, could identify a truck driver stealing fuel and
- An electric plug, which can be switched on and off via wi-fi.
However, there was one very special presentation at CEBIT that influenced my views on how AI might be delivered in the future. IBM presented a research project SyNAPSE – developing an AI chip “TrueNorth” that could deliver computing power equivalent to an ant brain, while consuming just 73mW of energy. The only clear disadvantage was that it cost about USD 1 million per piece at that time.
This example proved that bringing AI to the edge of the network will be possible. It was also obvious that within a few years the “Moore law” will drop the price. The question was how quick it will be and how many other similar solutions would emerge on the market? Already at that time, Neuron soundware started to pursue such IoT strategy – run AI algorithms at the edge of the network – and decided to develop own IoT edge devices with audio recording and AI processing capabilities.
A few months later, I created a graph which shows the relationship between energy consumption and intelligence as a function of computing power that a piece of HW can deliver:
- With a few mW, no intelligence could be achieved for a reasonable price at that time
- Smart phones consume several watts and provided enough computing for basic AI object recognition from images every second or so and
- Narrow AI, such as the capability to drive a car, would need HW with tens or a few hundreds of watts power consumption. The analysis cameras inputs about ten times per second required about 4 TFLOPS (4 trillion floating point operations per second (FLOPS, flops or flop/s) is a measure of computer performance). So, if translated to what we do in Neuron soundware, you want to use the same computing performance either to drive a car or to analyse sound of machines in order to detect an upcoming mechanical failure. Doing both would require computing power equivalent to the brain of an ant. And IBM made me see this power within a single ultra-low-energy-consuming chip coming.
Recent rise of edge computing
The edge computing capability was on a rise since then. I kept an eye on several other AI Hardware acceleration projects, too.
In 2017, Movidius Neural Compute Stick with less than 100 USD price provided 0.1 TFLOPS and about 0.5W power demand. It is designed to extend less computing capable boards such as Raspberry Pi providing about 10x computing power boost.
In 2018, Huawei introduced its Kirin 980 processor with 0.1W and almost 0.5 TFLOPS. Also, other vendors didn’t stay behind. Google announced their Edge TPU Units and Rockchip demonstrated RK3399 equipped with Neural Processing Unit. Both having performance about 3TFOLPS and cost just around 100 USD.
In 2019, specific microcomputers with hardware accelerators of AI technologies (specifically Neural Networks) become generally available for use. All key HW players have released edge optimised versions of the AI software stack, which further increases the performance. Generally available AI boards are, for example, Google’s Edge TPU is purpose-built ASIC design to run inference. Nvidia Jetson Nano brings 128 CUDA cores into action for less than 100 USD. ToyBrick RK3399 Pro is one of the first developer boards with Neural Processing Unit (it slightly outperforms even Nvidia Jetson).
This fast IoT technology advancement allowed us in Neuron soundware to develop nBox – the edge computing device that is capable not only to record hi-quality audio with up to 12 channels, but also deliver AI through edge computing. By edge computing, we mean run only few processes in the cloud or central platform and run majority of processes in the local platforms instead.
The importance of edge computing become obvious with Intel’s acquisition of Movidius for estimated 400 million USD and Mobileye, an autonomous car chip maker, for more than USD 15.3 billion. I was thrilled to watch online Tesla Motors’ presentation of their purposely built AI enhanced computer for their self-driving cars with 36 TFLOPS. That is enough computing to process more than 2000 high resolution images from the car cameras per second and Tesla claims it is sufficient performance to achieve autonomous driving.
Overall, I see four key advantages of edge computing:
1) Safety: All processed data can be stored locally with a tight control.
2) Speed: AI inference can process inputs in milliseconds, meaning minimal latency.
3) Efficiency: Embedded micro-computers are low power with affordable prices.
4) Offline: The AI algorithm is deployed in the field, where connectivity might be limited.
Advantages of edge computing over 5G
Are you asking why so much HW fuss and effort, why not just wait for 5G networks and leverage the abundant cloud computing power and infrastructure? Here’s a few ideas why such “waiting” might not be the best strategy.
- Imagine you sitting in a self-driving car when the car has lost 5G connectivity. The car will not only go blind, but it will literally lose its brain/decisioning power. Why risk this when computing capabilities required for the high-bandwidth and low latency communication might be practically for the same cost as an extra neural processing unit. In addition, the overall energy demand would be higher than for AI inference using specific hardware
- Mobile internet providers want to cash-out investment into development and deployment of 5G network. Although unlimited data plans might be technically possible, they might not be commercially available any time soon. For example, our nBox with 12 acoustic sensors can produce up to 1 TB of audio data per month. With the current price per GB in LTE, transferring this amount of data to cloud would cost a fortune and
- Finally, the network coverage will be primarily built in cities, leaving large parts of the country without 5G. In contrast, edge computing devices can be deployed immediately at right places with a clear one-off cost, which usually does not dramatically increase costs of the IoT solution.
Edge computing combined with AI will allow to process enormous amounts of data locally. The additional cost of hardware accelerators is marginal. The computing performance for neural networks does boost-up about 10x every year. This trend doesn’t seem to slow down as the data can be processed in parallel, hence outperform traditional CPU design.
The future is coming faster
Usage of edge computing in applications such as self-driving cars, facial recognition or predictive maintenance is just a beginning. We will have enough computing power to build truly independently operating machines soon. They will be able to move safely in cities, factories and be almost as competent in their work duties as humans. It is incredible that somebody envisioned this almost a century ago. In 2020, it will be 100 years since the word “ROBOT” was introduced in the science fiction play R.U.R by the Czech writer Karel Čapek. His vision of humanoid robots quickly spread over the world. In this drama, robots become self-aware and could gain emotions such as love. Seeing the pace of computer power increase and other IoT advancements I think that Čapek’s visions might become true much sooner than we think.
On June 17, we kicked off our already third significant interaction with the aerospace industry in our three year company history. The European Space Agency’s Business Incubation Centre (ESA BIC) in Prague has selected Neuron soundware among five new companies to join the current group of 16. These ESA’s centres provide the backing and support for innovative technology start-ups that work with space technologies, develop them further and seek their commercial use. In the real world, this means that we will be partaking a 12-month project during which we receive funding and mentorship. However, most importantly, we’ll be able to access ESA’s business units to work on some cool space stuff.
Needless to say the aerospace industry is a really exciting industry to be in. And it’s definitely not uncharted territory for Neuron soundware. By passing through the Airbus BizLab acceleration program in Hamburg in the first half of 2018, we gained our first aerospace experience, and through the realization of two proof-of-concept projects, we developed an initial set of capabilities that are key for the industry. Then, later in 2018, our technology made it among the top three companies in the Industry 4.0 category of the Innovationspreis der Deutschen Luftfahrt in Berlin, competing head to head with renowned companies like Fraunhofer or Premium AEROTEC. In 2019 and 2020, we are going to build on this success and start the next chapter of work in the aerospace industry.
What will the ESA BIC project mean for Neuron soundware’s business and capabilities?
- Improved IoT HW with an extended suite of certifications: We will continue to develop and adapt our HW (IoT devices and sensors), including attaining new industry-specific certifications.
- Better and even more accurate technology: We will work with ESA experts to improve and standardize our SW/HW development and testing processes and procedures.
- Commercial service development: We will get a chance to work on a commercial project, develop an MVP for a selected ESA business unit, which will hopefully result in a service provision agreement.
Our priority is to engage with the ESTEC Mechanical Data Lab, which provides the majority of ESA spacecraft pre-flight test including mechanical and physical vibration, acoustic and shock tests.
Read the official ESA BIC Prague’s press release that includes an infographic under this link.
AI World magazine (David Slouka – DS) recently interviewed CEO and co-founder of Neuron soundware (Pavel Konecny – PK). Here is the Part 1 about who they are, what they do, and how they do it.
DS: Please briefly introduce Neuron soundware – I know that you’re using sound for predictive maintenance of machines and prediction of machine failures. But how exactly does it work?
PK: We use artificial intelligence methods to teach neural networks to recognize standard machine sounds. We have a database of sounds and various failures that we record directly with clients. This database of records allows us to recognize some of the problems in a much more streamlined way. For example, with sounds from similar devices of the same character. Algorithms allow us to recognize ahead of time when an anomaly is going to happen – and what kind of defect it is. As computing power increases, we are able to squeeze the algorithm into a relatively small microcomputer. We have also recently developed a version that allows us to process the signal completely within the nBox, our core hardware unit. This means that going forward with all new installations, we will be able to process the entire signal on site and increase the number of channels twofold so that we do not have to move terabytes of data between the device and the cloud. It’s a reliable, sturdy solution, which is also cheaper in the long run.
DS: And how exactly does the process work between yourselves and the customer? Is there a piece of hardware that will record the sound and send it to you for processing?
PK: We use different types of microphones, the most common being the piezo contact microphone, which is connected to the nBox; the microphone signal consequently goes to the digitization platform in the box. Following that, we send the signal to the cloud for processing, or we can analyze the signal directly at our edge device.
DS: Do you use deep learning neural networks or something else?
PK: We use various algorithms, not just deep neural networks. It really depends on project to project and what requirements there are, as well as what the data situation is. There are instances where we have a lot of data, in which case we use a deep learning algorithm. When we have less data, which is more common, we use other techniques to solve the problem. What is not so common but what we in fact do very well is dealing with a severe lack of data. The technique we use to deal with these situation is what makes up the core of our technology, and we’re able to calibrate the algorithm for a particular machine very quickly. We have pre-trained models for different types of equipment and in the last step we calibrate them for the specific work of a particular machine.
DS: Detection itself runs how exactly? To compare it to a real world example, is it like listening to a hard drive and hearing it click, by which point I know it’s on its way out?
PK: That’s right, you could say that. We often mention a ‘story’ that in fact started it all in terms of our business and how we think about the work we do. My friend was driving in his car, when suddenly he heard a problem somewhere in the engine – the engine sounded different; the car was years old, and he clearly knew something is wrong. He went to the mechanic, but they couldn’t find where the real problem was. They looked at the dashboard, and said they didn’t see any obvious problem. So he left the mechanic, and kept driving. Lo and behold, a cylinder broke on the motorway 100 kilometers later and destroyed the entire engine. He told me back then, that he was very lucky to be 2 days before the end of the guarantee, otherwise this whole thing would have cost him an inordinate amount of money. We ask clients when they come up with a new case that we haven’t yet solved, if they have an experienced technician on site who can, thanks to extensive experience, detect the problem simply listening to the machines in question. Often, these engineers help us actually understand and label their data. They already know the sound of the device and know the problem: this is what often gives us the basic input for the building of the algorithm. Our basic philosophy deriving from this is, that if a person can learn it, so can a computer. In fact, there are acoustic phenomena where the algorithm is better than human – we have wider frequency sensors, which are more sensitive than the human ear and can detect even sounds we could never physically hear, which may be indicative of upcoming machine failure, for example.
DS: So examples of the applications of this technology beyond your friend’s car and the car industry are virtually any machines that make a sound?
PK: We do not install the hardware directly into the cars, even though we have also analyzed such data in the past. Instead, we are now focusing on datasets for machines that have a high added value for the client: that can for example mean various compressors in production, large diesel engines, cranes, turbines and so on. We are also performing more qualitative process-driven functions onsite, for example dealing with tasks like quality control or helping to organise workers and their shifts depending on what machines are running when. It’s a wide range of what we do and can do.
DS: So you work a lot in industrial production?
PK: Yes, it’s our main business; inside the factories, anything mechanical, really. We are now preparing to monitor warehouses, bearings, feeders that occasionally break etc. At the moment, businesses have a problem in logistics, and we can help solve that problem. In fact, our technology can be used for anything that has mechanical parts. The business can then leverage the findings strategically.
DS: What do you mean exactly re monitoring workers? Do you help organise them?
PK: We had such a project, yes. People control the function of products by hearing whether they work as they should: for example air conditioners or servomotors and so on. One stands at the end of the production line and checks to see if everything is working as it should; in other words, performing quality control – for example, with household appliances like fridges. This sometimes does not manifest itself as an explicit product malfunction (that the refrigerator would not freeze), but for example a weird noise the machine is giving off, without being broken as such. People will then return the product because it does not sound right, and they’re worried they’ve bought a broken fridge. As humans, we’re wired in a way that when something squeaks or creaks, you feel that it is broken because you have already experienced it somewhere else, and create this psychological association. Companies want to avoid these complaints and product returns; and this is where we step in. It’s an example of what we can do.
DS: So you’re actually replacing the people who fulfilled the role of your technology before you came along?
PK: Rather than replacing them, we support them. Of course, they are not able to listen to everything, they listen to a selection of samples when they’re walking next to the machine, or periodical checks. Companies are now interested in fully automating the process and really controlling everything and listening to everything. We therefore provide rather an advisory system, where we can identify a product where there is a high probability of a defect. Of course, a ‘real person’ can still check the machine. The second use case we are now testing in practice is when people are assembling something – we listen to how they mount it, whether they have done all the tasks they need to do with the product, and if they don’t, they might, for example, have to look again for the right number of screws. When a component is properly put into some machines, you can hear it, it makes a distinctive sound it – and if it doesn’t, it won’t be properly fitted. We’re able to detect this using the algorithm, which is useful for assembly.
DS: Is the acoustic monitoring continuous or does it turn on and off sometimes?
PK: Depends on the machine. For quality control it is continuous, for some devices we collect data once a minute for a few seconds; it depends.
DS: As far as your business customers are concerned, are they the big and medium business? Or generally smaller ones?
PK: I would say generally bigger companies, car makers or work for companies like Airbus. There are a lot of things with the big producers, energy or demand at chemical factories or refineries that are interesting use cases. I think that the whole industry, which is dealing with infrastructure digitization, is interested in this area. There are places where the sensor failure has already been dealt with; in those cases, we bring better algorithms. Then there are places where, thanks to the fact that the IoT world is cheaper, our solution can be installed on a much larger number of devices.
DS: So is it possible for critical energy infrastructure, say wind power plants?
PK: Yes, we have it in the tender, there is a lot of interest in this area. Last month, a request from a technology broker came from China. They have the power of 120 gigawatts of wind and hydroelectric power; even such big customers write to and care about us. Of course, it may take years, China is far away, but the market is enormous and we have calculated how many devices out of the 20 most common types can be monitored – how many are around the world. I had a study done by a market research company, and it is potentially a worldwide 65 billion euro turnover, which requires our type of predictive monitoring. Of course, this includes machines of a smaller size than a turbine in a power plant, such as air conditioning units or machine tools, but the number is high and we are preparing standardized solutions from those first installations. So far we offer it for the simpler machines types such as electric motors, pumps, compressors and the like. There, we are able to install the solution directly and start recording and predicting failure immediately. For more complicated or custom tasks, we collect specific datasets with the client. We have discussed the need for tunneling, for example. In custom projects, we listen to how the machines work. Iron smelters have also been interested; people hear that the metals are suddenly melting differently and they want to automate the process to shorten the melting process and save time and money. We work with clients who are interested in testing the technology and preparing it for sale at the same time. We are increasing the number of people in the sales department to be able to resell solutions from the first projects we prepare with clients.
DS: Interesting! And how many people do you currently have at your company?
PK: We have over 20 people in the team, some people are installing, designing hardware, some people writing software, training models, doing visualization of results, and some people in the machine learning team that develops artificial intelligence, and the rest is marketing, account management, project management, operations and finances.
DS: Are you going to expand?
PK: We are about to pick up the hiring tempo, we have a lot of open positions, it’s on our site for Startupjobs.
DS: And, provided you want to talk about it, do you have a problem filling capacity? Are you able to find enough people?
PK: We have a lot of candidates coming in; AI attracts a lot of people, but I think some of them suffer from a bit of an illusion about how it actually works in practice and then end up disappointed that they’re just pulling data into blackboxes that someone else has built. It’s a very detailed, pernickety work. Lots of small things to constantly do and think about. We don’t have a shortage of candidates, but there are not so many people out there who have really long-term experience or are able to take a scientific article and process it into a form of AI implementation. And the second thing, which is demanding, is that a lot of people are interested in working in the fintech sector, since they studied for example statistical models or management economics at university. Regarding the hiring as such, we have a testing role during the admission process, and some of the candidates give up and don’t complete it; we don’t have the clean, labelled data they might be used to, rather an endless stream of seemingly messy data you need to orientate yourself in it. And then there are other positions that people would be interested in and apply for and where candidates generally have very high demands – graphic designers or front-end developers, for example. But this is not ideal for our company because we prefer to outsource such things to someone who has these people at their disposal. For example, our mobile application – we just bought it on the market.