ARM Project Trillium

A message made by ARM on February 13, 2018 said that the announced Project Trillium opens a new era in the field of inference technologies, which are the second component of machine learning (ML). Recall that the first is the actual training of neural networks.

The novelty lies in the placement of output procedures not on powerful servers located somewhere in data centers, but on end devices (Edge Machine Learning), on ordinary gadgets. As a result, Project Trillium will allow you to create a wide variety of smart devices - cars, mobile gadgets, etc., create qualitatively new approaches to IoT and much more, which can be called smart objects. Without any exaggeration, we can say that Project Trillium is not inferior in its importance to such innovations as a smartphone and virtual reality. This is because, as one of Ray Bradbury's Dandelion Wine characters said: "Clearly like an orange."

In order to make sure that the novelty is radical, it is enough to compare programming as a means of controlling a computer to which we have all been accustomed for a long time, with ML. Both are nothing more than a way to give the computer information about what it needs to do. When programming, which actually follows from the name of this method, the entire sequence of actions is set, from the first command to the last. The adequacy of programming has been preserved for more than sixty years of computer history, but sooner or later, as computers collide with the outside world, it turns out to be difficult or impossible to foresee all the variability of the outside world, and then the limit for software computer control is revealed and approaches called artificial intelligence or AI come to the rescue . More precisely, it would be worth talking about computational intelligence, that is, intelligent computers, but AI has already become entrenched in mass consciousness.

To get practical AI results, you need to go through four stages:

• Create an unearthed neural network, a mathematical model of the machine primitive brain.
• Using, for example, deep learning frameworks and huge amounts of data, train it for certain types of actions (filtering images, working with natural languages, etc.).
• Separate the trained neural network and transfer it to a computer where this network will work, giving useful information.
• Perform the required outputs on the final device.

We see that everything is simple - the development of the program is replaced by network training, and not the program is loaded into the executive computer, but a trained network. In its logic, this process resembles the development of software for embedded systems, where first on a powerful universal computer using a cross-system, software for a low-power controller is developed, and then the finished program is loaded into it.

In the learning-inference link, there are completely different requirements for those systems where neural networks are trained, and for final systems where trained networks can work. Until now, the focus has been on the first part, since the real opportunity for training networks opened with the advent of GPU. The world does not stand still, various alternatives are being created, against this background, specialized processor technologies for implementing leads on end devices remained out of sight. This, as it is not difficult to guess the gigantic niche in terms of its volumes, is filled by Project Trillium. ARM is a large phabless company, it offers its partners Project Trillium in the form of intellectual property (IP). There are many such partners, they have great industrial potential, so there is hope for the emergence of practical results of Project Trillium soon and in a wide variety of species.

The new ARM project is named for a flower family with three Trillium petals

The project is not accidentally named for a color family with three petals, it consists of three interconnected components - two ARM ML and ARM OD processors and an ARM NN SDK tool set . This trinity, combined with traditional ARM processors (Cortex, Neon, DynamIQ), GPU ARM Mali, as well as FPGA and DSP, forms a platform on which neural networks created using the frameworks TensorFlow, Caffe, Caffe2, Mxnet, AndroidNNAPI and others can work.

The speed of execution depends on which processors the neural network works on. It is lower if the processor is universal, higher if the processor is graphical, and maximum on a special ML processor in combination with OD.

ARM Machine Learning Processor

The ARM ML processor is designed from scratch solely to accelerate the ML execution phase. This specialized (most likely not von Neumann) processor consists of two parts: one serves to perform a fixed set of standard convolutional operations, and the second programmable supports the remaining operations (non-convolution layer), including those that may be required during operation, which provides protection against future changes (futureproofing). The availability of local memory allows you to speed up your work. The ARM ML performance is 4.6 TOPs (10 in 12 degrees of operations per second), in terms of watts 3 TOPs/watt. The controller supports interaction with the most popular frameworks.

OD Processor (Object Detection)

The ARM Object Detection processor is highly specialized and serves exclusively for the selection of certain objects. Its work is supported by a pre-prepared set of metadata. Using this data, you can select objects from individual fragments, for example, a person by hand or head, and a car by wheel or headlight.

Neural Network SDK

ARM has developed an instrumental set (SDK) that allows you to effectively broadcast trained neural networks created using the TensorFlow, Caffe (1 and 2), MXNet, and can be integrated with Android NNAPI. As of February 2018, the NN SDK supports Cortex A and M, as well as the Mali GPU. The ML processor will receive support later this year. The Neural Network SDK is distributed freely under the MIT License^[1]

Manufacturing partners will receive design documentation for the ARM ML processor in mid-2018, usually it takes about 9 months to set up production and produce the first products. Therefore, the first smart gadgets can be expected by the spring of 2019.