A Blog by Jonathan Low

 

May 8, 2019

Scientists Have Now Made Neural Nets 90 Percent Smaller But Just As Smart

Speed, power and greater scalability through application to smaller, more ubiquitous devices. JL

Avery Thompson reports in Popular Mechanics:

An artificial intelligence that’s only a tenth the size, without losing any computational ability could build AI that are smaller, faster, and just as smart as those that exist today. Our brains 'prune' old connections that aren’t useful anymore But most artificial neural networks are only pruned once, at the end of training. Pruning during training produced neural networks that were just as good, but pruned networks 90% smaller and more efficient. They needed less training time and were more accurate. These networks could be powerful and lightweight, so people could use them with smaller electronic devices.
A pair of MIT researchers discovered a way to construct an artificial intelligence that’s only about a tenth the size, without losing any computational ability. The breakthrough could allow other researchers to build AI that are smaller, faster, and just as smart as those that exist today.
When people talk about artificial intelligence, they’re mostly referring to a class of computer programs called artificial neural networks. These programs are designed to mimic how our own brains operate, making them very intelligent and creative. They can identify the contents of photos, defeat humans at abstract games of strategy, and even drive vehicles all by themselves.
At their core, the programs are made up of collections of ‘neurons,’ just like in our own brains. These neurons are connected to a random number of other neurons. Each individual neuron can only perform a handful of basic calculations, but with enough of them all connected together, the computational power of the network is essentially limitless.
The most important thing for a solid neural network is the connections between neurons. Good connections make a good network, but bad connections leave you with nothing but junk. The process of making those connections is called training, and it’s similar to what our own brains do when we learn something new.
The only difference? Our brains regularly trim old connections that aren’t useful anymore, in a process called ‘pruning.’ We prune old or disused connections all the time, but most artificial neural networks are only pruned once, right at the end of training.
So the MIT researchers decided to try something new: prune the network regularly during training. They found that this method produced neural networks that were just as good as networks trained using the standard method, but these pruned networks were around 90 percent smaller and much more efficient. They also needed less training time and were more accurate.
In the near future, researchers might use this pruning method to design even better neural networks. These networks could be powerful and lightweight, so people could use them with smaller electronic devices. And in time, we could have neural networks operating nearly everywhere.

14 comments:

Webclick® Digital Pvt. Ltd. said...

Thats great post !! I like ur every post they always give me some new knowledge.
Top 5 Ecommerce Web Designing Company | Top 5 SEO Company In India

Sartojiva said...

This is a nice Post. Thanks For Sharing me a informative Knowledge.

At Sartojiva, we offer the best Men Bespoke Tailoring service. Our experts tailor handmade suits which meet all your requirements. From aesthetic to fine appearance, we keep everything in mind before dispatching the final design.

Ratanpara Overseas said...

Thanks for sharing this useful information with us.
Kesar Mango Exporters, Mango Pulp Exporters

Kumar Magnet Industries said...

Kumar Magnet Industries is a well-known company engaged in manufacturing Magnetic Separator. Being one of the leading Triple Drum Type Permanent Magnetic Separator Manufacturers, we offer the best at competitive market prices.

Denajee Health Care Products said...

Are you in need of a facial cream that can suit different skin types? If yes, then Denajee Health Care Products have the right product for you. A fruit massage cream that can solve various skin troubles and is 100% herbal. Being one of the elite Fruit Massage Cream Manufacturers, we have the best quality available just for you. You can view our website to explore our complete range.

Diets & More said...

Weight loss is no less than a task; get the online consultation from Shreya Katyal at Diets & More. She is an experienced Nutritionist For Weight Loss In Dubai known for her customized diet charts. Call now or leave your queries from the website.

Swarna Fasteners said...

Keeping in mind the precise demand of the customer, we Swarna Fasteners manufactures and supply good quality fasteners at the most reasonable price. Being the most ingenious Hex Nut Manufacturers In Delhi, we provide prompt delivery and after sale services to all our customers. If you want to acquire this or want more details, be in touch.

Transformer Manufacturers In Pune said...

Trutech Products is the growing Transformer Manufacturers In Pune that have rich experience in the domain of designing and developing industry’s best solutions that ensure client satisfaction. Dial our number now and, speak with our experts to share your requirements.

webclick digital Pvt Ltd said...
This comment has been removed by the author.
Anonymous said...

Trousseau means a lot for a bride to be, therefore, you need to pack it properly. We know the task require a lot of efforts and plenty of time, so, we Grace The Packing Studio offers you the professional Trousseau Packing Services In Delhi at a pocket-friendly charge. To know more about our services, you can be in touch with our experts, who are here to answer your queries.

Give Me Bread said...

Thai Food Restaurants In Christchurch have best treat for you to satisfy your cravings. Order from Give Me Bread and be ready to try the best authentic Thai recipes which fill your mouth with flavours. Order online.

Exoskeleton said...

Commercial exoskeletons are wearable devices, which can be used to support strength and endurance, and to protect soldiers from stress injury. What are the requirements of the US Army Soldier Center and which companies are leading research and development in exoskeleton design?


Link - http://exoskeleton360.com/index.php/2021/03/19/commercial-consumer-exoskeletons/

Post a Comment