Personal tools
You are here: Home Events ANC Workshop: Elliot Crowley, Chair: Oksana Sorokina

ANC Workshop: Elliot Crowley, Chair: Oksana Sorokina

— filed under:

What
  • ANC Workshop Talk
When Jul 09, 2019
from 11:00 AM to 12:00 PM
Where IF 4.31/4.33
Add event to calendar vCal
iCal

Elliot Crowley

Title:

BlockSwap: Fisher-guided Block Substitution for Network Compression

Abstract:

The desire to run neural networks on low-capacity edge devices has led to the development of a wealth of compression techniques. Moonshine (Crowley et al. 2018) is a simple and powerful example of this: one takes a large pre-trained network and substitutes each of its convolutional blocks with a selected cheap alternative block, then distills the resultant network with the original. However, not all blocks are created equally; for a required parameter budget there may exist a potent combination of many different cheap blocks. In this work, we find these by developing BlockSwap: an algorithm for choosing networks with interleaved block types by passing a single minibatch of training data through randomly initialised networks and gauging their Fisher potential. We show that block-wise cheapening yields more accurate networks than single block-type networks across a spectrum of parameter budgets.

 

Paper available below

BlockSwap: Fisher-guided Block Substitution for Network Compression