Multiple Instance Learning (MIL)

1 What are Channels in Convolutional Neural Network

Channels represent the depth of the 2D spectrogram. Channels R,G,B in photoshop are the same channels as the one here in CNN. For a 2D spectrogram that is in greyscale, the pixel information can only contain a value from 0 to 1 (normalized to floating point range) or 0 (black) to 255 (white) which is the raw pixel value that it can take.

2 Multiple Instance Learning

MIL is a form of weakly supervised learning where training instances are arranged in sets, called bags. These bags are labeled, in this case, drone or noise. These bags contain instances that are not labeled. A bag may be labeled a drone if there is at least one drone instance, or noise if all instances are noise. The probability of the bags being labelled a drone or noise is based on Bernoulli's principle.

Deep MIL model defines the probability of the bag having a drone signal as $\Theta (X)\in [0,1]$. The bag's probability must be permutation-invariant since we assume that instances are not ordered nor are they dependent on one another. This means that instance 1 then 2 then 3 is the same as instance 2 then 3 then 1, and will not change $\Theta (X)$.

In each instance, we are able to extract features. In different instances, different features are more prominent than the other. While being trained, the CNN model will try to guess which feature is more obvious in which instance. During training, the CNN learns which patterns (features) in each instance help to make the bag label correct (only thing that's given). For each instance xk, the CNN f outputs a feature vector hk. Let's say feature j in instance k has a higher activation. This means that "this instance k matches pattern j more strongly. Therefore, feature vector = activations of learned patterns for that instance. The model is trying to learn the network parameters such as filters and weights.

2.1 How does it work

2.1.1 Instance transformation, f

We take each instance $x1,x2,\dots ,xk$ in the bag $X$ and pass it to a neural network $f$ to extract a feature and output a feature vector, $hk=f(xk)$. It changes from $X=x1,x2,x3,\dots ,xk$ to $H=f(x1),f(x2),\dots ,f(xk)$, where $H$ is a set of feature vectors, one per instance.

2.1.2 A permutation-invariant aggregation function, σ

Currently we have $H$, which contains many feature vectors. However, bag-level decisions require one vector to represent the whole bag. Therefore, we will require a pooling function $\sigma$, that takes all $hk$ input and output a single vector $z$, that is meant to represent the whole bag.

2.1.2.1 Mean Pooling

Works by taking the average of all features across all instances in the bag. A vector $z$ is then returned. $z$ contains all the averaged features of all instances.

2.1.2.2 Max Pooling

For each feature dimension $j$, you take the max value across instances $k$. Do this for all features and form single vector $z$ containing all the highest activation features from their corresponding instances.

2.1.2.3 Attention Pooling

First is to compute a score for each instance. A small network takes all the $hk$ (feature vectors), and then outputs a $scor{e}_k$ (scalar). Each feature vector has a score. This score is like how important is instance $k$ for predicting the bag label, based on their particular feature vector $h$. A common way of score calculation is:

$$e_k=w^{\top }\tanh (V{h}_k^{\top })$$

$w$ and $V$ are learnable parameters. They are initialised as random, but learned along the way during training via backpropagation. $V$ learns how to collapse that projection into a scalar score. $w$ learns how to project hk into an attention-friendly space.

Normalize the scores with a softmax to get attention weights $a_k$:

$$a_k=\frac{\exp (e_k)}{{\sum }_{j=1}^n\exp (e_j)}$$

Ak in the formula is the fraction of attention placed on instance $k$. Attention can be seen as importance (weightage). Larger $a_k$ means it probably has more weightage on affecting the final bag decision.

$$z=\sum _{k=1}^n{a}_k{h}_k$$

Just to form the same final $z$ vector. If instance is important (large $a_k$), its feature vector hk has big influence on $z=\sigma (f(x1),\dots ,f(xk))$

2.1.3 A final Transformation, function g(z)

$$\Theta (X)=g(z)=g(\sigma (f(x1),\dots ,f(xk))$$

$g$ is also a learnable parameter that maps $z$ to bag-level prediction.