Pairlet Loss Formula Check [Help!]

help-me

Pairlet Loss

• Layer Setup in caffe
  
  • $batch = \{pair_1, pair_2,..., pair_n\}$ where $n$ is num_pairs
  • $pair = \{f_a, f_b\}$ so $batch\_size = 2 n$
• Loss Formula
  $$\mathcal{L}_{tuplet}=\frac1{n}\sum_{i=1}^nlog[1+\sum_{j\neq i}exp(f_j^bf_i^a-f_i^bf_i^a)+\sum_{i\neq j}exp(f_j^af_i^b-f_i^af_i^b)]$$
• Back Propagation
  Assume that
  $$\mathcal{T}_i=1+\sum_{j\neq i}exp(f_j^bf_i^a-f_i^bf_i^a)+\sum_{i\neq j}exp(f_j^af_i^b-f_i^af_i^b)$$
  then we can get
  $$\mathcal{L}_{pairlet}=\frac1{n}\sum_{i=1}^nlog(\mathcal{T}_i)$$
  and
  $$\frac {\partial \mathcal{L}_{pairlet}}{\partial \mathcal{T}_i} = \frac1n\sum_{i=1}^n\frac1{\mathcal{T}_i}$$
  we can extract those w.r.t $f_i^a$
  
  • if $k = i$ :
    $${\mathcal{T}_k}'={\mathcal{T}_i}'=\sum_{j\neq i}exp(f_j^bf_i^a-f_i^bf_i^a)+\sum_{i\neq j}exp(f_j^af_i^b-f_i^af_i^b)$$
  • else ($k\neq i$) :
    $${\mathcal{T}_k}'=exp(f_i^af_k^b-f_k^af_k^b)$$
    so
    $$\begin{split} \frac {\partial \mathcal{L}_{pairlet}}{\partial f_i^a} &= \frac1n\{\frac1{\mathcal{T}_i}[\sum_{j\neq i}exp(f_j^bf_i^a-f_i^bf_i^a)(f_j^b-f_i^b)+\sum_{i\neq j}exp(f_j^af_i^b-f_i^af_i^b)(-f_i^b)]\\ &+ \sum_{j\neq i}\frac1{\mathcal{T}_j}exp(f_i^af_j^b-f_j^af_j^b)(f_j^b)\} \end{split}$$
    in a similar way:
    $$\begin{split} \frac {\partial \mathcal{L}_{pairlet}}{\partial f_i^b} &= \frac1n\{\frac1{\mathcal{T}_i}[\sum_{j\neq i}exp(f_j^af_i^b-f_i^af_i^b)(f_j^a-f_i^a)+\sum_{i\neq j}exp(f_j^bf_i^a-f_i^bf_i^a)(-f_i^a)]\\ &+ \sum_{j\neq i}\frac1{\mathcal{T}_j}exp(f_i^bf_j^a-f_j^bf_j^a)(f_j^a)\} \end{split}$$