learning note : Understanding Pedestrian Behaviors from Stationary Crowd Groups

OPTIMAL

---

原文 ：Understanding Pedestrian Behaviors From Stationary Crowd Groups

---

文章主要有两点创新。

一是建立了一种模型 general energy map $\mathcal{M}(x, \Theta)$, 代表行人会走到这个地方的难易程度， 或者说一个行人会出现在场景中一个位置的概率。
$x$ 表示这个位置； $\Theta = [\theta_1, \theta_2, \theta_3, \theta_4]$ 是需要从data中学习到的参数.

模型主要由3个部分组成 :

• scene layout ：场景中的障碍物的影响 $f_{SL}(x, \theta_1)$
• Moving pedestrian : 场景中的行人的影响 $f_{MP}(x, \theta_2)$
• stationary crowd groups : 场景中的静态人群的影响 $f_{SG}(x, \theta_3, \theta_4)$ —— 创新点

通过场景中的K个观察点（observation of $x$）建立似然函数使用梯度下降法更新参数 $\Theta$

应用主要在于

• 给定起点和终点，预测行人的行动路线 ；
• 给定一个行人走了一半的路线，预测终点 ；
• 建立行人的性格模型，分为 侵略型，保守型，反常型三类。由于$\mathcal{M}$建立的基础是 行人选择最方便和有效的路径，因此那些与模型匹配的很好的人为侵略型，选择尽量远离障碍物的人为保守型，极度不匹配的人为反常类型。

静态人群的检测方法来源于参考文章 [39],笔记最后detecte stationary crowd group region的链接即是。
路径生成的方法主要来源于参考的两篇文章标号[17, 32]，在笔记最后generate pedestrian walking path的两篇链接即是。

第二个创新点是建立了一个新的pedestrian walking path dataset，在一个video中标注了约12000人行为信息。

---

contribution

1. A novel model is proposed form pedestrian behavior modeling by including stationary crowd groups as a key component.
   Through inference based on the interactions between stationary crowd groups and pedestrians, our model can be used to investigate pedestrian behaviors.

2. A large pedestrian walking path dataset is built.
   The walking routes of more than 12, 000 pedestrians from a one-hour crowd video are annotated.

3. The effectiveness of the proposed model is demonstrated by multiple applications on the proposed dataset. Including

   • pedestrian walking path prediction
   • pedestrian destination prediction
   • pedestrian personality estimation and classification
   • abnormal event detection

4. compare with existing agent based models, three points :

   • the factor of stationary crowd groups is introduced for the first time to model pedestrian behavior.
   • The proposed model can be dynamically update with time to adapt the change of stationary crowd group.
   • model personality, which is a key factor that makes each individual behave differently.

Pedestrian Behavior Modeling

A pedestrian usually selects the most convenient and effcient path for reaching the destination ;
propose a general scene energy map $\mathcal{M}$ to model the traveling diffculty of enery location of the scene .
$\mathcal{M}$ is also a probability map shows the probability of pedestrian appearing at each location.

Personalized energy maps $\mathcal{M}_P$ are generated based on the general energy map $\mathcal{M}$ and a personality parameter P;
$\mathcal{M}_P$ can be view as a different pedestrians' interpretations of the general map $\mathcal{M}$.

General energy map modeling

$\mathcal{M}$ is modeled with three channel caculated based on

• $f_{SL}$ : Scene Layout
• $f_{MP}$ : Moving Pedestrians
• $f_{SG}$ : Stationary Groups

$$\mathcal{M}(x; \Theta) = f_{SL}(x; \theta_1) f_{MP}(x; \theta_2) f_{SG}(x; \theta_3, \theta_4)$$

$\Theta = [\theta_1, \theta_2, \theta_3, \theta_4]^T$ are weight paramters for different terms

Scene layout factor

$$f_{SL}(x; \theta_1) = \exp(-\frac {\theta_1} {d_1(x, SL)})$$

• $SL$ : a set of locations occuied by scene obstacles which are unreachables
• $d_1(x, SL) = \min_{y\in SL}\|x - y\|_2^2$
  measures the distance from the current location $x$ to its nearest scene obstacle location $y$
• $\theta_1$ : a parameter indicating the influence bandwidth(importance).

Influence of moving pedestrians

$$f_{MP}(x; \theta_2) = \exp(-\sum_{i = 1}^m \frac {\theta_2} {d_2(x, MP_i)})$$

• $MP_i (i \in [1, m])$ : the $i$th moving pedestrian
• $x_t^{MP_i}$ : the spatical location of $MP_i$ at current time $t$
• $d_2(x, MP_i) = (\|x - x_t^{MP_i}\| + \|x - x_{t+1}^{MP_i}\|)^2 - (\|x_t^{MP_i} - x_{t+1}^{MP_i})^2$
  measures the distance from the current location $x$ to the moving pedestrian $MP_i$
• $\theta_2$ : the influence bandwidth of the moving pedestrian term.

Influence of stationary crowd groups

$$f_{SG}(x; \theta_3, \theta_4) = \exp(- \sum_{i=1}^n\frac {\theta_3}{d_3(x, SG_i) + \theta_4d_4(SG_i)})$$

• $SG_i(i \in [1, n])$ : the $i$th stationary crowd group region automatically dected
• $d_3(x, SG_i) = \min_{y \in SG_i}\|x - y \|_2^2$
  measures the distance from $x$ to the stationary crowd group region $SG_i$
• $\theta_3$ : the influence bandwidth of the stationary crowd group term
• $d_4(SG_i) \in (0, +\infty)$ : calculated as the average distance among group members
  measure the sparsity of stationary crowd group region $SG_i$
• $\theta_4$ : control the influence of group sparsity on estimation result

Personalized energy map modeling

$$\mathcal{M}_p = \exp(P \times \ln\mathcal{M}(x; \Theta))$$

$P$ : personality parameter, equivalent to the influence bandwidth of the terms $(\theta_1, \theta_2, \theta_3)$

• large $P$ means the energy values are small at locations near obstacles and stationary crowd group.
• small $P$ means that the pedestrian is walking aggressively and cares less about abstacles

Path generation

$$\widehat{T} = f_{FM}(\mathcal{M}(\mathcal{M}_P), x_s, x_d)$$

$\widehat{T}$ : the most effcient and probable route from $x_s$ to $x_d$ according to the current energy map $\mathcal{M}$ or $\mathcal{M}_p$

Model learning

by dividing a marginalization term, $Z(\Theta)$, the energy map $\mathcal{M}(x;\Theta)$ can be transformed to a probility distribution :

$$p(x; \Theta) = \frac 1 {Z(\Theta)} \mathcal{M}(x;\Theta)$$

where $Z(\Theta) = \int\mathcal{M}(x;\Theta)dx$

Gaven $X = \{x_1, \dots, x_k, \dots, x_K\}$ as $K$ independent observation of $x$, the likelihood of these observation is :

$$p(X; \Theta) = \sum_{k=1}^K \frac 1 {Z(\Theta)} \mathcal{M}(x_k; \Theta)$$

Parameter $\Theta$ can be then be optimized as

$$\widehat{\Theta} = \arg \max \log_p(X; \Theta)$$

Gradient descent is used for updating parameters

$$\Theta_{new} = \Theta_{old} + \eta \frac {\partial\log p(X;\Theta_{old})} {\partial\Theta_{old}}$$

Pedestrian walking route dataset

Dataset details

1. much longer than any existing one with ground truth on tracking
2. a crowd surveillance dataset which is difficult and challenging for vison task.
3. The dataset is well annotated

Statiscal analysis of the annotated data

• (a) The percentage of stationary pedestrians
• (b) Average walking path length
• (c) Average traveling time

The strong correlations between (a) and (b)-(c) indicate that stationary crowd is a key factor that decreases traffic effcienty.

Learning Result

A pedestrian is not sensitive to scene obstacles

Apedestrian might prefer to adjust walking speed rather than change predecided walking direction to avoid close contact with other moving pedestrians.

When stationary crowds emerge in front of a pedestrian, he/she has to change his/her walking route to bypass the stationary crowds.

Applications

Prediction on pedestrian walking paths

over cost value $\eta$ is define as

$$\eta = \frac {\mathcal{C}(T_O, \mathcal{M}) - \mathcal{C}(\widehat{T}, \mathcal{M})} {\mathcal{C}(\widehat{T}, \mathcal{M})}$$

• $\mathcal{C}(T_O, \mathcal{M})$ : walking cost of the observed route $T_O$ based on the current map $\mathcal{M}$
• $\mathcal{C}(\widehat{T}, \mathcal{M})$ : the cost of the optimized route $\widehat{T}$
• small $\eta$ indicates better match

conclusion : the influence of stationary crowd groups is necessary when modeling pedestrian behaviors, and the stationary crowd groups should be modeled differently from scene obstacles.

Prediction of pedestrian destinations

Gaven $x_s$ and part of the walking path, we can also predict the destination of this pedestrian

$$\mathcal{L}(i) = \min_{x'_d \in \mathcal{S}_i} \mathcal{D}(T_{0.5}, \widehat{T_{0.5}}(x'_d))$$

• $T_{0.5}$ : the first half of observated trajectory
• $\widehat{T_{0.5}}(x'_d)$ : the first half of $\widehat{T}(x'_d) = f_{FM}(\mathcal{M}, x_s, x'_d)$ which is the optimized route ended with $x'_d$
• $\mathcal{D}(\cdot, \cdot)$ : the distance between the two half trajectories
• Smaller $\mathcal{L}(i)$ indicates that the pedestrian is more likely to go to the destination $\mathcal{S}_i$ which are manually labeled.

Personality attribute estimation

$$\hat{P} =\arg \min_{P} \mathcal{D}(T_O, \widehat{T}(P))$$

• $T_O$ : the observed trajectory of current pedestrian
• $\widehat{T}(P) = f_{FM}(\mathcal{M}_P(P), x_s, x_d)$ : the optimal walking path calculated using personalized energy $\mathcal{M}_P$
• $\mathcal{D}(\cdot, \cdot)$ : the distance between the two trajectories

All the pedestrians can be classified into three categories based on their walking behaviors: aggressive, conservative, and abnormal.

Conclusion

• a novel pedestrian behavior mode includeing the stationary crowd group influence
• A new pedestrian walking route dataset is proposed

---

拓展研读

前继文章

• Agent-based model
  E. Bonabeau. Agent-based modeling: Methods and techniques for simulating human systems.

• detecte stationary crowd group region
  S. Yi, X. Wang, C. Lu, and J. Jia L0 Regularized Stationary Time Estimation for Crowd Group Analysis

• generate pedestrian walking path
  R. Kimmel, A. Amir, and A. M. Bruckstein Finding shortest paths on surfaces using level sets propagation
  J. A. Sethian. A fast marching level set method for monotonically advancing fronts

• MDA model
  B. Zhou, X. Wang, and X. Tang Understanding collective crowd behaviors: Learning a Mixture model of Dynamic pedestrian-Agents

进一步学习

• a large scale crowd dataset for forecasting pedestrian destinations
  A. Alahi, V. Ramanathan, and L. Fei-Fei Socially-aware Large-scale Crowd Forecasting

• social force model
  D. Helbing and P. Molnar. Social force model for pedestrian dynamics

  • proposed for crowd simulation
    Dirk Helbing, Illes Farkas, Tamas Vicsek Simulating Dynamical Features of Escape Panic
  • used in tracking
    S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool You'll never walk alone: Modeling social behavior for multi-target tracking
  • interaction analysis
    P. Scovanner and M. F. Tappen.Learning pedestrian dynamics from the real world
  • abnormal event detection
    R. Mehran, A. Oyama, and M. Shah.[Abnormal crowd behavior detection using social force model][12]

  [12e.ieee.org/abstract/document/5459224/