[리뷰] Hard to Track Object with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space

이번에는 WACV 2023에 발표된 논문인 Hard to Track Object with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space를 읽고, 리뷰해보고자 합니다.

Index
1. Background
    1.1. Generalized IoU
    1.2. Distance IoU
    1.3. Complete IoU
2. Abstract
3. Introduction
4. Related Work
5. Method
    5.1. Tracking Pipeline
    5.2. Buffered IoU
    5.3. Simple Motion Estimation
    5.4. Track Management
6. Experiment
7. Conclusion 

1. Background

1.1. ~ 1.3. GIoU, DIoU, CIoU

다양한 IoU 종류

 

2. Abstract

• 불규칙한 motion과 구분하기 힘든 외관을 가진 object(기하학적 특징에 혼란이 있음)를 tracking하기 위한 tracker 제안
• detection과 tracking의 matching space를 확장하기 위해 buffer를 추가하여, 아래의 증상을 완화
  • 동일한 object이지만, 인접 frame에서 IoU가 0인 경우 matching
  • motion estimation bias compensation
• buffer를 이용한 방식에 matching cascade 방식 적용
  • 작은 buffer로 먼저 association 후, 큰 buffer를 이용하여 matching 시키지 못한 detection과 track에 대해 association 수행

 

3. Introduction

• 최근에는 구별하기 힘든 appearance와 irregular motion을 가진 object들을 tracking하는 과제가 주목을 끌고 있음
  • DanceTrack 같은 dataset이 그 예시
• 위의 문제에 대해 기존 tracker의 HOTA 점수가 크게 떨어지는 이유를 아래와 같이 추정
  • 인접한 frame들에서 object의 IoU가 0인 경우(object가 빠르게 이동)
  • occluded object들의 motion 추정이 부정확(object가 갑작스러운 가속 혹은 회전)
• detection area의 축척 비율을 확장한 buffered IoU를 이용함으로써, 제시된 문제 완화
• matching cascade 방식 이용

 

4. Related Work

• Appearance Consistency and Geometric Consistency in MOT
  • 인접한 frame에서 detection을 association하기 위해 사용
    • 그러나, DanceTrack, SoccerNet, GMOT-40 등과 같은 dataset을 통해 object들의 appearance가 유사할 경우 성능 하락
  • transformer가 MOT에 도입된 후, cross-frame detection의 similarity 측정 성능이 높아져, 높은 tracking 성능 달성
    • Trackformer, Transtrack, MOTR, … etc
  • geometric matching은 appearance로 인한 모호성을 줄이기 위해 사용하며, 일반적으로 IoU 이용
  • motion을 추정하기 위해 network 혹은 bayesian filter 이용
  • kalman filter 혹은 개선된 kalman filter 이용
  • camera motion compensation 이용
• Geometric Consistency Measurement
  • 기하학적 일관성 측정에 IoU를 사용하면 fully-occlusion이나 fast moving인 경우 값이 0으로 나와 tracking이 실패할 수 있음
  • GIoU나 DIoU를 사용하면 위의 문제를 어느정도 완화 가능
• Cascaded Matching
  • MOT에서 일반적으로 이용되는 방법
    • 쉬운 detection들을 먼저 matching한 다음, 어려운 detection들을 matching
  • ByteTrack, DeepSORT 등의 algorithm이 있음

 

5. Method

5.1. Tracking Pipeline

• SORT를 일부 상속받아 이용
• YOLOX를 이용한 detector와 tracker로 이루어진 2-stage 방식

5.2. Buffered IoU

• IoU를 계산하기 위해 detection과 track에 비례하는 buffer 추가
  • 원래 detection의 중심, 크기 비율, 모양을 동일하게 유지하는 확장된 공간
• original detection $o=(x,y,w,h)$
  • $(x, y)$ is top-left corner, $w$는 너비, $h$는 높이
  • buffer의 크기가 $b$일 때, buffered detection $o_{b}=(x-bw,y-bh,w+bw,h+bh)$
• 최적의 $b$를 찾아 cascaded matching 하기 위해, grid research 방식을 이용하여 training set에서 2개의 $b$($b_{1}$, $b_{2}$)를 찾고, 이를 training set과 validation set에 적용
  • $0.1 \leq b \leq 0.7 \text{ and } b_{1} < b_{2}$인 조건에서 탐색하기 때문에, 허용 가능한 계산 시간을 가짐

5.3. Simple Motion Estimation

• state 추정을 위해 kalman filter를 사용하지 않음
• 예측할 수 없는 motion 변화에 대응하기 위해 최근 frame의 motion을 평균화
• $t$시점에서 track이 $n$개 이상의 frame에 대해 detection이 matching되었을 때, track의 상태 $\boldsymbol{s}$는 아래와 같이 표현 가능
  • $\boldsymbol{s}^{t+\Delta}=o^{t}+\frac{\Delta}{n-1}\sum^{t}_{i=t-n+1}(\boldsymbol{o}^{i}-\boldsymbol{o}^{i-1})$
    
    • $\Delta$는 matching되지 않은 frame, 실험에서 $2 \leq n \leq 5$로 설정
    • $t-n$ ~ $t$ frame의 matching된 detection은 motion을 계산하는데 이용

5.4. Track Management

• SORT에서 도입한 track 초기화, 업데이트 방법을 이용
• 1st association
  • 첫 번째 frame의 경우 모든 detection을 새로운 track으로 초기화 후, detection을 메모리에 기록
    • motion 추정의 초기화를 위해 최소 2개의 tracking된 frame이 필요하므로, 새로운 track은 motion 추정 없이 바로 track에 할당
  • 다음 frame의 경우 detection과 활성 track 사이의 geometric affinity를 계산하기 위해  크기의 buffer를 가진 BIoU 이용
    • association은 hungarian algorithm과 같은 linear assignment 방식 이용
• 2nd association
  • 이전 단계에서 track과 detection이 일치하지 않는 경우가 생길 수 있으며, 이를 완화하기 위해 $b_{2}$ 크기의 buffer를 가진 BIoU 이용
• 두 단계의 association이 끝난 후, matching되지 않은 detection에서 새로운 track을 생성하고, 이전의 활성화된 track을 비활성화
• track의 age 관리
  • matching되지 않은 track의 age을 증가시키고, matching된 track의 age을 0으로 재설정
  • 사용 기간을 threshold값과 비교하여 track의 비활성화 여부를 판단

 

6. Experiment

• MOT17 test, DanceTrack test dataset에서의 성능 비교

• DanceTrack validation, SoccerNet test, GMOT-40 test dataset에서의 성능 비교

• 사용하는 IoU의 종류에 따른 성능 비교

• detection quality에 따른 성능 비교

 

7. Conclusion 

• 불규칙한 motion과 구분하기 힘든 외관을 가진 object에 대해 잘 tracking하는 tracker 제안
  • buffered matching space 이용을 통해 문제 해결
• 아래와 같은 한계가 존재
  • 매우 noisy한 detection들에 robust하지 못함
  • tracking 전, detection을 수정하기 위해 human resource가 필요

 

 

논문 링크

 

https://arxiv.org/abs/2211.14317