key: cord-0047475-pt70kqc1
authors: Wu, Di; Bu, Hui; Xu, Benlian; Lu, Mingli; Sun, Zhen
title: An Ant-Inspired Track-to-Track Recovery Approach for Construction of Cell Lineage Trees
date: 2020-06-22
journal: Advances in Swarm Intelligence
DOI: 10.1007/978-3-030-53956-6_20
sha: 151acca2a73054d60667adbfaa12e717da5d15c4
doc_id: 47475
cord_uid: pt70kqc1

Correct track-to-track association is crucial to the construction of cell lineage trees as well as the discovery of novel biological phenomenon that occur at rare frequencies. In this paper, an ant colony optimization based heuristic approach is proposed to link potential tracks through minimizing the cost function that mainly occurs on the fragmented intervals with the constraint of maximum inter-frame displacement. Specifically, both cell motion and morphology are emphasized in the defined cost function, and two decisions are made respectively to recover the mitotic and non-mitotic cases. Our method has proven to be feasible that can repair the broken tracklets caused by large migration, occlusion and mitosis missing, as well as false positives and missed detections, and can effectively help the construction of reliable cell lineage trees.

The analysis of cellular behaviors is crucial for human disease diagnose and biomedical research. The existing automated cell tracking methods mainly focus on frame-by-frame association that can achieve high tracking accuracy in continuous image sequences [1, 2] , but it still exists some problems due to noise or segmentation error. For example, when a false negative segmentation occurs among a neighboring cell cluster, the normal tracking methods may cause a track missing. To resolve the problem, many global spatio-temporal data association approaches have been proposed to associate multiple trajectories over time of which the most well-known are the Multi-Hypothesis Tracking (MHT) [3] and Joint Probabilistic Data Association Filters (JPDAF) [4] . In [5] , a batch algorithm, which utilizes information from all images in the image sequence, incorporates mitosis, apoptosis and other events into the same probabilistic framework for track linking. To reduce the computational cost, Huang et al. [6] first generate reliable track fragments and then stitch them by Hungarian algorithm.

In this paper, we introduce an ant colony optimization [7] based on heuristic approach linking potential tracks through minimizing the cost function that mainly occurs on the fragmented intervals with the constraint of maximum interframe displacement. In our work, each track segment is assumed to be associated with an ant colony. All ants in the colony start their foraging behaviors from the initial position of the track, and each ant makes decision via both the heuristic function and the pheromone intensity on the track. During decision, two events, namely, migration without mitosis and migration with mitosis, are considered to link the potential tracks with identical ancestry. As iterations increase, the trail pheromone field is continuously strengthened, and the trails with high pheromone intensity above a prespecified threshold are finally extracted. The rest of this paper is organized as follow: Sect. 2 details our ant-inspired trackto-track recovery method. We provide our experimental results in Sect. 3 and conclude this paper in Sect. 4. Figure 1 (a) illustrates an example of a set of broken cell tracks produced by an automated tracking algorithm, where cell occlusion, erratic motion and division probably cause the undetected candidates, while the presence of clutter usually generates a set of short track candidates. In order to construct correct cell lineage trees, a track recovery approach is needed to link the associated broken tracks and remove the false alarms, as shown in Fig. 1(b) . 

where α 1 is a constant range in [0, 1], E dis (p k u , p k+d v ) and E area (p k u , p k+d v ), respectively, represent costs of distance and overlap of area between cells p k u and p k+d v . The cost E dis is defined as

where H and W denote the height and width of an image respectively. The cost E area is formulated by

where A(p k u ) refers to the area of the cell p k u , A(p k+d v ) refers to the area of the cell p k+d v . We observe that the closer the two cells, the smaller the cost function. An one-one mapping is defined between a track and an ant colony. With the known cell states of each track, we put a group of ant colony starting foraging from the initial position of a tracklet. Then each ant makes decision to move to the next cell in sequence with constraint of maximum inter-frame displacement.

Firstly, each ant a at the current position p k u finds the set of candidates N k+d a to be visited in frame k+d. If |N k+d a | ≥ 2, the detection of mitotic event is carried out. Let p k+d m and p k+d n denote the states of any two cell candidates, namely cell m and cell n, in frame k + d, the similarity score S(p k u , p k+d m , p k+d n ) is defined as

where β i is a constant range in [0, 1] and

respectively, denote the similarity scores of area, distance and deformation of cells, which are defined as

where L h (·) and L w (·), respectively, denote the length of major and minor axes of the estimated cell contour. If the similarity score S(p k u , p k+d m , p k+d n ) is above a prespecified threshold ε 1 , two ants will select p k+d m and p k+d n respectively and the selected cells are disjoint with p k+d m ∩ p k+d n = ∅. If no mitotic event is detected, that is, the similarity score S(p k u , p k+d m , p k+d n ) < ε 1 or |N k+d a | = 1, each ant a at the current position p k u selects a candidate p k+d v according to

where τ p k u ,p k+d v denotes the amount of trail pheromone on path (p k u , p k+d v ) and

. In our method, we first set d = 1. If |N k+d a | = 0, we expand the ant's step size as d ← d + 1, and then the ant repeats the above decision-making process.

Once ant a has made a decision, a local update process of trail pheromone is conducted on its visited path by

where Δτ c denotes a fixed amount of pheromone. When all ants in the colony complete their tours, an individual global trail pheromone update is defined as

where Q 1 is a constant, E bt denotes the cost of the best tour P bt . Once all ant colonies complete their foraging, a global update trail pheromone on all ant colonies is performed by

where m i=1 E i bt denotes total cost of the set of best tours {P i bt } m i=1 of all ant colonies in current iteration, P i bt denotes the ith best tour of {P i bt } m i=1 . As iterations increase, the trail pheromone field is continuously strengthened. Once reaching the maximum value of iteration, the trails with high pheromone intensity above a prespecified threshold ε 2 are finally extracted by morphological operations. The main framework of our method is summarized in Algorithm 1.

initialization, the iteration parameter t = 1, and the maximum iteration parameter tmax, mitotic similarity threshold ε1 and pheromone extracting threshold ε2. 1: while t ≤ tmax do 2: whileW = ∅ do 3:

for each ant colony inW do 4:

for a group of ant colony do 5:

if find candidates then 6:

if above two candidates then 7:

Calculate mitotic similarity S via (4). 8:

if S > ε1 then 9:

Mitotic decision made by two ants. 10: else 11:

Migration decision made by an ant via (8 Global pheromone update via (12). 25: t = t + 1 26: end while 27: Extract the trails lasting more than 3 frames with pheromone intensity above ε2.

To test the performance of our proposed track recovery method, we implemented it in MATLAB(R2014a) on a 2.6 GHz processor computer with 16G random access memory. In this section, we use two image sequences from publicly available website at http://www.celltrackingchallenge.net/. The two cell image sequences include real N2DL-Hela+ dataset and pancreatic stem cell dataset. With the reliable track fragments results, our track recovery method proved to be effective. The parameters used in our approach mainly focus on ant colony setting, we have α = 3, β = 2, and ρ = 0.3. The initial value of the pheromone field is c = 0.0005. We set the mitotic similarity threshold ε 1 = 0.85, the pheromone extracting threshold ε 2 = 0.05 and the maximum iteration parameter t max = 20. We find that three factors of similarity score in (4), i.e., area, distance and deformation of cells, are crucial to the recovery of mitotic events, which affects the track accuracy. To observe the effect of these factors on similarity score, we compare the performance of different combinations using several measure metrics.

The track recovery results on two cell image sequences are, respectively, shown in Fig. 2 and Fig. 3 , which illustrate the track recovery results including lineages and cell contours based on the raw fragmented tracks. In N2DL-Hela+ dataset, there are 25 tracklets with different track colors before track recovery in Fig. 2(a) , which is generated by cell erratic motion, cell division, and cell occlusion. After the proposed recovery method, the true reliable tracks have been stitched to be 19 entire trajectories as shown in Fig. 2(b) . From the lineage trees as shown in Fig. 2(c) , the track segments marked in red circles have been correctly linked into longer trajectories as illustrated in Fig. 2(d) . In pancreatic stem cell dataset, the 35 tracks in Fig. 3(a) including false positives and false negatives have been recovered to be 25 entire tracks, and false positive tracks are removed as shown in Fig. 3(b) . The comparison lineage trees are also shown in Fig. 3 (c) and Fig. 3(d) with red circle marks.

Obviously, our track recovery method can fix broken tracks and restore mitotic events, and finally can obtain the whole reliable cell lineage trees. Figure 4 shows the resulting trail pheromone field. The color corresponds to pheromone intensity, and the trails with the amount of pheromone greater than a prespecified threshold are extracted. For performance evaluation, we use the complete tracks (CT) [8] , which shows how good an approach is at reconstructing complete reference tracks and is defined as CT = 2T rc T gt + T c , where T rc denotes the number of completely reconstructed reference tracks, T gt denotes the number of all reference tracks and T c denotes the number of all computed tracks. To jointly evaluate the errors of label and cardinality of tracks, the metric of optimal sub-pattern assignment metric for track (OSPA-T) [9] is introduced as well. We compare the performance of different combinations of three factors of similarity score in (4), i.e., area, distance and deformation of cells, as shown in Table 1 . We observe that the performance is better when all three factors are taken into account simultaneously, in which the mitotic events are accurately detected. 

In this paper, we proposed an ant-inspired track recovery approach linking potential tracks through minimizing the cost function that mainly occurs on the fragmented intervals with the constraint of maximum interframe displacement. Through constant iterations, the completed repaired trails were obtained through minimizing total cost. It is demonstrated that our proposed approach successfully obtain reliable cell trajectories and lineage trees by stitching sets of true track segments and reconstructing missing mitosis branching.

Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis

An automated cell tracking approach with multi-bernoulli filtering and ant colony labor division

An algorithm for tracking multiple targets

Sonar tracking of multiple targets using joint probabilistic data association

Global linking of cell tracks using the Viterbi algorithm

Robust object tracking by hierarchical association of detection responses

Ant colony system: a cooperative learning approach to the traveling salesman problem

An objective comparison of cell-tracking algorithms

A metric for performance evaluation of multi-target tracking algorithms