link_df_iter giving TypeError when using pd.concat?

[zimengwu]

Hi everyone!

I've recently started using trackpy and it's been amazing. I started using the link_df function on cropped datasets (centroids from segmented data) to figure out the parameters for linking and it worked a charm. However, when I tried using the same pipeline on my full timelapse data (~7000 labels over 60 frames) I found it took quite a long time. So, I turned to link_df_iter as I read it takes a much shorter amount of time to link and I used the same parameters I used with link_df! However, whenever I try to run this code:

linked_df = pd.concat(tp.link_df_iter(df, t_column='frame', pos_columns=['z', 'y', 'x'], search_range=20, memory=8), ignore_index=True)

I've been getting this TypeError which points to the t_column from my dataframe.

---

TypeError Traceback (most recent call last)
Cell In[9], line 6
3 velocity = np.array((0, 0, -5)) # (v_z, v_y, v_x)
4 return particle.pos + velocity * (t1 - particle.t)
----> 6 linked_df = pd.concat(tp.link_df_iter(df, t_column='frame', pos_columns=['z', 'y', 'x'], search_range=20, memory=8), ignore_index=True)

File ~\anaconda3\envs\tracking\lib\site-packages\pandas\core\reshape\concat.py:380, in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
377 elif copy and using_copy_on_write():
378 copy = False
--> 380 op = _Concatenator(
381 objs,
382 axis=axis,
383 ignore_index=ignore_index,
384 join=join,
385 keys=keys,
386 levels=levels,
387 names=names,
388 verify_integrity=verify_integrity,
389 copy=copy,
390 sort=sort,
391 )
393 return op.get_result()

File ~\anaconda3\envs\tracking\lib\site-packages\pandas\core\reshape\concat.py:443, in _Concatenator.init(self, objs, axis, join, keys, levels, names, ignore_index, verify_integrity, copy, sort)
440 self.verify_integrity = verify_integrity
441 self.copy = copy
--> 443 objs, keys = self._clean_keys_and_objs(objs, keys)
445 # figure out what our result ndim is going to be
446 ndims = self._get_ndims(objs)

File ~\anaconda3\envs\tracking\lib\site-packages\pandas\core\reshape\concat.py:502, in _Concatenator._clean_keys_and_objs(self, objs, keys)
500 objs_list = [objs[k] for k in keys]
501 else:
--> 502 objs_list = list(objs)
504 if len(objs_list) == 0:
505 raise ValueError("No objects to concatenate")

File ~\anaconda3\envs\tracking\lib\site-packages\trackpy\linking\linking.py:278, in link_df_iter(f_iter, search_range, pos_columns, t_column, **kwargs)
274 coords_iter = coords_from_df_iter(f_coords_iter, pos_columns, t_column)
276 ids_iter = (_ids for _i, _ids in
277 link_iter(coords_iter, search_range, **kwargs))
--> 278 for df, ids in zip(f_iter, ids_iter):
279 df_linked = df.copy()
280 df_linked['particle'] = ids

File ~\anaconda3\envs\tracking\lib\site-packages\trackpy\linking\linking.py:276, in (.0)
273 f_iter, f_coords_iter = itertools.tee(f_iter)
274 coords_iter = coords_from_df_iter(f_coords_iter, pos_columns, t_column)
--> 276 ids_iter = (_ids for _i, _ids in
277 link_iter(coords_iter, search_range, **kwargs))
278 for df, ids in zip(f_iter, ids_iter):
279 df_linked = df.copy()

File ~\anaconda3\envs\tracking\lib\site-packages\trackpy\linking\linking.py:88, in link_iter(coords_iter, search_range, **kwargs)
85 coords_iter = iter(coords_iter)
87 # interpret the first element of the iterable
---> 88 val = next(coords_iter)
89 if isinstance(val, np.ndarray):
90 # the iterable was not enumerated, so enumerate the remainder
91 coords_iter = enumerate(coords_iter, start=1)

File ~\anaconda3\envs\tracking\lib\site-packages\trackpy\linking\utils.py:70, in coords_from_df_iter(df_iter, pos_columns, t_column)
68 yield None, np.empty((0, ndim))
69 else:
---> 70 yield df[t_column].iloc[0], df[pos_columns].values

TypeError: string indices must be integers

When I also run just the tp.link_df_iter(df, t_column='frame', pos_columns=['z', 'y', 'x'], search_range=20, memory=8) line no error and an output:

<generator object link_df_iter at 0x0000018D83335A10>

Which makes me think it's not to do with the link_df_iter but how it interacts with pd.concat?

I've checked my original df going into and my 'frame' column are all ints and I have no NaN values either. I was just wondering if anyone else experienced anything like this and would know how I should troubleshoot? Thanks for any help!