The eruption of solar filaments is one of the main causes of hazardous space weather, and their complex evolution is often the key factor leading to the eruption. Therefore, achieving precise tracking of the complete lifecycle of solar filaments is significant for enhancing the capabilities of space weather monitoring and forecasting. In this study, a total of 584  Hα  full-disk solar images from three observatories (BBSO, GONG, and SMART) were used to build a data set, including a training set and five testing sets. They represent different observation instruments, varying density of filament trajectories, and different time intervals. A new deep learning method, the bidirectional autoregressive tracking method, named BF-TrackFormer, is proposed to detect and track solar filaments. The designed backward autoregressive branch consisting of an ID Embedding Head and a Recheck Network effectively alleviates the problem of incomplete detection of fragmented filaments caused by splitting, as well as the problem of the solar filament lifecycle being divided into multiple discontinuous trajectories due to incomplete and missed detection. The average evaluation metrics IDF1, MOTA, Prec, Reca, and IDSR of the five testing sets are 66.6%, 40.3%, 72.6%, 67.8%, and 22.8%, respectively. With decreasing intervals, the metrics get better and better. The experimental results show that BF-TrackFormer performs well in detection and tracking solar filaments, especially for fragmented filaments.