As one of the weakest locations in the track superstructure, the rail joint encounters different types of defects and failures, including rail bolt-hole cracking, rail head-web cracking or separation, broken or missing bolts, and joint bar cracking. The defects and failures are mainly initiated by the discontinuities of both geometric and mechanical properties due to the rail joint, and the high impact loads induced by the discontinuities. Continuous welded rail (CWR) overcomes most disadvantages of the rail joints. However, a large number of rail joints still exist in North American Railroads for a variety of reasons, and bolted joints are especially prevalent in early-built rail transit systems. Cracks are often found to initiate in the area of the first bolt-hole and rail-head-to-web fillet (upper fillet) at the rail end among bolted rail joints, which might cause further defects, such as rail breaks or loss of rail running surface. Previous research conducted at the University of Illinois at Urbana-Champaign (UIUC) has established an elastic static Finite Element (FE) model to study the stress distribution of the bolted rail joint with particular emphasis on rail end bolt-hole and upper fillet areas. Based on the stress calculated from the FE models, this paper focuses on the fatigue performance of upper fillet under different impact wheel load factors and crosstie support configurations. Preliminary results show that the estimated fatigue life of rail end upper fillet decreases as impact factor increases, and that a supported joint performs better than a suspended joint on upper fillet fatigue life.