The influence of temperature on the inverse Hall–Petch effect in nanocrystalline (NC) materials is investigated using phase field crystal simulation method. Simulated results indicate that the inverse Hall–Petch effect in NC materials becomes weakened at low temperature. The results also show that the change in microscopic deformation mechanism with temperature variation is the main reason for the weakening of the inverse Hall–Petch effect. At elevated temperature, grain rotation and grain boundary (GB) migration seriously reduce the yield stress so that the NC materials exhibit the inverse Hall–Petch effect. However, at low temperature, both grain rotation and GB migration occur with great difficulty, instead, the dislocations nucleated from the cusp of serrated GBs become active. The lack of grain rotation and GB migration during deformation is mainly responsible for the weakening of the inverse Hall–Petch effect. Furthermore, it is found that since small grain size is favorable for GB migration, the degree of weakening decreases with decreasing average grain size at low temperature.