In this paper, a new approach is presented to reduce the nonlinear characteristics of a stressed power system by reducing its second-order modal interaction through retuning some parameters of the generator excitation system. In order to determine the second-order modal interaction of the system, a new index on nonlinearity is developed using normal form theory. Using the proposed index of nonlinearity, a sensitivity function is formed to indicate the most effective excitation system parameters in the nonlinear behavior of the system. These dominant parameters are tuned to reduce the second-order modal interaction of the system and to reduce the index on nonlinearity. The efficiency of the proposed method is validated using a four-machine two-area test system. Simulation results show that a proper tuning of the excitation controller can reduce the second-order modal interaction of the system and can even improve the transient stability margin of the network.