The relationship between budget deficits and macroeconomic variables (such as growth, interest rates, trade deficit, exchange rate, among others) represents one of the most widely debated topics among economists and policy makers in both developed and developing countries. However, the purpose of this paper is to review the extensive literature to such a relationship, concentrating on theoretical debates and empirical studies, in order to derive substantive conclusions, which can be beneficial in the macroeconomics area; policy analysis; or in terms of constructing or developing a macroeconomic model for analyzing the impact of budget deficits on macroeconomic variables. The majority of these studies regress a macroeconomic variable on the deficit variable. These studies are cross-country and utilize time series data. In general the key outcomes from the studies presented in this paper indicated that both the method of financing and the components of government expenditures could have different effects. Therefore, it is crucial for the government to distinguish between consumption and investment expenditures especially when the government is in the process of evaluating the impact of fiscal policy on private investment and output growth or in the process of cutting expenditures to reduce the fiscal imbalances in the country. Even though the overall results from the empirical literature with respect to the impact of public investment on private investment and growth are ambiguous, the bulk of the empirical studies find a significantly negative effect of public consumption expenditure on growth, while the effects of public investment expenditure (such as on education, healthcare) are found to be positive although less robust. The key findings from these studies is important in particular for developing countries to be aware of the importance of government investment expenditures in the area of education, healthcare, infrastructure to long-term economic growth and the benefits from which are an important contributor to welfare and well-being. The key outcome from all of the studies presented in this paper while investigating the relationship between the budget deficit and current account deficit showed strong evidence in both developed and developing countries towards supporting the Keynesian proposition (conventional view) which suggests that an increase in the budget deficit would induce domestic absorption and, hence import expansion, causing a current account deficit. The key findings from the empirical studies investigating the relationship between the budget deficit and interest rates indicated strong evidence towards supporting the Keynesian model of a significant and positive relationship between budget deficits and interest rates. The major outcomes from the empirical studies examining the relationship between budget deficits and inflation showed strong evidence that the budget deficit financed through monetization and a rising money supply could lead to inflation.