The existing theoretical literature on fiscal competition has to a large extent ignored the role of government debt as a determinant of taxes and productive public spending. We develop a simple model of fiscal competition with government borrowing. If a default on government debt is no option, initial debt levels play no role in fiscal competition. This neutrality result is overturned when a default is possible. A government that is constrained in its borrowing due to a possible default responds optimally by lowering spending on durable public infrastructure, which in turn induces more aggressive tax setting. A rise in exogenous firm mobility reinforces the link between legacy debt and fiscal competition. Our model may help explain the observation that highly indebted countries in Europe have decreased corporate tax rates over-proportionally. Our model may also be useful for evaluating decentralization reforms in which the power to tax firms is devolved to lower levels of governments which differ in their initial debt levels.