Australia is being paid to borrow. For those funds at least, borrowing is the opposite to a burden on the budget.
To understand what bonds are, it helps to see a picture.
Every one of Australia's coronavirus support payments is due to end by the end of October. It'll leave a gaping hole.