All transactions are linked to money records. Therefore, they are deleted with the money records.
Go to the Finance app → the Money section, select one or more records and click the Delete button. The payment record will be deleted along with its transactions: