You can only delete a connection when it is not in used by any device, make sure all your devices have merged to another connection or that you have unclaimed all your devices using the connection!
The delete connection button will appear when no devices are using the connection.