In most places where selection is supported, shift-click will extend selection (select more items without deselecting the old) and ctrl-click (command-click on Mac) will toggle the selection of the clicked item.
In the data table, you can also clear row selection or column selection by clicking in the areas shown: