Sorry, in my earlier comment, I thought the original post was referencing a Graph Builder output, but that may not be the case.
In Graph Builder, I suppose it depends on the Graph Builder output type. For boxplots, I see I can right click and customize the text for the 5 point summary as Mark mentions. This appears to be the only labeling apart from hover labels. For a bar charts, which I use for pareto plots, there's an option to add a label (label by value, for example). I don't see an option to customize this text.