The nonlinear regression in Fit Curve and Nonlinear both require starting values. Fit Curve uses heuristics associated with each model to obtain starting values from the data. Nonlinear does not have this feature. You must specify the starting values when you define the parameters in the Formula Editor.
The heuristics simply consider the interpretation of the parameter and estimate the starting value. For example, a parameter might represent the central tendency of the curve, so the mean or median are good choices. Or another formula might have an inflection point, so the mid-point might be a good choice.
The regression then searches for estimates (minimizes loss function) until any of the convergence criteria are met. It is not a closed-form exact solution.
Yes, some functions are particularly difficult and demand more careful selection of the starting values.