It makes a difference - many people may not see it, but it does.
By the time a file has gone to print, lots of details and color are lost.
RGB (16.7 million) -> CMYK (10 million) -> Film -> Plate -> Paper.
I no longer have the formulas for figuring out exactly how many shades are possible depending on the linescreen, but printed CMYK has far fewer colors than the original CMYK file. On most papers, and presses there is no difference between 0% color, and 2%, just as there's no difference between 98% and 100% ink.
Working in RGB gives you more colors than working in CMYK. Converting to CMYK as the last step, rather than working in CMYK gives you a better, richer range of colors that isn't possible working in CMYK, simply because you have 65% greater colors. The more colors you start with, the more that will end up on the paper, even though you've gone from tens of millions of colors, to tens of thousands of colors.
Think of it like painting with oils. Theoretically, you can create all the colors you need for print with just Cyan, Black, Magenta and Yellow. However, how many painters do you know that use *just* those colors for painting book/comic covers?
Some painters have dozens or more different tubes of oil paints that are outside the CMYK gamut. Why? because they get a greater range of colors, tones that make for better blends of color.
That's what RGB gives you over CMYK.
I made a simple gradient, in RGB using CMYK colors (not out of gamut). I made the same simple gradient with the same colors in CMYK.
I also took the RGB version, converted to CMYK, then back to RGB (Converting from CMYK to RGB doesn't increase the range of colors, it's just done because you can't really show RGB on websites).
It should illustrate the difference for you.
As you can see with the version done in CMYK, the colors tend toward greys in the middle where the two colors blend. In RGB, they are still vivid, bright colors, not desaturated/muddy.
Even the version originally done in RGB, converted to CMYK (and back) shows a greater range of color than the original CMYK (this has the original CMYK gradient below for comparison).
I hope this explains it a little better.