Speaking about my preferences and experience, the only way I can determine what an ideal screen size for me is to buy a few phones one with each of the various sizes of displays. There's no substitute for first hand usage. Thus I've had sizes 4.3, 4.5, 4.7, 4.8, 5.0, 5.5, and 5.7".
Of these I like 5.0" the best by a very wide margin. It's the size featured on my Nexus 5 and quite ideal. A very close second is the 5.7" of my Note 3. Because I use mine for all the reasons a smartphone exists, I need the working room for fast typing input via SWYPE. In addition it greatly reduces the amount of scrolling required by a tiny 4.0" display.
If the iPhone 6 is only 4.7" that's not enough for me. Personal preference, heavy frequent usage for emails, web research and the like demands a 5". So that's the sweet spot for me.
YMMV