This is a damn good question. As I've suggested in this post, functions should be short and do just one thing. The Linux kernel coding style sets even a maximum length of fifty rows for a function.
But following to the letter these rules on big projects, leads to thousands of micro snippets that make the code almost impossible to understand. This situation is more evident in object oriented languages where you may be tempted to create classes even if a simple structure would be enough. One of the techniques that make this thing evident in C++ is to implement every trivial class in its own .hpp file.
The general rule is that, if a function or a class is used only in a single piece of code, maybe it doesn't need to be separated. It seems reasonable but it may lead to awful functions difficult to understand. Another exception may done if you have the feeling that, in the future, that function/class will be used somewhere else.
The truth is that it's all about your sensitivity as a programmer. In this case, the experience should guide you to the right choice, because
And, in any case, a good refactoring from time to time can improve your code.