Similarly, we may feel many points of pride when working on the work project. But we will also for the most part feel not very much at all, as one does when they 'enter the zone' for several hours a day. If feelings of pleasure were paramount, one could spend that time on pleasure-seeking.
On the other hand, it would be a little silly to assume happiness plays no role. There is a sense in which we jump in the pool to save the child because we find the state of things where the child lives a happier one than the one where the child dies. And there is a sense in which we enter into this long and tiresome work project at the expense of many pleasures we could instead pursue, because the worthiness of the project makes us feel more fulfilled than anything else could.
Our problem is that, once we broaden the definition of 'happiness', to include all these different types of things--pride, fulfilment, abstract happiness, and physical happiness--and anything else they may be included, we face the dilemma that happinesses conflict, and it is not obvious which we maximise and which we leave behind. Mill's solution to this problem was a rather unsuccessful appeal to human nature. He divided happiness up into 'higher pleasures' and 'lower pleasures'. It was essentially a kind of Victorian snobbery, which mistook the trends of his time and class for human nature. Indeed, his mistake was in thinking that 'human nature' could be any help to us at all in solving the problem.
But if we put aside human nature, what are we left with? Nurture. We are left with the ideas and values of an individual, to which our happiness (of any kind) are subservient.
Happiness is an end in itself then, not because happiness is intrinsically good, because in fact happiness isn't intrinsically anything it seems, but because some form of happiness or another is the end expression of actions that realises our values. Likewise, some form of unhappiness or another is the end expression of actions that don't. What determines the hierarchy then, is not kinds or quantities of happiness, but what we value most highly.