Maybe it's because our brains are always bouncing around. Thinking about work, and life, and what to have for lunch, and bills, and memories. Like a humming bird bouncing between flowers. Where as fish have only 1 thing to concentrate on.....surviving. In order to catch them we have to immerse ourselves in a underwater world that we are not meant to live in.
Besides, for me at least, fish being hard to catch is why we do it. Don't get me wrong they are delicious, and the meat is a nice side affect, but the chase is where the fun is. If all I wanted was the immediate reward of a fish in the freezer I would just go to the supermarket.