Prosper,
All streams/rivers are different; however, most will be better in the morning and later in the afternoon. The reason is, under lower light conditions of the morning and later afternoon (and at night), the little critters the fish eat feel more secure so they move about more and become fish food.
Are you fishing on the surface or subsurface? Trout feed about 80 percent (or more) of the time below the surface because there is more food available at any time of the year. You generally catch trout on the surface when there is a hatch or when you find places where you can present an attractor pattern in a non-hatch situation which trout may take. There isn't always a hatch happening. This doesn't mean you can't catch fish below or on the surface in the middle of the day.
Some anglers, who are dry fly purists and would never be caught dead with a nymph anywhere near them, must wait for a hatch to begin or use an attractor pattern. I fish mostly below the surface then if I'm in the right place at the right time and a hatch begins, I will change to a surface pattern.
Are you catching fish in the short time you have?