When I have PS open, I change the bottom number to show the efficiency. That number may be going down for you which would indicate PS needs more memory to run and it is running out. It using your graphics card's memory (if you have it set to that) and then your scratch disk. Once it using the amount you allotted for the disk, it will use your main drive (if it doesn't already) and at that point you could start to see some lag (i.e. moving objects is very slow, slow loading, freezing).
Not sure if you are on a Mac or Windows PC, but in Windows I usually open the task manager when it happens to see what programs are using memory and taking away from PS. I have 2 drives now so thankfully I don't have to worry about it anymore. It can be a pain. Hope you get it worked out!