Memory Paging - Tweak Request

Discussion in 'Windows Desktop Systems' started by Nick, Jun 23, 2005.

  1. Nick

    Nick OSNN Lurker

    Messages:
    147
    Location:
    North West, UK
    I've got my computer set up quite happily and it's on most of the time (media server, web server, etc).

    I'm running Windows XP which, very usefully, decides that if I've not been using an app for a while, pages most of it's memory to virtual memory, even though I've got 1GB of RAM and leaving it in RAM wouldn't affect the performance noticeably of the running apps. In fact since I've turned my machine on (I would give you the uptime, but systeminfo reports N/A for uptime) my peak memory load has been 924MB, so there hasn't been any need to page anything out really. If I can reduce what windows pages out to an absolute minimum, then when I come back to my computer I won't have to sit there waiting for apps to be paged back into memory.

    Does anyone have any ideas on how to acheive this?

    (Found a different way to get uptime - look at the modified date of pagefile.sys as it only gets changed on startup)
     
  2. Mastershakes

    Mastershakes Moderator

    Messages:
    1,721
    Location:
    Montreal
    Turn virtual memory off (not recommended)

    XP's core is designed to use the pagefile... Getting rid of it can seriously mess up your OS and could lead to data corruption. Windows does a great job of managing it, though I have just given it a static value so it may not fragment.

    Perris? You should be weighing in here.
     
  3. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    there's no way to turn vm off and you wouldn't want to if you could...well, unless you liked a really slow computer

    if you wanted to run without virtual memory or paging, a gig of memory isn't enough yet, you would need as much physcial memory as the data on your disc, or at least as much memory as the entire working sets of the programs you run includling the os and the files you access.

    think a gig of memory is enough for this amount of data?

    because of virtual memory, data from programs and the os that'a hardly ever used don't have use physical memory...things like your history and view source in the browser shouldn't waste memory untill you need those features

    your system is faster because of paging not slower
     
    Last edited: Jun 23, 2005
  4. Steevo

    Steevo Spammer representing. Political User Folding Team

    Messages:
    2,566
    Would I be correct in thinking that the program remains in resident memory but it becomes listed as free memory when it has not been accecssed for awhile? Thus making room for other applications that might be started and have a higher priority?
     
  5. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    ya, that's called the standby list...data remains in the free memory untill that physical memory is claimed...this way memory is available even though it's got data i written
     
    Last edited: Jun 23, 2005
  6. kcnychief

    kcnychief █▄█ ▀█▄ █ Political User Folding Team

    Messages:
    16,948
    Location:
    Massachusetts
    My personal theory on this has been pretty simple, here we go...

    My personal PC has 2GB of RAM, so I have turned off the Virtual Memory. Obviously, not all computers are that blessed, so here is the rule that I follow.

    If you have more than a gigabyte of RAM, set your swap file size to 1.5 times the amount of RAM you have. If you have between 256MB and 1GB, set it to two times the amount of RAM you have. If you're running less than 256MB of RAM, we recommend you let Windows manage your swap file for you.

    If you have a swap file, and an extra hard drive, I would recommend putting the swap file on an entirely seperate disc, perhaps one used for storage. If you do get into the access of said file, it thrashes your hard disc which can slow down loading of applications and such.

    So, to make a long story short, use the math and relocate, or kill it entirely, if your machine has the muscle.
     
  7. American Zombie

    American Zombie Moderator Staff Member Political User

    Messages:
    2,934
    Location:
    Seattle
    You should never disable the pagefile in XP no matter how much ram you have. Your system will run slower and some applications may not run at all.

    Search around for XPTC as that guide has lots of good info for setting pagefile and memory management.
     
  8. kcnychief

    kcnychief █▄█ ▀█▄ █ Political User Folding Team

    Messages:
    16,948
    Location:
    Massachusetts
    I disagree, while it really isn't recommended, I have noticed increases in system performance. That, and in combination with System Restore turned off, my disks stay much less fragmented (since I can de-frag in real-time).
     
  9. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    just not true, a pagefile can't possibly add to fragments on your drive, and turning it off more often results in more paging not less

    your computer will release memory based on a memory management model, it happens with or without the pagefile...every file on your drive is a pagefile for that specific data, the pagefile is the area for new data, when new data can't be paged because the pagefile is too small or doesn't exist, the os simply goes to a candidate that is more likely to slow you down
     
  10. kcnychief

    kcnychief █▄█ ▀█▄ █ Political User Folding Team

    Messages:
    16,948
    Location:
    Massachusetts
    Yeah, I agree, however you can't effectively defrag your hard drive when there is an active pagefile. That is why, if you do run a pagefile, the only TRUE way to defrag is to turn off system restore, turn off the pagefile, reboot, defrag, turn both back on, then reboot. Or, get a boot-time defragment utility.

    I probably didn't word it correctly previously, I didn't necessarily mean that it could add to the fragmentation level, I meant to say that you can't defragment that area of the hard drive. My apologies.
     
  11. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    why can't you effectively defrag the drive when the pagefile is active?

    once the pagefile is contiguos it will never become fragmented on a healthy drive, defrag it once and you're done with it

    system restore does fragment the drive, but I love that utility, most people that do good backups can turn that off safely, I agree
     
    Last edited: Jun 30, 2005
  12. kcnychief

    kcnychief █▄█ ▀█▄ █ Political User Folding Team

    Messages:
    16,948
    Location:
    Massachusetts
    Yes, yet again I agree in the sense that if you set it to "static" the file itself will not become fragmented. However, that "area" of the hard disk where the file resides is open to fragmentation. It's really up for debate.

    Best practice would be after a fresh install of XP, do the afore mentioned steps, turn off sys restore, turn off page file, reboot, defrag, turn both back on, reboot, then your page file and that area of the drive are good forever.

    I don't really like system restore myself, but from what I hear it is going to be MUCH improved when Longhorn comes out.
     
  13. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    "static or dynamic, the pagefile can't become fragmented once it's contiguos...that was a myth that was repeated by people who had an incomplete understanding of how expansion works

    if the pagefile is the right size, (not too small), the OS WON'T expand it, if the pagefile is too smal and the OS DOES need to expand the pagefile, the new extent is discarded on reboot, and there is once again a contiguous pagefile
     
  14. kcnychief

    kcnychief █▄█ ▀█▄ █ Political User Folding Team

    Messages:
    16,948
    Location:
    Massachusetts
    As far as I remember, if the pagefile becomes expanded it only clears at shutdown if you have it set to do so. I believe the default is NOT to clear at shutdown, but I could be mistaken. Do set it to clear, do this...

    Start Regedit.
    Navigate to
    HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlSesssionManagerMemory Management
    Select ClearPageFileAtShutdown from the list on the right.
    Right click on it and select Modify.
    Change the value to 1 to enable.
    Reboot your computer.
     
  15. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    the pagefile is ignored on every reboot regardless of the reg hack

    that reg setting is a security measure to overwrite the file with zeros (a worthless security feature)
     
  16. Son Goku

    Son Goku No lover of dogma

    OK, here's my take:

    - Personally, I would not recommend shutting it off. As I remember also NT (which 2k and XP are based off the code base for) also used the page file in a manner where if the copy of a given something in physical RAM gets corrupted, rather then a program error, it would read the copy back in from the page file and over-write the copy in physical memory. If the copy in the page file was not corrupted, it would "self correct" without having to notify the user of an error here.

    Course there would be a slight delay (to load the memory space back from disk) but against the backdrop of re-loading the entire program following a program crash...

    - Meh, I've never done 2x physical RAM. Course memory useage requirements on the system (aka there was no way to get NT 4 to even try to use 192 MB RAM, at least with the programs I was useing without some very heavy registry editing and other memory management tweaks). Anyhow, I've always set it to about 1.5 times, and that's with 512 MB now. I don't have any probs...

    - If you have 2 hard drives, it is possible to split the page file over 2 hard drives. Depending on your setup, this might or might not help.

    * If you have SCSI drives as I do, it will help, regardless if they're on the same controller. This is due to SCSI disconnection. Basically, in the case of a SCSI drive, if it isn't actively sending data over the bus, it disconnects from it (while it's doing it's thing internally but has nothing to send) so as to allow another device to send. It'll connect when it has something to do...

    This is distinctly different from IDE drives... In the case of SCSI drives, splitting your swapfile over multiple drives will improve performance. My newer x10 Cheetah drive has a sustained transfer rate about 2x faster then my older one. Given the relative performance (that I benched in SCSI bench 32 a long time back), I put 2/3 the swapfile on the new drive, and 1/3 on the old...

    * If it's an IDE drive and they're on the same controller, forget it. It's been recommended, and I'd agree with the recommendation here, leave it on one hard drive.

    * If the drives are on 2 seperate controllers, even though they're IDE, splitting should be fine. They're not useing the same controller anyhow, so won't be competing for the bus...

    - Be extremely cafeful. Various memory management settings can be found here...

    H_Key_Local_Machine\ System\CurrentControlSet\Session Manager\ Memory Management

    You get here through regedt32... But a few warnings:

    - If you don't know what something does do not change it. Not everything in the registry is in plain text, human readable format. With some understanding of the registry structure, and how it's organized, one can figure some of it out. Other things, one can look up and get some documentation on. The rest, best if left alone.

    - Take care of what values you're entering. For instance if some value is a 0x something or other, the value is in hex or base 16. Do not, for instance with the manual setting of the L2 cache size on the CPU (SecondLevelDataCache under the above mentioned key), give the number of KB in decimal (or base 10 math, that we normally use). You'd have to convert the value to base 16 first. Easiest way would be to use the calculator in Windows, enter the number in base 10 (with it set to dec mode), and then click on the Hex radio button. It'll convert it for you.

    0x512 != 512 KB (in the above example), but rather 1,298 KB (or over 1.5 MB L2)...

    - I found a little info here, but only used it to search for the key again

    http://windows.about.com/od/customizingwindows/l/aa001008c.htm

    There are a lot of other values in there. If you aren't certain about something, search on Microsoft's support site (I know the resource kits have a registry entry help file) or ask...

    About the only thing I can think of that might help, is a large cache size (for your file cache) though this doesn't effect paging directly, or the DisablePagineExecutive (to decrease paging), but this won't effect programs, just the system kernel...
     
  17. Sinster

    Sinster Moderator

    Messages:
    520
    Location:
    M*ssholechusetts
    I keep getting VM is low warning when I play BF2. I'm running a AMD 3000+ Venice, MSI NEO2 Plat, 1 GB 512x2 PC3200, BFG 6800Ultra APG 256Megs, I have my 120 Gb HDD partitioned into "O/S", "Programs" and "Page File" and my page file was set to 2048 max and 512 min.. with 15% free on the "O/S" partition and 40% free on "Programs" partition. But yet I keep getting this warning.
     
  18. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    why is your minimum so low? you made your intitial minimum lower the what the os tells you it wants...the minumum is 1.5

    anyone that told you to set it lower has an incomplete understanding
     
    Last edited: Jul 19, 2005
  19. Sinster

    Sinster Moderator

    Messages:
    520
    Location:
    M*ssholechusetts
  20. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,332
    Location:
    new york
    he's wrong, and yes I have spoken to the memory management team at microsoft

    click the link in my sig for a more complete understanding...that paper is old and needs some corrections of it's own, but it is more informative then alex's paper.

    in addition, if you read his paper, he doesn't suggest any gain by lowering the default, he just wants you to do it to "save space"

    in my discussion with him his reason was "I abhor wasting disk space"...which is of course an obsession he has yet to overcome

    plus a maximum that's the same as the minimum serves no perpose, expansion isn't invoked unless it's needed and there's no reason eliminate the os's ability to accomodate you if your use increases
     
    Last edited: Jul 19, 2005