re- How To Limit The Cache In XP

Discussion in 'Windows Desktop Systems' started by sixthofmay, Dec 1, 2002.

  1. sixthofmay

    sixthofmay Guest

    seems the thread is closed so...

    Dealer wrote-
    "use your ram as a back up for your ram...hehe...sounds like a real good idea from here doesn't it"

    heheheheh.. It does sound absurd doesn't it? [​IMG]

    But MS makes one resort to absurd measures sometimes...

    Elkinm- if you try the ramdisk pagefile thing, don't make it too small or you will get a popup error message, VM low, and Windows will create one for you on the harddisk. I'm creating a web page that I hope to have done over the xmas break with all my research on it that will explain why. I'll just say right now you need enough VM space to hold all the possible trimmings from your working set. Which could be substantial. To be on the safe side, keep your OS RAM and RAMdisk disk ram equal in size. You can experiment with a smaller RAMdisk pagefile. The worst that will happen is getting the popup error message.

    I found some very good evidence supporting my gradual slowdown theory caused by internal pagefile fragmentation. I'll have the reference on my web page when it's done.

    Can anyone tell me if XP slows down if left running 24/7, that a reboot cures? If not, perhaps one of the VM enhancements MS did to XP was to optimize the page placement algorithm to minimize internel pagefile fragmentation.

    I'm at almost 5 days uptime now with no slowdown whatsoever and stable as can be. Truly amazing. Oh, I loaded Perfmon finally, and am watching Memory:page Reads/sec. The hard page fault counter. MS says they should be under 20 a sec for good performance. Some of the things I do cause it to be in the 200 to 300 range for tens of minutes. I hit 5700 peak and averaged 3000 a sec for 10 seconds closing one of my apps down. When I used to have the pagefile on the harddisk, that app would take about 3 or 4 minutes to close down and slow the system down in the process. And people wonder why I'm excited about this...

    Dealer, again I have no affiliation with anyone. You are free to contact Cenatek and ask if you'd like.
     
  2. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    first of all, there is no fragmentation of the inside of the pagefile

    the information in the pagefile is ignored on each new boot...to defrag the internal pagefile, all you need to do is boot.

    six...this surely looks like an advertisement, though you say it is not...

    I have no problem with anyone trying your product.

    so you got your message across

    again, good luck
     
  3. sixthofmay

    sixthofmay Guest

    Dealer wrote:
    "first of all, there is no fragmentation of the inside of the pagefile"

    The makers of Diskkeeper would disagree. Their white paper describes it and the detrimental effects on performance. I don't know if Diskkeeper corrects it. XP might minimize it with placement optimizations. Still, a RAMdisk pagefile would be a few orders of magnitude faster. I found several other documents describing the problem, and also on VMS and MacOS. It's not a new problem. Guess you've never studied low level database design..

    And now you're gonna think I work for the makers of Diskkeeper and Cenatek....
     
  4. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    diskeeper resolves a fragmented pagefile...nothing whatsoever to do with the inside of a pagefile

    the information inside the pagefile is totally ignored on every boot...this is common knowledge, six

    there is no internal fragmentation of the pagefile that is not resolved with a reboot.

    sorry

    I'll be back tomorrow...have a good night six
     
  5. sixthofmay

    sixthofmay Guest

    First you say:
    "first of all, there is no fragmentation of the inside of the pagefile"

    Then you say:
    "Rebooting resolves any fragmentation inside the pagefile."
    "There is no internal fragmentation that is not resolved with a reboot."

    ummm.. so is there internal fragmentation or not?
    hehehehe.. sorry..

    True about the rebooting part. The pagefile index is erased, clearing the fragmentation.

    "diskeeper resolves a fragmented pagefile"
    Diskkeeper does correct file level fragmentation of the pagefile. Internal structure fragmentation, I don't know as I've never used it. I'll have to read their program manual to see if it mentions it.

    I'm going to bed too. Goodnight. [​IMG]
     
  6. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    ah...let me add something here.

    I will agree with you six, if a computer is running for a long period of time, the internal of the pagefiel can be fragmented...(resolved on reboot)

    I will agree, a ram disc would resolve that issue without rebooting.

    ps...
    I never suggested diskeeper would defrag the internal of the pagefile...why would it?..you would have to reboot for the utility yo access the pf, and the very reboot defrags the intenals of the pf
     
  7. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    another btw, six

    perfmon only demonstrates potential pagefile use, not actual pagefile use.

    the attached program will tell you exactly how mush is actually written to the pagefile

    for instance, right now, my pagefile useage is 240mb...the program demonstrates only 12mb written of this potential

    a good tool for your experiments...it will also log your pf use if youe want, the peak.
     
  8. Fungiver

    Fungiver Guest

    Pagefile minitor

    Thanks for the neat little tool Dealer. My Norton Anit-Virus doesn't like it though. Calls it a malicious script. Nasty script. But I authorized it and now I'll use it to track my pagfile usage just for curiosity. I look forward to your page on the pagefile and wonder if there is any way I could get on a mailing list so that I can be sure I don't miss it. Thanks.
     
  9. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    OK, gave this ramdisc a go..512 MB ram..50% to ramdisc

    I loaded, at differant intervals..Maya, ms word, 4 ie pages, photo-shop, fox mail, iespell check..coupla other normal computing utilities as I work

    everyone knows, I am predisposed against the idea of commiting your ram as a storage divice for other ram, so let me say to everyone before anyone else does, I was ready for this product to fail

    it did

    as expected, every working set on any of my programs decreased.(obviously)

    lowering the working set of any program will obviously make some features slower to access

    six...you I guess think this is a good idea...lowering the over all working set totals...I do not think anyone could possibly think lowering your working sets could possibly speed you up

    as expected, my desktop icons would need to refresh almost everytime I went to the desktop, instead of never, as in the past...though I will admit, the refresh was almost intantaneous.(as expected)

    Maya, an animation program almost refused to allow me some editing procedures, and I had to wait for maya to decide to let me go forward...once these features were loaded the first time, they remained responive, even when I went back to this hours later...this was a trade in performance...some may appreciate the trade...

    even "Microsoft's word" was less intuitive, and slagged on opening further documents...but again, the document would open more quickly later on...another trade in performance

    loading my fox mail program actually froze xp for a small period of time...no trade here...just a slow down

    my spellcheck in internet browser took about 2 seconds to load...it usually loads in less then a second...even once this is loaded, if I turn it off, and reload, this takes longer, now that I am using ram to store other ram...no trade on this either...just a slow down

    an unexpected result, photoshop showed me no difference that I could measure either way.

    none of my browsing, or loading browser pages showed any change I could detect

    I expected ie to crash, it did not

    in general, this ram disc made my operating system able to access half the ram it actually had, and my computer almost entirely behaved just as it did when I had 256.

    there were several occasions, where opening long old windows was indeed faster...

    these are obvious trades between the smaller working sets that became available



    now, that told

    I will cede another point.

    if you have such an abundance of ram, that XP would not know what to do with the excess, if XP has put every waking bit of your ram to every waking excuse for a use, and XP still has plenty to spare, in this case, I would agree, you might as well put the ram to some use...and maybe a ramdisc would make sense here.

    for some of you, that figure might be a gig...I don't know
     
  10. Fungiver

    Fungiver Guest

    Missed something

    Hey Dealer did I miss something. I thought from the articles I have read on the subject, that XP uses all the RAM all the time even if the RAM is given redundant operations. It is designed that way. The same goes for the swapfile. XP is a Swapfile OS. Designed from the ground up to be that way. It simply has to use the swapfile and will, no matter what configuration anyone tries to use. I have used your tool(god that sounds rude) to see just how much virtual memory I am using and it is very very little. Arounc 37 MB with a peak of 50. Not much on here though after that &^%$*# patch for messenger crashed me.
     
  11. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    not sure I follow your question or assumptions fungiver.

    first, xp uses the pagefile for kernal stability...

    then, it also uses the pagefile to make sure you can always have the ram you've installed both in use, and still usable to other purposes at the same time.

    creating a ram disc, will in essence, hide the ram...turning it into a disc, and XP will not be able to assign it as ram...it will just hang around, as if it's a harddrive...hard to believe, but that's the very principle of this ram disc...to make sure that XP does not assign the ram somewhere else

    Like I said, I have no problem with anyone that thinks XP is assigning your ram badly to give this program a shot.

    as a matter of fact, if you would like your ram to be specifically assigned to the process you consciously open, and nowhere else, this ram disc might serve you well.

    this ram disc did in fact, slow me down...some of you might actually find value to the situations where this ramdisc will actually open old windows faster...(this will obviously be at the expense of fewer features being available to programs you are currently working with)

    if you are referring to my paragraph, where I created a scenario where you might have more ram then XP can find a use for...

    I believe there is a point in the increasing of ram, where there would be diminishing returns, and you might have a better use of ram as long term storage, rather then for the immediate access we typically prefer ram to be used.

    but that is just a speculation in my part...I have no idea where that point might be.

    that answer will have to come from someone else.
     
  12. sixthofmay

    sixthofmay Guest

    Dealer-
    Very interesting results you had. Thank you for trying the experiment! It certainly appears that XP is handling memory decently. But there are other variables in the equation here:

    1. Overall system speed, as determined by the CPU type, RAM type, chipset, etc.
    2. Harddisk speed and bus type (SCSI, IDE, etc.)

    What kind of hardware setup are your running? Please do tell so I can see if there is some kind of correlation.

    My theory is that faster machines are much less likely to have significant paging delays. The Pentium IV and Athlon/Duron generation systems are much faster than the older systems. Every new machine I've played with was just blazingly fast compared to my PIII 700 machine. One test I like to do is note the performance when viewing Acrobat PDF files. On my machine, redraws are quite evident and one can't easily scroll through a document. On an Athlon machine, redraws are instant.

    My theory is that there is significant CPU overhead in handling I/O from the harddisk during a page fault, and less overhead if a RAMdisk pagefile is being used. I'll be doing some experiments with Perfmon to see if this is the case.

    Thus those using new machines may not see much if any benefit from this tweak. Older machines it may be quite beneficial. And the tweak may be quite beneficial to those running 24/7 (where internal fragmentation may be an issue) and keeping many windows open all the time, and doing lots of file I/O.

    Do most people fall into the latter category above? No. So the tweak would likely be a waste of time for them. People doing audio/video production on the computer who never reboot and who like to keep lots of windows open might like this. Of course one seriously into AV probably already knows about this tweak and has 4 GB RAM with a 3 GB RAMdisk for editing scratch space and maybe their page file..

    One other place the tweak may be beneficial is on a 24/7 database/web server that is experiencing lots of hard page faults despite having sufficient RAM. The more hard page faults you have, the more internal fragmentation you'll have, and the quicker things will slow down. Running 24/7, I can see always eventually reaching a state of being badly internally fragmented. It might take a month, but eventually it will happen. The father of a friend of mine runs a small ISP/web hosting company on Windows 2000 Server. He has to reboot the web server every morning because it slows down too much. Internal fragmentation may be the cause and if so, that machine would be a good candidate for the tweak.

    I run 24/7 on my Windows 2000 machine, and the gradual slowdown was the whole reason I'd reboot. Sometimes I'd have two reboot twice a day if I was really hammering the machine hard. I'm on day 5 now with no slowdown. :)

    Now imagine if XP had a smart placement algorithm for new pages being written to the pagefile, so that it does a best fit and the data is contiguous. Also imagine if XP automatically defrags internally during times of low use. The pagefile would remain reasonably internally unfragmented. Has anyone found any documents saying XP is doing this? Or any third party program that does this?

    Why you are seeing a slowdown or no improvment with some operations? I dunno.. it is interesting though. Now I wonder what would happen if you did the same tests after adding more RAM, say 1 GB to the RAMdisk pagefile and 1 GB to the OS?

    Thanks again for experimenting. We wouldn't be called tweakers if we didn't occasionally try crazy things..

    RAM to back up your RAM... does sound nuts.. hehehehe..
     
  13. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    six...no user uses a computer the same.

    as far as a slowdown due to a fragmented internal pagefile.

    on a server, running 24/7, which couldn't afford an abundant pagefile...yes, an internally fragmented pf might indeed present a performance hit..

    if they are using ntfs, the pagefile would only have to be about twice as large as the peak commit charge to circumvent any degradation due to internal pagefile fragmentation...as you know, fragmenting only affects ntfs when the drive is about 85% full...so, you would make sure your pf is twice your peak comiit charge, and the issue should dissapear

    now, to make a statement on behalf of this poduct.

    there are general practices that are counter productive for some users.

    these users can do whatever they think is neccessary to resolve and personalize their efficiency.

    so, I grant...there might be people that have benefit to making sure xp does not put the ram anywhere but whith the windows that they open...I can see, in this fassion, the current work will be more responsive.

    the trade...other work being less responsive, could be worth it to some people...and this ram disc might serv their purpose.

    I'm guessing ms should give some gui, where you can let the os look for ram uses as it does, or you can have it only use the ram on the specific windows a user opens.

    not a bad thought, though most users would slow down, some users would speed up...and that's your point.

    good conversation six...I'm going to move on...(unless claims are made as facts, when they are not)
     
  14. sixthofmay

    sixthofmay Guest

    One note:
    It's probably best to do the tweak only if you already have sufficient RAM, such that the working sets aren't trimmed as much. But I wonder, does the OS really trim less when you have more RAM for the OS? On the older OSes, I don't think so. XP may be doing this. I'll have to experiment with this. So to really be a fair test, don't take any RAM away from your OS. Buy more memory and double your RAM before doing the test.

    Interestingly, my Commit Charge is at about 412 MB right now and I have 368 MB available for the OS. Even though about 246 MB is currently paged onto my RAMdisk, I'm still seeing much better performance than when I had 768 MB total for the OS.

    YMMV..
     
  15. Perris Calderon

    Perris Calderon Moderator Staff Member Political User

    Messages:
    12,333
    Location:
    new york
    Re: Pagefile minitor

    muwahahhahahah

    now, I've got you.

    muwahahaha
     
  16. sixthofmay

    sixthofmay Guest

    "good conversation six...I'm going to move on..."

    Ya me too. I'm supposed to be studying for a math test... [​IMG]