How to limit cache in XP

fungiver...that is entirely correct, perfmon is demonstratting the potential more then the actual...it is creating addresses, for which it will possibly use in the future
 
Hello again Dealer. I do need to apologize to you. In fact XP is setup to use pagefile. It simply has to have it. If though you have 512MB+ of RAM the Microsoft people claim you only need a pagefile of 100MB. You are correct when you say that RAM "has" to be used by the OS. The OS will assign duties to RAM even if those duties are repetative. Kinda like those workers sweeping the floor even though the floor is already clean. But the system will assign new tasks if they arrive and then if all the workers have something to do it will go to the pagefile to get more workers. Cool. A tip of the Hat to you. You really know what you are doing and thanks for tolerating my misassumptions. A lot of this is based upon a stand alone system with only one user. The more users the more pagefile you need and if you have fast switching enabled then you need to set it on both sides. Two thumbs up dealer. I am indepted to you.
 
fungiver...thanx for doing the resarch

as far as the os only needing 100mb.

this is optimistic, and that is all

since there is no penalty for having a pagefile that is too big, and there is quite a penalty for having a pagefile too small, experimenting with a pagefile that is smaller then the default is an experiment that at it's very best, does absolutely nothing.

and of course, if the experiment is flawed, and you have underestimated your pf use, you will suffer performance.

in the end, there is absolutely no reasin to lower the pagefile below default.

default is system managed.

the system will try to give you 1.5 ram.

if you are short on disc space, the system will actually try to give you as small a pagefile as it can stand.

therefore, even if you are short on disc space, there is no reason to lower the initial minimum, and obviously, absolutlely never take away the systems ability to get a bigger pagefile when it needs one

I'm glad you did the reading, fungiver.

welcome to this board, and hope to see more of you
 
All good ideas with flaws. The page file is desighned to hold aditional memory mappings to the regular ram, but the page table to map to the memory is quite large and so can be the alocated memory in ram. There can be a split between paging levels were the page file comes in which is basicly a last resort maping to alocated memory. Modern programs ore hungry for ram but that can be controled in ram by alocating only needed memory as needed as it will be fast in ram and with enough ram the page file is not needed, take the pree pagefile systems from long ago, add more ram and they run forever and when the other program loads it takes needed memory by placing inactive ram memory to the respective hard disc points (not into the pagefile) and hold just wnough in physical RAM. All this can be done with some work which I don't need to do now.
With 256 MB I had little to no problems nor slowdowns with 384 MB PF (1.5xRam) so wy not trick windows to use half the ram normaly and the other half as the page file with 128 more megabytes on the hard drive, I would get the same memory as before without most of the slowdowns of using the pagefile from before, but windows takes the 512 MB ram. But since I use the same programs, 256 MB should be enough as programs sholuld not alocate more RAM then before but instead windows badly takes it, mostly with cache.
Disk cache to be specific which is basicaly writeback cache for normal disk writes that can stay in RAM instead of being writen to disk. If those addresses keep being used then only the RAM is used and everything runs slowly. But as more is stored in cache less is available for programs, so when a program needs to run, all the cache must be written to the hard drive in one large MEMORY DUMP (as I call it) slowing down the system A LOT which I believe is happening to me as it reaches a point where cache is always emptied out and the system runs very slow, (not with the pagefile). With 256 MB there was more of a balnce such that the cache was writen out more often or more eficiently on idle times alowing the RAM to be freed quickly. Servers need lots of ram and cache as it usualy doesn't run many programs but does have many disk accesses hence it stands to gain a lot from the cache and does not lose much cache dumps to emory as it whould need to use the hard drive anyway. I on the other had with many programs tend to slow down. Cacheman has setings that specify cache and ram priority setings which show that the cache is confugurable but it only states priority so eventualy almost all ram becomes cache the settings only control how quickly that happens and when the ram will be released. Putting cache in pagefile will only put a double slowdown buy fist going to disk for the cache and then again to write the data.

What I want is to limit and/or control the usage of the RAM as cache as I want to actualy have a peek and a gradual write directly to the disk and not through the page file.
So if you realy have enough ram to alocate data for programs then you don't need the pagefile even with the extra space but if windows keeps taking to much cache space then everything goes south.
Cacheman alowed cache fixing for win 9x, I want something similar in XP as it clearly cannot handle it on its own.

Thanks
 
sorry, elkinm, your assumptions are not correct, as I've already enumerated, so this becomes circular.

I can't help you accomplish what you're looking for, and as i've already said, when you do accomplish what you believe you want to accomplish, your computer will be slower.

now, if you really believe what you've posted, just decrease your ram, and then you'll be so much faster, and more ballanced, as you've documented for yourself

so that would be your solution.

I might even try this myself...I'll lower my ram, so everything is more ballanced, and of course, that much faster

beyond this suggestion, someone else may be able to help you
 
Well Dealer I am back. I have found that while you are indeed MR. Pagefile some of what you say does have it's flaws. You CAN force XP to use the RAM and thus limit the need for very much pagefile. I do want to specify that this is only in the case of a single computer not networked or used as a server. While a pagefile is indeed needed as XP is designed to page, allowing to large a pagefile results in slowdowns. Witness the fact that I did what you suggested and went to a "system managed" size and my shutdown times went from 7 seconds to over 1 minute with absolutely 0 performance increase. After a while the system had boosted my pagefile to over 764MB. Why? Simply because it could. Now not only does my system allocate useless activity to the RAM but because it has a large pagefile it does the same to it. Still reading but so far all tests have shown that while a page file is indeed needed limiting the pagefile does result in a faster system. I have since gone back to a start of 50MB and a max of 150 MB as this seems to be the optimum for my useage. This of course is by no means the norm. This is based solely upon my usage and what I do with the computer. Everyone will be different. The jury is out but I have days of reading to do. For now I will simply have to defer to a higher authority. You Mr. Pagefile AKA Dealer. lol
 
your shutdown time increased because you are overwriting the pagefile with zeros

a waste of time, by the way...the purpose of this "clearing the pagefile" , was an effort to circumvent anyone from reading the pf...it doesn't work, and alll this zero writing does is slow you down...but that's another thread as well.

obviously, the bigger the pagefile, the longer this overwriting will waste your time

as far as increasing the pagefile...sorry, the system will give you 1.5 if it's available...if I remember, you have 512 ram, so the system gave you the correct size...764

not because it can, but because this is a fine beginning...that is the reason the system gave you 1.5

microsoft has rearched this...every single nt generation has again increased the minimum recomendation...

this is from the research that ms has benchmarked.

now, in order for the pagefile to increase beyond 1.5, the mechanism that causes this is the commit charge, versus the commit limit.

if your pagefile is increasing, it is for this reason, and no other...which would suggest suggest a memory leak if the system is giving you more then 1.5...but that is not the case..the case is, the os has managed your pf to the correct value.

for your needs, go ahead please, and set the pagefile to whatever value you believe will help you...that's the answer in the end...to run the box the way we as individuals want to
 
True I can set the box to whatever way I want to but that is not very enlightened. While you are very knowledgeable I trust you do not get upset at us newbies challenging your thinking. This is the way I learn and, trust me, I have learned more from reading this post and your threads than from any other source. Someone suggested I create a swap file to test your theory but I think what I shall do instead is undo the tweak "clear pagefile on shutdown" and see just how much quicker the shutdown is. Once again please don't take offence to my questions they are a sincere effort to understand this issue from someone who it appears knows his stuff. Thanks Dealer. There we go. Undone now to test. God I love this banter. Kinda gets the old mind lubed up and the blood pounding. Have a good night dealer.
 
fungiver

thanx for the kick in the pants

I agree, I think I got snappy...sorry

on the other hand, we are going in circles, yes?

let's have banter on another thread.

or, if you'd like, feel free to pm me...talk to you soon
 
"How to limit cache in XP"

Exclude a little more than half your memory and create a ramdisk in the excluded space. Put the page file in it. You probably need at least 512 MB total RAM before you try this. Results on my Windows 2000 system have been nothing short of amazing. Try it and post the results please.

http://forums.storagereview.net/viewtopic.php?t=6807

*Read the whole thread* before you try it, especially my postings. Note the performance tests I did too. Cool eh?

http://www.cenatek.com/product_ramdisk.cfm

XP notes from their faq:
Will RAMDisk NT/2000 work with Windows XP?
We have heard from some users that RAMDiskNT/2000 works on XP. This is not our official position, however, because we have heard from some users that they have had problems with this. We are working on full XP compatibility at the moment and will have an announcement it in the near future.

In the meantime, you are welcome to try out our trial version to determine if you are among those for whom it works. Our formal suggestion is to wait until our XP version is released.

I want to beta test RAMDisk XP! May I get a copy?
We actually have enough beta testers for RAMDisk XP. Thanks for asking. Please watch for our final version to be released soon!
 
six...a ram disc for the pagefile will slow you down.,

what happens, is ram is used for the disc, (obviously), and you run out of ram sooner, and therefore, you begin to use page sooner...a little obsurd, don't you think?

by the time the ram disc is then used up, you would have otherwise still be using ram if you didn't tie up your ram with the ram disc for a pagefile.

in the end, you will use hardrive more with a ram disc as a pagefile then without
 
"six...a ram disc for the pagefile will slow you down.,"

Obviously you didn't bother to read much of the storagereview thread I linked to. READ THE WHOLE THING! Especially my *TESTS* posted near the end of the thread and the links I have referenced (ignore Jeh on the 2CPU thread...). You'll see in my tests that responsiveness whether the system is loaded with tasks or not is improved, and quite dramatically if it is loaded. The product manual for WarpNT is very informative and well summarizes the issues. The manual was written for NT4, but the VM material is still relevant. Note I do not use or did not try WarpNT and don't know if it would work with XP. I just liked the info in the manual. My experience is with RAMDiskNT.

"what happens, is ram is used for the disk, (obviously), and you run out of ram sooner, and therefore, you begin to use page sooner...a little obsurd, don't you think?"

What you are not grasping is that when doing large file I/O operations (multi GB sized operations), Windows will fill all available RAM with the disk cache. Including resorting to stealing inactive memory pages to then be used for the disc cache. I DON'T WANT THOSE PAGES PAGED OUT. The MS definition of "inactive" is somewhere between 5 and 10 minutes. My definition is days or weeks. When I click on a button on the task bar, I DEMAND that the Window pop up immediately, regardless of system load, and regardless of how many hours, days, weeks have gone by since I last clicked on it. Try browsing through 25 paged out browser windows on the task bar while copying a few gigs of files and doing audio video editing which is pegging the CPU. You'll see that each one takes a while to pop up. And in my case, the GUI locks while waiting.

It doesn't matter how much RAM you have. Got 4 GB? Copy 4 GB worth of files. Windows will cache them and page out inactive pages. The OS might think they are inactive, but to me, I'm still using them! I've got 768 MB of RAM and don't want them paged! The ramdisk kludge *FORCE FEEDS* the RAM back to the OS and effectively solves the problem. The root of the issue is that the NT virtual memory system is general purpose and not very smart or tweakable (*read* the WarpNT manual). *BEFORE POSTING THE KNOWN VM REGISTRY TWEAKS*, understand that I've tried them and they did nothing, that they have been covered in the other thread and elsewhere for years, and no one has ever reported them having any effect....

So you don't push your computer that hard or maybe you have a 2 GHz dually with SCSI I/O which will help mask the issue? If that's true, goody for you... I'm a broke college student trying to get by with some relatively old hardware.

"in the end, you will use hardrive more with a ram disc as a pagefile then without"

My page file is entirely in RAM.

I ask that you carefully read the links I posted before replying. Perhaps even try the tweak? If you try it back up your boot partiton with Ghost or DriveImage just in case.. I'm not sure how XP will react if that's what you are running.

I apologize if I seem a little annoyed. While trying to convey my experience, I had several people in the other thread not bother to read carefully what I posted and the links I referenced before posting nonsense that was already covered. ***AND NONE OF THEM ACTUALLY TRIED IT***, so who are they to say it doesn't work? I know of only one other person who is doing the tweak. His results are the same as mine.

I really would like some others to give this a shot and report back their experience, good or bad. I'm especially interested in those with older 700 MHz or less CPUs in their systems. My gut feeling is the lastest systems are so damned fast that they tend to mask the issue, and would not show improvements as dramatic as I have experienced.

PS. please read the links.
oh and read the links.....
did I mention to read the links? the links are your friend.. they'll make you feel good.. feel good.. feel.....
 
six...as everyone knows, I love to read...I will read your links, and refrain from further comment then this post, till I've read the links.

I have to tell you though, I have some trouble following your points, so this will take some time for me.

so, for now, my initial response, where you've just said this;

I DON'T WANT THOSE PAGES PAGED OUT. The MS definition of "inactive" is somewhere between 5 and 10 minutes. My definition is days or weeks.

and this

It doesn't matter how much RAM you have. Got 4 GB? Copy 4 GB worth of files. Windows will cache them and page out inactive pages. The OS might think they are inactive, but to me, I'm still using them! I've got 768 MB of RAM and don't want them paged!

first of all, you are misslead here...the fact that they are written to page does not at all mean that the information is lost from the ram that it was written

the opposite is true, and the ram that this information camefrom is still in the ram, untill the ram is claimed, which will happen sooner if you are using ram for the pagefile

you are obviously looking for differant speed variables then I am, and the majority of users...when I have an app sitting for five or ten minuts, sorry, I want that ram to be the first available for release.,,you want it to be resident in your ram disc...where it would have otherwise been more easily available if it were in the ram that the disc is taking for it's creation.


as far as not wanting the information paged...you too are misslead...the information will always be in ram untill the ram is claimed...so, you are using ram, to store ram, which wasn't claimed in the first place..you're using twice as much ram in some cases as you would if you left the ram available, and let the hd do it's job

I will ceed your point so far as follows;

there might be users that can find some benefit to using there ram for a ram disc, though I've known many that have tried this idea, none have had the positive experience that you have..and none have been able to document reprduceable benefit

I will read your links..and see if I can follow the info,

thanx for providing them
 
ok...did some reading, though can't really go through all of it, as I said, I do indeed have trouble following alot of what you are trying to post

my feelings stand, what I've learned counter indicates your claims...your first post looks exactly like you are advertising this product, and you make the same inacurate complaints about the pagefile being used instead of ram...thsi doesn't happen

as you can see in alex nicholes papers, xp will attempt to make use of every bit of ram available...this will obviously speed you up.

well, the ram disc you are trying to promote cricumvents this strategy, and, it obviously eliminates ram from uses the os would make of it...so this is the part that you like...I would hate it.

I will grant that there might be the occasion where the ram disc might seem to speed something you're working in...

but, this is obviously at the price of something else you are working on, or somewhere else the os would have assigned this ram...exactly like the ram management programs claim to do...this is easy to follow, isn't it.

now, for your benefit, six, I will admitt, what I've learned may be inacurate...or missinturperated,

so I invite anyone that thinks paging ram to ram (this is as redundant as it sounds...obviously, and there will be times you are using twice the amount of ram you would be using without the ram disc), is a good idea, to please give it a shot...

I will not be giving this a shot

I will also ceed, I am passing a judgement without trying your product.

I will not commit ram to the purpose of storing other ram...not yet, anyway.

yes, I do have those associates that have tried commiting there ram to store ram,as this program suggests...to no benefiit

my feeling by the way, is that your are somehow involved with this product...I wish you good luck with it
 
Dealer, please read all of this and I hope you see my reasoning:

First off I have no affiliation whatsoever with Cenatek and I am running a demo copy of their program. You'll note on their web site that even they say there is little benefit to a ramdisk pagefile. Which I've discovered is not quite true. If you read carefully the whole storagereview thread, and note all my postings, it should be pretty clear I have no affiliation with anyone, and I am just one somewhat wacky and excited person that has finally been able to implement a fix for a long suffered with problem (5 years now.. I had the same issue with NT4, and I thought about this fix back then but RAM prices were insane and no suitable large sized ramdisk program existed).

Dealer wrote:
"well, the ram disc you are trying to promote cricumvents this strategy, and, it obviously eliminates ram from uses the os would make of it...so this is the part that you like...I would hate it."

My programs typically only total 150 MB. Why do I need a disk cache of over 500 MB???

[vcache]
MinFileCache=16384
MaxFileCache=16384

Remember this system.ini tweak to Win9x and why it was done?

It was to prevent the OS from using all the available RAM for disk cache which then caused paging of the apps. Sound familiar to what I've been saying? The NT kernel does the same damn thing only there is no nice and simple fix. Ever wonder why the tweakers say to put your pagefile on a separate disk drive to improve performance? I did that and it helped, but still not enough. So I've taken the tweakers' advice a step further and put the pagefile on the fastest drive I could afford- a ramdisk. Why does everyone think this is a bad idea? I honestly don't understand the confusion. It's a no brainer to me. How about if I posted that I just bought a $2000 solid state diskdrive to put my page file on? That's certainly a fast drive like the tweakers suggest. How is using a solid state drive different from using a RAMdisk for the page file?

1. Price obviously. Computer RAM is dirt cheap.
2. The RAMdisk will likely be faster since it's not going through the PCI bus.
3. People would probably just think, "lucky rich dude now has an even faster computer" rather than "that's insane to use system RAM for a pagefile".

Are you beginning to understand the reasoning yet?

I have maybe 150 MB of programs I'll typically load into RAM. I have 768 MB available. Why not take half that RAM and use it for a very fast drive? I certainly don't need a disk cache of several hundred MB in size. Since Windows insists on filling up all my RAM with cached files, and insists on paging out programs I'm still using, I'm going to need the fastest page drive I can find. Even though I only need to use those programs a few times an hour, or sometimes much less often, I want them *instantly* available. Since my programs are only using a fraction of my available RAM, why the heck would I want the OS to page them out? Why does the disk cache have priority to the RAM? Shouldn't apps have priority? I DON'T EVEN USE THE DAMNED DISK CACHE ANYWAY. The majority of my file I/O usage is *RANDOM*. Stuff that's not in the cache. That's why I have a fast RAID array. What were they smoking when they designed this VM system?

A quote from WarpNT's manual is relevant:
--------------------
WarpNT allows the user to allocate physical RAM, and to use this RAM to create a Cache disk, for acceleration of the systems management of its virtual memory paging file. This can have fantastic speed and operational advantages. NT uses a file (usually on hard disk) to implement virtual paging for memory extension and sharing of process space for multi-tasking.

Without WarpNT, the system relies on slow disk I/O for all paging activity. Also, as demand on the paging file increases, the disk activity slows down processing overall. WarpNT can be used to move the paging activity to RAM, thus alleviating this problem.
--------------------

Again note I'm not using WarpNT, nor have tried it. I'm just quoting some relevant stuff from their manual. WarpNT may work just as well as RAMDiskNT and it's less expensive at $14 a seat.

Summary of the tests I did (test methodology/system config is in the storagereview link posted previously):

Creating a new folder (OS resources for this task are paged out):
Page file on harddisk, CPU and I/O idle : 6 seconds
Page file on harddisk, CPU and I/O loaded: 16 seconds
Page file on RAMdisk, CPU and I/O idle : 1/2 second
Page file on RAMdisk, CPU and I/O loaded : 1 second

Switching to a paged out IE window on the taskbar:
Page file on harddisk, CPU and I/O idle : 3 seconds
Page file on harddisk, CPU and I/O loaded: 19 seconds
Page file on RAMdisk, CPU and I/O idle : 1/2 second
Page file on RAMdisk, CPU and I/O loaded : 1 second

Opening a new window in Internet Explorer with ctrl-N:
Page file on harddisk, CPU and I/O idle : 4 seconds
Page file on harddisk, CPU and I/O loaded: 17 seconds
Page file on RAMdisk, CPU and I/O idle : 2 seconds
Page file on RAMdisk, CPU and I/O loaded : 3 seconds

Opening the Control Panel:
Page file on harddisk, CPU and I/O idle : 5 seconds
Page file on harddisk, CPU and I/O loaded: 14 seconds
Page file on RAMdisk, CPU and I/O idle : 2 seconds
Page file on RAMdisk, CPU and I/O loaded : 3 seconds

Switching to a paged out NetOp Remote Control window to another machine on the network:
Page file on harddisk, CPU and I/O idle : 6 seconds
Page file on harddisk, CPU and I/O loaded: 26 seconds
Page file on RAMdisk, CPU and I/O idle : 1/2 second
Page file on RAMdisk, CPU and I/O loaded : 1 second

Browsing a directory with 11,444 files, time for hourglass to go away:
Page file on harddisk, CPU and I/O idle : 5 seconds
Page file on harddisk, CPU and I/O loaded: 21 seconds
Page file on RAMdisk, CPU and I/O idle : 2 seconds
Page file on RAMdisk, CPU and I/O loaded : 9 seconds

Not only are the RAMdisk numbers impressive, my computer feels much more responsive doing anything. I find myself actually loading programs on a whim now. Before, I'd have to think, what kind of impact is this going to have on the responsiveness of the system. Do I really want to wait for it to load? I avoided playing videos just because Media Player would take so damn long to start. I can now switch to any task on the taskbar instantly. The OS now works properly as it should have in the first place. I finally have an OS as fast as Win9x but with the stability of the NT kernel! Just look at the times above when running the pagefile off the hard disk. Imagine suffering with those slow response times for years and then finally being able to implement a fix dreamed up years ago. Makes one giddy and a little batty..

To summarize:
If you are not seeing any slowdowns, don't bother with this tweak. If you notice your computer getting slower and slower after a few hours or a day or two of operation, then this tweak will fix it. If you do a lot of file I/O per hour and notice unusual lags when switching to other windows, when starting programs, or creating new folders, then this will fix it.

Well that's pretty much it folks. If anyone else trys this, please post about it.
 
hello six

thank you for changing the tone...I appreciate that.

I don't want to sound redundant, and I don't want to insult you. but, you are posting as fact, some activity that does not take place.

then the conclusions follow from your inaccurate assumptions of what the pagefile is.

this paragraph;

have maybe 150 MB of programs I'll typically load into RAM. I have 768 MB available. Why not take half that RAM and use it for a very fast drive? I certainly don't need a disk cache of several hundred MB in size. Since Windows insists on filling up all my RAM with cached files, and insists on paging out programs I'm still using, I'm going to need the fastest page drive I can find. Even though I only need to use those programs a few times an hour, or sometimes much less often, I want them *instantly* available. Since my programs are only using a fraction of my available RAM, why the heck would I want the OS to page them out? Why does the disk cache have priority to the RAM? Shouldn't apps have priority? I DON'T EVEN USE THE DAMNED DISK CACHE ANYWAY. The majority of my file I/O usage is *RANDOM*. Stuff that's not in the cache. That's why I have a fast RAID array. What were they smoking when they designed this VM system?

is just loaded with incorrect assumptions, therefore asking questions that don't exist...the statement that if you load 150MB of programs, you therefore have 768MB of ram doing nothing is flat out opposite if actuality

in this case, when you make a ram disc, that is when you will have 768MB of ram doing absolutely nothing.

you are creating the very issue that you incorrectly believe is occurring

read again Alex Nichole's explanation of how much ram you have doing nothing..in XP, your system will make use of whatever ram it can make use of.

if you want to use ram for long term storage purposes, instead of assigning the ram wherever possible, this is obviously circumventing the strategy of where the ram would have otherwise been used.

you missed my point...of course you will find occasion that your strategy could show a faster performance...but, it is definitely at the expense of performance in a norther process.

I said I cede, and that there might be occasion where the XP strategy for ram use is not ideal for a specific purpose, and so, I also say, if a person believes, or experiences their harddrive being used before their ram, they can try this product.

this btw, six does not happen..it is just a perception that 9x people believe...all of these memory issues from 9x are addressed in this article...these issues do not apply at all to xp

my harddrive is never used before my ram...when there is occasion that my box slowed down, die to a cached process, it is because my box was running that much and more faster, since the OS assigned that ram to what I was doing, instead of what I was waiting to do.

you are making another mistake, and comparing a 9x OS and philosophy with XP.

six, XP is not 9x...you can not make any such comparisons

you know six, You can get the same what i believe is an illusion of this benefit you are experiencing with a simple memory management program

these programs will release the 768 MB of ram that you would like to be available for other use besides cache

so the ram would be hangn out, waiting for your bidding, instead of being assigned to other processes.

this would give an even faster use of the ram then you are suggesting, as you are suggesting to take ram from immediately being used, and use it as the longterm storage device that is the pagefile.

six...i have no problem with anyone experimenting with this ram drive, so that you know that.

I can see why the idea of it would make sense to a 9xer, so those people that believe XP is not putting your ram to the best use, should definitely experiment with reassigning the ram use the way you prefer it would be used.

I can't debate this further, for as you say, I have not personally experimented with this idea, so I leave this discussion to those of you that have.

six...welcome to XP_erience

good to meet you



a
 
"you are posting as fact, some activity that does not take place."

After reading many threads and web pages on the subject over the past 6 years, and experiencing it on all 9x/ME/NT4/2000 systems, I must disagree with you. Important note: XP *may not* suffer from the problems, thus explaining your experience. I run Windows 2000 now and have little experience with XP, but was curious about its VM system, that's why I came here in the first place. XP may be less agressive on trimming working sets. XP may well have automatic pagefile internal database defragmentation as well as pagefile filelevel defragmentation. MS does claim to have improved the VM system so it may be true:
http://www.microsoft.com/HWDEV/driver/XP_kernel.asp#Memory
Speaking from experience, I can assure you that the older OSes have no pagefile optimization and are much too agressive at trimming working sets and paging them out.

"if you want to use ram for long term storage purposes, this is obviously circumventing the strategy of where the ram would have otherwise been used."

A file cache of 500+ MB is no benefit to me or the majority of users. Keeping apps in RAM *IS* beneficial. And I've got plently of RAM, so there's no reason to trim the apps' working sets. They should only be trimmed if free RAM is low (and not so the OS can build up a massive file cache that I'm never going to use...).

"my harddrive is never used before my ram"

What CPU are your running? I've noticed paging is much faster on Athlon/Duron/Pentium IV machines. Do you do much file I/O? If you aren't doing several GB of file I/O per hour and/or have a fast machine, you won't likely experience the problem. And if you are running XP, as I stated above, the problem may well be fixed.

"you know six, You can get the same what i believe is an illusion of this benefit you are experiencing with a simple memory management program"

These programs *DONT* work. I tried O&O Clevercache, Systernals Cacheset, and Outer Technologies Cacheman. Tried the default and various custom settings with those programs. And I tried the registry tweaks that are listed on this site and others. Nothing had any effect. Slowdowns persisted.

"you are making another mistake, and comparing a 9x OS and philosophy with XP."

I didn't mean to compare it with XP. The VM system in XP may well be fixed.

One question of mine remains:
"How is using a solid state drive different from using a RAMdisk for the page file?"
I noticed you didn't answer this one.
biggrin.gif


Here's one of the best discussions I've found on the pre XP Windows VM systems. If you're curious take a look at the complete thread:
http://groups.google.com/groups?hl=...=34ec79c4.44008984@msnews.microsoft.com#link1

A few postings from the thread:

---------------------------
From: Wayne J. Hyde (wjh@cise.ufl.edu)
Subject: NT's braindead VM subsystem
Newsgroups: microsoft.public.windowsnt.misc
Date: 1998/02/18

Ok, I've done some more testing. This is getting *really* depressing.
NT's memory management subsystem is completely braindead. Come on,
should I really have 87MB paged out when I have 113MB free and 192MB
physical RAM? Every time I minimize an application, NT trims the
working set like mad. This is ludicrous. NT's performance could be
so much better, yet it is crippled by such a poor VM subsystem and
paging method (FIFO).

It is very easy to check for yourself just by using NT's Task Manager.

Load up Task Manager and view the "Processes" information. Change
your update speed to "low" and select the following columns: Mem
Usage, Memory Usage Delta, Page Faults, Page Faults Delta, and Virtual
Memory Size. The update speed should be on "low" so it only updates
every four seconds or so -- that way you have time to see what
happens.

You can also use the program 'pmon' from the Resource Kit. It pretty
much shows the same information except it is a command-line utility.
Task Manager is nice because you can sort columns, etc. You may also
want to load up Performance Monitor and make a graph with "Page
Reads/sec" and "Page Writes/sec" from the "Memory" group. Now, you
will be able to see what NT is doing with memory when you use an
application, minimize it, maximize it, etc.

My machine has plenty of RAM (192MB) and swap (400MB). You would
think that NT wouldn't need to do much paging on my machine until I
started overcommiting RAM. "Not so fast my friend." NT doesn't wait
around until you really need to page; it pages out long before you run
out of RAM. Not only that, It will page out a process as _soon_ as
you minimize it. No joking here.

Load up a bunch of your applications. Right now, my commit charge is
over 150MB. Netscape, MSVC++ 5.0, Outlook, Word, Backup Exec, What's
Up, a few copies of Forte Agent (for different servers), GateKeeper,
Diskeeper, Exceed, a few hostexplorer's, mIRC, and a bunch of other
apps are all loaded. I've sorted my Task Manager window by "VM Size"
since the largest apps will usually cause the most paging. Now for
the fun...

Right now, Netscape is taking up 2380k of RAM and 14MB VM. It is
minimzed with three windows total. When I restore one of the windows,
the following occurs:

Memory usage Delta jumped to 2168k.
A bunch of page faults occured.
Performance Monitor showed that pages were read from disk (meaning
that they weren't in the standby list)

If I restore the other Netscape windows, more pages are swapped back
into RAM. (It jumped back up to 5504K resident) So far this doesn't
look bad, especially if you don't know how NT is handling RAM. It
becomes apparent that NT is braindead once you minimize an
application: NT __immediately__ trims the working set down to zilch.

After minimizing Netscape, the following occurs:

Memory Usage Delta is -3756K (NT trimmed the working set by 3.7MB)
Memory Usage dropped to 1748K from 5504K

If I restore Netscape once again, it pages 2184K back into the working
set. If I do this immediately after I minimized Netscape, the pages
will usually be in the Standby list and NT won't have to go to disk.
Microsoft claims that this is a good thing since a "soft" page-fault
can be resolved [relatively] quickly compared to a "hard" fault. I
think it is incredibly dumb because NT is wasting cycles on too many
damned page faults. Why trim Netscape's working set immediately after
the window is minimized? *especially* when I have so much RAM
available. (104MB right now)

NT exhibits this behavior for just about every application. I just
minimized MS Outlook 98b and NT trimmed the working set by 4.6MB (down
to 1.3MB). After restoring the window, NT pages the memory back into
the working set.

I just minimized Lview Pro (an image viewer). The Mem Usage dropped
from 9576K to 328K! A drop of 9248K! Yes, that is 9 MEGABYTES
trimmed from the working set. I noticed some Page Write/sec in the
Performance Monitor, so it appears that NT had to write the updated
memory pages from LView to the pagefile. I wont immediately restore
Lview since I want to see how long the pages stay in the standby list.


Now, lets see what happens when a bunch of applications are restored
at once: Netscape, Word97, Outlook98b, HostExplorer, Agent, POV-Ray,
and bookshelf basics.

App: Mem Usage (before) Mem Usage (after)

Netscape: 1748K 5260K
Word97: 572K 1832K
OutLook98: 1220K 2700K
HostExpl: 268K 980K
Agent: 1568K 1588K
POV: 384K 1076K
Bookshelf: 684K 1528K

As you can see, NT paged in quite a bit. The pagefiles were hit also
as reported by PerfMon. I minimized the applications and guess
what... yes, NT immediately trimmed the working sets. NT also paged
out some other pages to make room for the programs being swapped in.
I guess NT has something against trying to use _FREE_ RAM. No sir, it
would make too much sense to use free RAM to swap memory back in. NT
needs to swap more data out to make room. Just in case you run a
program like clearmem.exe from the reskit. Yeah, that happens a
bunch.

It is sickening. I've got 116MB RAM "Available" right now and NT is
paging out programs to disk as soon as I minimize them. And of
course, I restored the previous LView, and it had to page in from the
pagefile -- meaning that the pages were not in the standby list.

Now is it just me, or is this just a _very_ poor design? I've got
*gobs* of RAM -- more than enough to hold all of the programs I am
currently running without paging out a single page, yet NT is swapping
like there is no tomorrow. My Pagefile in use is 100MB right now.
Peak is 170MB.

NT handles CPU-intensive applications pretty well. It just doesn't
know what the hell to do with RAM. Perhaps I would have better
performance on a system with much less RAM. Maybe then NT would stop
paging everything out to disk as soon as it is minimized.

Maybe I just haven't been told by Microsoft how a "workstation" is
supposed to behave and perform. Maybe I'm just too used to working on
my Solaris machines at work where the OS doesn't leave half of my
memory wasted.

-Wayne
---------------------------


Interesting eh? How about this:

---------------------------
From: Tim Hill/MVP (timhill@pacbell.net)
Subject: Re: NT's braindead VM subsystem
Newsgroups: microsoft.public.windowsnt.misc
Date: 1998/02/18

>Solaris doesn't go crazy trimming working sets and paging out until it
>is necessary. NT pages out immediately, whether it needs to or not.

"until it is necessary"??? And when is that? When the pages are needed,
that's when. But by that time, it's *too late*. So any OS (Unix included)
has some form of background swap-out algorithm which frees pages which are
not seen as needed (the WST, typically). The trick, of course, is choosing
(a) which pages to swap and (b) when to swap them.

One of the choices the NT designers made was to trim an apps working set
when an app is minimized. The rationale is that when you minimize an app two
things happen: (a) you've finished with it, for the time being and (b)
you're probably about to run another app. So NT pre-emptively trims the
working set. Sometimes this is correct, sometimes not. It's a statistical
thing, and in this case a UI thing. I'm not sure I agree with this decision,
but I can certainly understand it, and saying it's brain-dead is pretty
silly.

Another thing to understand. NT itself isn't actually responsible for
trimming the working set of an app when minimized. The app actually does it
during processing of the minimize request. The Win32 sub-system tells NT to
trim the set. NT does so. The fault here (IMHO) is that the Win32 code
should be more intelligent, and not request a trim unless it sees (say) >70%
RAM used.
---------------------------


"Sometimes this is correct, sometimes not."
In my case it's never correct. When I minimize a window, I most definitely am *NOT* done with it and don't want its working set trimmed and then soon written to the pagefile. When I'm done with it, I'll close it myself thank you.

My post continues below due to the 15000 char posting limit:
 
And the reply:

---------------------------
From: Wayne J. Hyde (wjh@cise.ufl.edu)
Subject: Re: NT's braindead VM subsystem
View: Complete Thread (26 articles)
Original Format
Newsgroups: microsoft.public.windowsnt.misc
Date: 1998/02/18


On Wed, 18 Feb 1998 12:59:04 -0800, "Tim Hill/MVP"
<timhill@pacbell.net> wrote:

>>Solaris doesn't go crazy trimming working sets and paging out until it
>>is necessary. NT pages out immediately, whether it needs to or not.
>"until it is necessary"??? And when is that? When the pages are needed,
>that's when. But by that time, it's *too late*. So any OS (Unix included)
>has some form of background swap-out algorithm which frees pages which are
>not seen as needed (the WST, typically). The trick, of course, is choosing
>(a) which pages to swap and (b) when to swap them.

I did not say that NT should wait until all RAM is exhausted to start
paging out memory. I am saying that the current implementation is
severely broken. This can be monitored using performance monitor and
watching all of the thrashing that goes on, even when you are working
with an application set which should fit entirely in RAM.

To your point (a): FIFO page replacement is stupid. Since x86 and
Alpha support LRU, why not switch. It can't be *that* difficult of a
task. Hell, Win95 has a LRU implementation. Who here is going to
argue that FIFO is a good page-replacement algorithm? It stinks.

To point (b): Once again, NT swaps way too early. In fact, it starts
swapping immediately causing a ton of page faults. Right now,
Netscape has cause over 200,000 page-faults in just over 11 minutes
and 4 seconds of CPU time. I just caused another 1000 page-faults in
Netscape just by restoring and minimizing it. Another 1000 faults in
just over 5 seconds of use. (time to minimize & restore twice)

>One of the choices the NT designers made was to trim an apps working set
>when an app is minimized. The rationale is that when you minimize an app two
>things happen: (a) you've finished with it, for the time being and (b)
>you're probably about to run another app. So NT pre-emptively trims the
>working set. Sometimes this is correct, sometimes not. It's a statistical
>thing, and in this case a UI thing. I'm not sure I agree with this decision,
>but I can certainly understand it, and saying it's brain-dead is pretty
>silly.

I don't understand it at all. NT should not immediately page out and
trim a working set just because an application is minimized.
ESPECIALLY if so much RAM is free. NT causes unecessary swapping,
which *does* degrade performance. It also causes a ton of
page-faults, which, get this, take up CPU cycles.

How about this one: even with over 100MB "Available", NT will start
paging out data and trimming working sets of other apps when I load up
a new application. Why, when NT could have just used RAM from the
free page pool.

NT could perform so much better, especially on higher-end Intel
hardware.

>Another thing to understand. NT itself isn't actually responsible for
>trimming the working set of an app when minimized. The app actually does it
>during processing of the minimize request. The Win32 sub-system tells NT to
>trim the set. NT does so. The fault here (IMHO) is that the Win32 code
>should be more intelligent, and not request a trim unless it sees (say) >70%
>RAM used.

It is not the duty of an application to do memory management.

>The numbers you are getting are interesting. I'll retry some of these tests
>with some low-level test tools I have. I'll test on a 64meg and 256meg
>machine.

I'm going to try to do some testing tonight with 64MB and 128MB.

-Wayne
---------------------------


Yep. I'm in total agreement with Wayne.

Note in my tests that even when my computer is idling, *response times are still faster with the RAMdisk*. When I load it with tasks, well you saw the poor response times in my tests... One other note: I've had the machine on for three days straight now and nary of hint of a gradual slowdown. In the past, I'd reboot every day, sometimes twice a day, because the computer would slowdown. This is because the pagefile gets fragmented internally. Note that this is *DIFFERENT* from filelevel fragmentation and has a big impact on performance. I don't know if there are any tools to correct internal fragmentation, but even if there are, an unfragmented pagefile on a hard disk is still going to be much slower than a ramdisk pagefile. A RAMdisk pagefile works just as well, fragmented or not, so thus the lack of a slowdown when using it. I have experienced the slowdown on every non-tweaked 9x/ME/NT4/2000 box I've used.

Some people have said using a RAMdisk pagefile causes other problems without ever saying what the other problems are. I find that amusing. I'm not seeing any problems. None of my apps leak badly and I'm only using a fraction of my total memory. And even if I did have a higher total memory load than 768 MB (I never do), Windows will pop up an error message saying VM is low and *DOES* create automatically a secondary page file on one of my hard drives. I'm having my cake and eating it too. What's wrong with that?

I'll leave it at that, and if you still are in disagreement, we'll just have to agree to disagree. Good meeting you too. :)
 
as I keep saying, you are using non facts, and repeating them as facts...simply as when something is repeated often enough, it becomes more true.

you would like to believe that xp is incorrectly assigning ram, you keep repeating it as though if you repeat it in deifferant forms, it is true...it is not true

and you think that using your ram as the long term storage device known as the pagefile is the best use for it...and I've said, for a secect few, for a selct number of activities,this might actually be true...I won't argue this point

I understand all your points...

I don't subscribe to the idea that xp is doing a bad job managing memory...

quite the contrary, it is often repeated that xp manages memory incredibly well...and this is the notion I subscribe to

I don't subscribe to the notion of using ram to store ram...you do.

the notion I subscribe, is my ram should be used for speeding my box as often as possible, whenever it is possible...

ou would rather use some of your ram to wait and be called upon at your whim...I want the os using ram before my whim

and that's the differance.

I agree with you on the following though, and I will end here,...

if you believe that xp is improperly assigning ram, and you think it is a good idea to store ram in ram, then you should try this product.

so we agree on that, and again, now you and I are going in circles.

when I believe that my hardrive is used before my ram is used, that is when I will give this ram disc for the pagefile notion a shot..

I have not seen my hardrive used before my ram, and I will surely not assign ram to a pagefile!!!!

here are some actual numbers;..right now, as I type

512 mb ram

ram used...354

pagefile used...398

mb written to the pagefile...19mb, (this is what you're not understanding...pagefile used does not correspond to mbs written, does it.)

now, for 19mb of information, you would have me tie how much ram?... what...however big you want me to make a ram disc...sorry, it's not going to happen




I mean just repeating the idea

"asign ram to a pagefile" and "ram to store ram" speaks volumes as it stands
...again, good luck to you and this producuct
 
"quite the contrary, it is often repeated that xp manages memory incredibly well...and this is the notion I subscribe to"

As I stated in my previous post, I have essentially no experience with Windows XP. If I previously stated that XP has the issue, I did so in error. I believe you when you say that Windows XP's memory management works fine. I don't have sufficient evidence proving otherwise. So we both agree that XP is fine.

My whole point was that the pre XP OSes (Windows 95,98,ME,NT4, and 2000), all suffer from the paging issue when a user is frequently working with datasets larger than the system RAM and I was simply trying to find out if XP suffered the same problem. Yes I got carried away in the process... I apologize. If you say that Windows 95,98,ME,NT4, and 2000 work fine out of the box with larger than system RAM datasets, then we'll just have to agree to disagree. But since this is a forum for Windows XP, discussing the issue with those other OSes is kind of offtopic so I'll say no more.

I'm going to play with XP at a friend's house this week and see how I like it. :)

Cheers,
Sixthofmay
 

Members online

No members online now.

Latest profile posts

Also Hi EP and people. I found this place again while looking through a oooollllllldddd backup. I have filled over 10TB and was looking at my collection of antiques. Any bids on the 500Mhz Win 95 fix?
Any of the SP crew still out there?
Xie wrote on Electronic Punk's profile.
Impressed you have kept this alive this long EP! So many sites have come and gone. :(

Just did some crude math and I apparently joined almost 18yrs ago, how is that possible???
hello peeps... is been some time since i last came here.
Electronic Punk wrote on Sazar's profile.
Rest in peace my friend, been trying to find you and finally did in the worst way imaginable.

Forum statistics

Threads
62,015
Messages
673,494
Members
5,623
Latest member
AndersonLo
Back