Pagefile = fragmentation?

[Dark Atheist]

have been running a little test here for the post month, where by for 2 weeks i didn't have a pagefile, and the last 2 weeks had an 8gb pagefile.

During the 2 weeks without i had very little to no fragmentation on my C drive, used the normal way, surfing, email, photoshop work, rendering, and not once was there any frgmentation over 1-2%.

Now the last 2 weeks there has been excesive framentation when i do those same tasks but with an 8gb pagefile.

So the question is with PC's becoming more powerfile and able to handle more and more memory, do we really need a pagefile anymore?

 

[LordOfLA]

Yes you do. Windows will create a temporary one when you disable the permanent one.

Operating systems these days must swap and will swap whether you like it or not

Best thing to do is make a fixed size page file on a seperate partition and ideally on a seperate disk.

 

[Perris Calderon]

have been running a little test here for the post month, where by for 2 weeks i didn't have a pagefile, and the last 2 weeks had an 8gb pagefile.

During the 2 weeks without i had very little to no fragmentation on my C drive, used the normal way, surfing, email, photoshop work, rendering, and not once was there any frgmentation over 1-2%.

Now the last 2 weeks there has been excesive framentation when i do those same tasks but with an 8gb pagefile.

So the question is with PC's becoming more powerfile and able to handle more and more memory, do we really need a pagefile anymore?

+
when you started with a pagefile it was probably contiguous, then when you reconfigured that pagefile it was almost definately not contiguous unless you did a defrag before the volume was mounted specifically directed at the pagefile with a program like perfect disc, now, if you have an 8 gig file that's fragmented, all other new files are gonna get fragemnted going between the pagefile that can
t be deframented while the volume is mounted

therefore more fragmentation on subsequent files since there is less contigous free space

whenever you are making the pagefile bigger you have to do an offline defrag of that file

I also don't think you're gonna need an 8 gig pagefile int the first place, I would just set it to the amount of memory and allow expansion

as far as needing it, if you don't have one and the memory manager wants to swap something out it will still swap, it just won't be swapping the info that the pagefile is for, which is new data that isn't already on your disc

in other words, if a pagefile is not there and you have new data, that new data will remain in memory since the pagefile is for new data swap, that does not mean no data will be swapped, it means the memory manager will go to a differant file for it's memory management

my opinion is set it to about 2 to 4 gigs to start, leave it to be allowed to expand, if it does expand then make it bigger, if it doesn't expand leave it alone

 

[Dark Atheist]

been doing that since win95 (cant on laptop though only one drive/partition) - just thought with amount of ram now you could near enough just have all operations done in memory, as i believe pagefile is only used when you run out of physical mem

@ perris

page file is always contiguous - i make sure of it, i have always been told to use the formula 1.5X the amount of ram you have as standard, 2X if you have the space, so i tend to use 6.5GB as the initial size and allow it to go to 8GB should it require it.

I do a lot of renders and that hits the **** out of the system, and seeing as i have the space a few GB being given to the pagefile dont hurt

 

[Perris Calderon]

i believe pagefile is only used when you run out of physical mem

the pagefile is used to image new information, the old information already has a file to source when memory needs to be swapped

8 gig pagefile doesn't make sense though, I believe the default max the os sets it to is a two gig pagefile no matter how much memory you have and then if it does need bigger the os just makes it bigger for that episode and reduces again on the next boot

go back to two gigs, set to expansion, defrag off line and forget it

 

[Dark Atheist]

windows is telling me it wants 6133meg as the recommended size - after i went back to two gig, thing was near 3.5gb when i first installed it 3 months ago.

you mean go back to the option of system managed pagefile ?

 

[Perris Calderon]

windows is telling me it wants 6133meg as the recommended size - after i went back to two gig, thing was near 3.5gb when i first installed it 3 months ago.

you mean go back to the option of system managed pagefile ?

I would go back to managed but you might also try two gigs with a huge max and see if the system resizes it to make it bigger, if it does, then start with the size no smaller then that the system picks for itself which in your case is 6 gigs.

in xp it used to default to two gigs if you had more then a gig and a half, the initial minimum would never go automatically over two gigs, I guess vista sets that higher

try two gigs and with an 8 gig max and see if the system makes it bigger then that, or leave it system managed but now that you've messed with it you're gonna have to defrag it when you settle on a particular initial minimum

as far as the pagefile always being contiguous, this is only true if you didn't mess with it or if you made it smaller while it was still contiguous, if you made it bigger, that extent will only be contigous if there was an unused area right beside the pagefile to accomodate, otherwise it has to be fragmented, no choice

but ya, once the pagefile is contiguous it doesn't then fragment unless there is a bad sector and the os remaps the file, and unless the os resizes it, then the extent is on a differant part of the disc (but is discarded on reboot and the original extent is contiguous once again)

 

[JPRuss]

For the most part, I would agree with LordOfLA, the best idea is to have your page file on a completely different disk and on a seperate IDE (or SATA cable).

The CPU can easily manage 2 hard disks at once and it will greatly increase your performance.

The part I would disagree with is that you don't "need a pagefile". If Windows has a page file, it will use it, period there's no way around it no matter how much memory you have

but from a theoretical perspective if you have say 32GB of memory and the O/S needs to use more, something is "wrong" with the OS. IMHO

 

[Perris Calderon]

For the most part, I would agree with LordOfLA, the best idea is to have your page file on a completely different disk and on a seperate IDE (or SATA cable).

The CPU can easily manage 2 hard disks at once and it will greatly increase your performance.

that part is correct, the next isn't;

The part I would disagree with is that you don't "need a pagefile". If Windows has a page file, it will use it, period there's no way around it no matter how much memory you have

but from a theoretical perspective if you have say 32GB of memory and the O/S needs to use more, something is "wrong" with the OS. IMHO

I'll start with the last part first and then revisit at the end

but from a theoretical perspective if you have say 32GB of memory and the O/S needs to use more, something is "wrong" with the OS. IMHO

if you have 32 gigs of memory and 33 gigs of data, one gig is gonna be paged in and out when accessed even in best case scenario, not too many people have less data on their hardrive then memory on board

now for the first part of yor post;

you're right the system will use a pagefile if you put it there no matter how much memory on board, you are wrong that it will not page anyway if you don't put it there, it will page more without the file

the system pages whether you have a pagefile or not, the only differance without the pagefile is which items get paged, there is not less paging without a pagefile in fact there are times you will page more without that file, you will never page less.

if there is more memory then needed the system sends unused pages to "the standby list", the data is not released from memory, this is data still in memory but marked to release if memory goes under pressure, this is also how programs expand their working sets when you have more memory then you are using, unused pages are sent to the standby list, then when new data needs to be loaded, if there is an abundnce of memory unused memory is used instead of that which is on the standby list...this is a pretty cool mechanism for expanding working sets I might add

if something has no existing image then it can't get sent to "the standby list", it cannot be considered in the memory management model

when there is no image since there is nothing to back that data on release, it can't be re-loaded

when there is a pagefile, new items which otherwise had no image now get an image written to that pagefile, that's the main purpose of this file, to back data that has no other backing.

as you can see, without the pagefile, new data that does not have an image (but might be the best candidate) will be removed from memory management consideration

when that happens, items that are not the best candidate get paged when under pressure instead, in other words, you have more actual paging without a pagefile then with one

the best choice is having it on a second hardrive so there are no bottlnecks

this is a virtual memory operating system, everything is paged in and out of memory, it is not a real time operating system

the only time you will have so much memory you won't be paging is when you have more memory then hardrive data

for instance, I have a 250 gig hardrive on this laptop, how can I possibly load the entire hardrive with only three gigs of memory?

I can't, when I close a file that should be paged out of memory, however if there is available memory not under pressure, that data will goe to the standby list so when I re-open it there is no slowdown, it's as if it were never paged out.

follow?

the only time you would have enough memory not to need paging is if you had more memory then hardrive data

that's not gonna happen any time soon

I have a some movies on my hardrive that are a few gigs all by thesmelves, not too many systems can play entire movies completely from memory, that file is going to be constantly paging data in and out

 

[ming]

A tip if you wish to take notice of it, is to use C: for the OS only. Use a separate drive/partition for storing files such as games/movies/documents. That way the system C: drive fragments far more slowly.

I use my 36gb raptor as C: and it's only got 2% fragmentation since the last defrag 3 weeks ago. I do however have 4.5gb pagefile compared to your 8gb.

Also, this might have been mentioned already... you might not have defragged before running your 2nd test. shouldn't fragment too much unless you have a lot of files you use often.

 

[Dark Atheist]

i don't have an 8gb pagefile - i said i have told it it can expand up to a maximum of 8gb if required, atm i have a 6.5gb pagefile, as windows asks for that size, my c drive gets fragmented a lot if i am doing renders or updating newgroup headers.

 

[Perris Calderon]

i don't have an 8gb pagefile - i said i have told it it can expand up to a maximum of 8gb if required, atm i have a 6.5gb pagefile, as windows asks for that size, my c drive gets fragmented a lot if i am doing renders or updating newgroup headers.

give two gigs a try with max set to 8 gigs carpo, see if the os expands it, if it doesn't you're fine

 

[Dark Atheist]

will give that a try in the morning, see what happens

 

[LeeJend]

Umm,, actually the worst source of fragmentation I have identified is MS Patch Tuesday. My system files are scattered all over the place on the following Wednesday.

Since you did 2 weeks at a time, by any chance did your test have patch Tuesday fall during the Pagefile ON period?

As for the pagefile choices, I did a lot of experiments a few years back and finally decided on a fixed size, 2 gig pagefile set up on a D: partition on the primary (OS) drive. Reasons:
1) dynamic pagefile did affect fragmentation
2) making a small C: system drive to hold just the OS gave some improved data corruption protection. A virii formatting the C; partition left my D: ok
3) Putting the pagefile at the front of D: placed it close (in between) to the system files and to the program/data files on D: reducing head seek time
4) Placing pagefile on a seperate drive can cause jerky operation when the second drive spins down for power saving. I only use a 2nd drive for bulk storage or mirroring.
5) having system on a small partition allows for very fast defrag of system boot files after MS fragmentation Tuesday..

 

[Perris Calderon]

Umm,, actually the worst source of fragmentation I have identified is MS Patch Tuesday. My system files are scattered all over the place on the following Wednesday.

Since you did 2 weeks at a time, by any chance did your test have patch Tuesday fall during the Pagefile ON period?

As for the pagefile choices, I did a lot of experiments a few years back and finally decided on a fixed size, 2 gig pagefile set up on a D: partition on the primary (OS) drive. Reasons:
1) dynamic pagefile did affect fragmentation
2) making a small C: system drive to hold just the OS gave some improved data corruption protection. A virii formatting the C; partition left my D: ok
3) Putting the pagefile at the front of D: placed it close (in between) to the system files and to the program/data files on D: reducing head seek time
4) Placing pagefile on a seperate drive can cause jerky operation when the second drive spins down for power saving. I only use a 2nd drive for bulk storage or mirroring.
5) having system on a small partition allows for very fast defrag of system boot files after MS fragmentation Tuesday..

4) Placing pagefile on a seperate drive can cause jerky operation when the second drive spins down for power saving. I only use a 2nd drive for bulk storage or mirroring.

1) dynamic pagefile cannot affect fragmentation, the the original extent of the pagefile does not fragment no matter what size it is (once it's already coontiguous)with or withouy expansion set,, also do not forget, if you have the original extent set to the right size it does NOT expand

if you set it to have the ability to expand it won't unless it needs to

if it needs to you want it to

if it does expand the extent is discarded on the next boot, leaving it to 2 gigs fixed might have served you when you didn't need more pagefile and the os never wanted to expand it, then 2 gigs was fine, nowadays that's really not likely, you need to allow expansion in case the os does need a bigger file

4) if you put the pagefile on a second drive you need to set it to not spin down, it definately helps performance to have it on a second drive

 

[Dark Atheist]

Umm,, actually the worst source of fragmentation I have identified is MS Patch Tuesday. My system files are scattered all over the place on the following Wednesday.

Since you did 2 weeks at a time, by any chance did your test have patch Tuesday fall during the Pagefile ON period?

no as i ran during that time with it off, its on now though so i will see what happens this coming patch Tuesday

 

[LordOfLA]

I don't recommend dynamic page files. THe process of expanding it to perform an operation will kill performance of the app you are using and even crash it.

Set a static page file with min and max sizes the windows recommends and leave it at that.

If you have a second disk, put it on there and make sure to disable hard disk power saving. You will do your disk a favour as well. Constant power on/off will wear a disk down much sooner than if you leave it on all the time and the power savings are not worth it.

 

[Perris Calderon]

I don't recommend dynamic page files. THe process of expanding it to perform an operation will kill performance of the app you are using and even crash it.

this is incorrect, the pagefile IS static even when you set it to dymanic IF the original extent is the right size.

second of all, the pagefile expands with headroom and I cannot believe a program even sees the process much less crashes from it

the right size is the size the os never sees the need to expand it, if your dynamic pagefile has that size as the intitial minimum it IS static

there are no resources used other then finding a spot on the disc, which would NOT happen if you had the correct initial minimum for your workload

when the need arises for more pagefile, the only programs that might crash are those that might have crashed because the pagefile was too small

Set a static page file with min and max sizes the windows recommends and leave it at that.

not the best advice, if you want to set a static pagefile set the correct initial minimum, then it is static, you with the ability to expand if the estimate you came up with was wrong, the pagefile WILL be static if your estimate is correct...

if you are locked into the "static pagefile" myth then you are left with setting it to the max ms reccomends, not within the two values

here's your solution lord, set your initial minimum to 2 gigs with expansion enabled, if your os expands the pf then your intitial minimum is obviously too low, no the os will not expand that file unless it needs to, it doesn't look at the size of it before it expands the file it looks at the use of it...so go ahead and set it to two gigs if you think that's the right size, when or if the pagefile expands the file you know it was not the right size and you will have to reset the initial minimum to a setting that is better for your workload, of course defrag the file when you increase the size

If you have a second disk, put it on there and make sure to disable hard disk power saving. You will do your disk a favour as well. Constant power on/off will wear a disk down much sooner than if you leave it on all the time and the power savings are not worth it.

this is good advice, if the pagefile is on a second drive it needs to keep spinning, no powerdown alowed for pagefile hardrives

 

[LordOfLA]

The minimum sensible size for the page file is 2.5x ram capacity. 2GB is only sensible as a minimum if you have 768MB of ram.

A machine with 2GB of ram should have a minimum of 5GB for swap, windows XP recommends a touch more for me as it is seing 2.75GB of the 6GB I have installed in the machine.

If you need your page file to expand beyond that you need more ram and a 64bit windows install. Allowing windows to grow and shrink as needed does slow applications down and can cause them to crash.

 

[Perris Calderon]

The minimum sensible size for the page file is 2.5x ram capacity. 2GB is only sensible as a minimum if you have 768MB of ram.

A machine with 2GB of ram should have a minimum of 5GB for swap, windows XP recommends a touch more for me as it is seing 2.75GB of the 6GB I have installed in the machine.

If you need your page file to expand beyond that you need more ram and a 64bit windows install. Allowing windows to grow and shrink as needed does slow applications down and can cause them to crash.

you're missing the point lord

if your pagefile does not need to expand it does NOT, a dynamic file does NOT "expand and shrink", it IS static when you have the correct initial minimum...all you do by turning off expansion in BEST case scenario is the following;

nothing

on the other hand, in worst case snenario by setting the pagefile static, IF the os wants to expand it (which only happens if it NEEDS to), it can't, pages are then made to other candidates instead of those for best performance

there is no purpose served by eliminating expansion if you have the initial minimum set right, no advantage, it remains static, that's the point, it IS static when the inititial minimum is correct

in other words, when you have the correct inititial minimum with expansion enabled, you get your wish, you HAVE a static pagefile

it does NOT "grow and shrink as needed", it remains the size you set it unless you set it wrong.

this coversation is going nowhere

nobody should set their pagefile to a static size unless they are short of hardrive space and would rather lose the performance and keep hardrive area available

that and if they've already got the pagefile set to the max

those are the only times it makes any sense what so ever to keep a static setting, the pagefile IS static with a dynamic setting if you've set the initial minimum correctly

you are responding as if the pagefile expands willy nilly, it ONLY expands when it needs to expand, that is IT