Use All the Memory You are Entitled to Have on Windows

[alexander]

Yes, you are probably sitting there, confused, thinking "Huh, what in the world does he mean?":shrug:

 

Well, let me clear some fog for you, so you understand what i am talking about.

 

But first, a history lesson :)

 

Back in time, when Windows 3.0 came out, people thought they wanted a bit more out of their memory. It supported 16 megs, but some people wanted more, much more, so Microsoft came up with a work around, an XMS driver that replaced Highmem DOS module and enabled addressing up from 16 megs of ram, and as reported, up to 512 megs... (astronomical values at the time)

 

Soon Microsoft did away with the 8096k bit descriptors and 64K possible combinations (which bring the theoretical limit of such a system to 512mb (multiply the two))

 

Fast forward to today's world of XP and Vista (and Standard Server Editions) memory support has been greatly improved. We are now reaching the theoretical maxims of the 32 bit processors, which can not use more then 4GB of ram, by design. (2 bits 32 registers equates to 2^32 or 4GB)

64 bit processors are in the petabyte ranges, 128 bit processors limits are so astronomical, doubt there is enough silicon to fill it for now anyways. ([math]3.1691265*10^{29}Gb[/math])

 

Anyhow, there have been methods devised that allow one to access more then the hardware limit of memory and we have discussed those here at some point, if you are still wodnering, just Google for highmem.

 

Anyways, so what i have come to find is that windows memory drivers, by default, do not allow you to have access to full 4GB, and no process can take up more then 2GB of Ram (i think microsoft is limiting itself there) and the system, can only access 3.5GB of memory (no matter what it tells you). Luckily they have created Physical Address Extension (PAE), and even though on most systems it does not increase support of signifficant amount of memory, it alows you to use the memory you already have, to its fullest :) (assuming i'm not the only person who has multiple systems pushing the limits)

 

Enabling PAE is pretty straight forward:

Edit the boot up line in C:\boot.ini and add /PAE to the line

 

ex:

 

Before

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Standard" /noexecute=optout /fastdetect

 

After

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003, Standard" /noexecute=optout /fastdetect /PAE

 

This enables you to use:

4Gb windows xp

4Gb windows vista

4Gb windows server 2003 standard

4Gb windows server 2008 standard

64Gb on enterprise and datacenter editions for both 2003 and 2008

8GB on 2000 advanced server

and 32Gb on 2000 datacenter edition

 

Enjoy :| err, well, whatever

[Boerseun]

...and change the /noexecute=optout to /noexecute=alwaysoff while you're at it. Gratuitous system crashes and freezes down by 99%!

 

Thanks for the info, btw. On a 4Gb RAM box, why would Windows limit itself to 3.5Gb? If that's bad design, and it can be cleared with a single boot.ini switch, why would current-edition vanilla installs still default to the previous state? Why not make the /PAE switch the default, with currently-shipping versions of Windoze?

 

Am I just dumb, or am I missing something here?

[Donk]

Enabling PAE is pretty straight forward:

Edit the boot up line in C:boot.ini and add PAE to the line

 

...

 

After

multi(0)disk(0)rdisk(0)partition(1)WINDOWS="Windows Server 2003, Standard" /noexecute=optout /fastdetect /PAE

PAE or /PAE ?

Or doesn't it make a difference?

[Buffy]

Windows flags are always forward-slash, so in this case /PAE is correct.

 

Yes, it matters a lot!

 

Keyboard work creates a class of unwanted things - one-letter typos, failures of phrasing, bad punctuation. If you don't want to delete these entirely, you can use the Return key to push them to the bottom of the screen, B)

Buffy

[alexander]

oooops typo there, i fixed it, ofcourse it is /PAE

 

Why not make the /PAE switch the default

 

because the problem is that they like to not have to create virtual memory tables, as it complicates the kernel a little bit.... and also there is no need for the PAE support for MOST, meaning 98% of Windows users... (hope that answers that)

[Boerseun]

Won't a Windoze machine running 4Gb with virtual memory tables adressing the initial missing 0.5Gb run a tad slower than the same machine using the default 3.5Gb with all the memory tables in RAM?

[alexander]

not that noticeably

[Buffy]

...and change the /noexecute=optout to /noexecute=alwaysoff while you're at it. Gratuitous system crashes and freezes down by 99%!

 

Finally got around to fiddling with this. But I have a note and some questions.

 

Note: Apparently if you turn off Data Execution Prevention (DEP, that's what the "/noexecute=alwaysoff" flag does), it also turns off PAE, but PAE is ON if you have DEP enabled! SO apparently there's no effect from adding the PAE flag as described by alex unless you do what B described with noexecute. This is apparently true for everything since XPSP2 and WS2003...

 

DEP looks like it does indeed cause lots of software to display (*ahem*) "anomalous behavior", but it's also a feature designed to protect against nasty buffer-overflow attacks.

 

SO:

Q1) What kind of risk am I really running if DEP is off?

Q2) What is the interaction between /PAE and another flag that MS touts known as /3GB?

 

A man cannot be too careful in the choice of his enemies, :beer:

Buffy

[UncleAl]

We run nasty number crunching software for months at a whack. Compile and run WinXP or compile and run Knoppix Live! Linux from the DvD drive. Same box, same source code. Linux generates output 40% faster than WinXP. Imagine that.

 

Win7 ships with a freeze-dried Asian kid and his mother. Soak in warm water overnight. The next day, the mother yells at the kid that he is shaming her in front of strangers as he installs and debugs the new OS. Doesn't take more than 15 hours. Return postage must be paid by the user.

 

Beware of bootleg Win7 that has freeze-dried Canadian First Americans instead.

[freeztar]

Note: Apparently if you turn off Data Execution Prevention (DEP, that's what the "/noexecute=alwaysoff" flag does), it also turns off PAE, but PAE is ON if you have DEP enabled! SO apparently there's no effect from adding the PAE flag as described by alex unless you do what B described with noexecute. This is apparently true for everything since XPSP2 and WS2003...

 

Not exactly...Check here:

 

Boot Parameters to Configure DEP and PAE

 

I recommend keeping it on for most systems and using the exclusions list to filter incompatible software.

 

Also, note that turning off DEP completely disables your hardware DEP the processor may have, supposedly.

[Experiment Garden]

I haven't been too impressed with Windows use of memory in the first place. Usually there is plenty of RAM but it seems afraid to use it. It will use lots of swap to make sure that it keeps half of the memory empty, and if you have 2 gigs of memory that doesn't make sense.

 

I am much more impressed by Ubuntu Linux's avoidance of using swap unless it absolutely has to. My OS of choice, Mac OS X, is pretty good at that too, but it is still a little more bloated than Ubuntu.

[alexander]

there is no such thing as one's linux avoidance of using swap, its all of linux as of kernel version blank or none of linux :naughty: unless ofcourse its a third party patch, in which case it could be a particular distribution, but linux will, by default try not to use swap if it can get away with it, swap=SLOOOW, and vista proved it with their dumb "flash drive for swap" initiative :)

[Experiment Garden]

Yeah, hopefully Windows 7 will be more like Linux as far as memory goes. Either way it won't effect me much, though, because I personally don't use Windows at home. My main machine is Mac OS X, and my other one is an Ubuntu box.

[alexander]

OS X 10.6, Windows/Xubuntu, Mythbuntu, gentoo, gentoo, gentoo, gentoo, gentoo, OpenBSD (sparc), IOS, Ubuntu (on my PS3).... and then OS X 10.4, Windows, OS X 10.5, Windows, Hackingtosh/Ubuntu for the rest of the family (my sister and dad each having 2 computers)

[Experiment Garden]

That's quite a collection of computers!

[alexander]

hehe, well, i did leave out the EEE which runs mostly backtrack, the neuros osd, which runs a linux-based firmware, and the wireless router, which also runs a linux-based firmware... also Cisco IOS technically does not run on a computer...

[freeztar]

I am considering a memory upgrade for my lappy. It runs fine, but I have 2GB, 1.6 available (I hate shared memory GPUs). I usually idle around 900 free. Gravy right? Nope.

 

Windows can't handle it's own background demons. :eek:

I love it when the msg pops up that Windows has to increase "virtual memory". Dude? Where's my memory? :)

 

This is never a problem in Linux. My background processes (anti-virus, anti-malware, etc.) and registry is optimally tweaked in linux. ;)

 

Ubuntu is awesome. Next step is away from debian. Where to? :)