|Anonymous | Login | Signup for a new account||2019-04-25 20:54 PDT|
|Main | My View | View Issues | Change Log | Roadmap|
|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0000136||Dwarf Fortress||Technical -- General||public||2010-04-02 08:17||2018-01-13 10:27|
|Platform||PC||OS||Windows Vista||OS Version||Ultimate 64 bit|
|Target Version||Fixed in Version|
|Summary||0000136: When embarking on large area, DF hits 2GB memory limit and crashes|
some minor changes to init.txt
|Steps To Reproduce||tried every compatibility mode - no change|
windowed/full screen - no change
|Additional Information||Adventurer mode works without problems|
|Tags||crash, embark, memory, modding|
|I am getting this error whenever I try to embark with an embark area larger than 10x10 squares selected on the embark screen.|
i tried 13x13 every single time.
checking something smaller now
It does not crash with an embark area of 9x9.
|I confirm as well. 9x9 is the largest embark area I can select without a CTD. Major for me as I like huge embark areas. :||
edited on: 2010-04-02 16:54
okay, I genned up a quick pocket region simply to test this.
16x16 crashes. eventually, hung for a good couple minutes before throwing a C++ exception at me and finally closing.
My theory at this exact moment leads me to believe that this is a hardware limitation on your end. Not specifically a bug with the embark size. Insufficient memory most likely. The fact that I'm running this test on a pocket region probably does lower the memory requirements.
I'm not sure this can be blamed entirely on insufficient memory. I have 6GB of ram, but I still can not embark on anything larger than 10x10. While the game is running in a 9x9 map, it uses about 1.8GB of memory, with about 28% free.
I am running Windows 7 64 bit.
edited on: 2010-04-02 20:49
not sure what exactly to blame it on, but if I'm able to embark on 10x10 and you are not.... it does sort of imply some sort of hardware issue.
Still, you might try again on a pocket region and see if it lets you embark on a larger area that way.
I run DF on a Quadcore Intel i7 920 with 6GB RAM
OS is Windows Vista Ultimate 64bit
the hardware theory cant be right as i see it
maybe it got something to do with memory allocation in a 64bit OS?
What are your specs DoctorZuber
DF is a 32-Bit application. It -cannot- use more than 2GB at once. So at 1.8GB it's already running near the limit. Hit 2GB and it dies horribly. Increasing from 9x9 to 10x10 increases the amount of embark tiles from 81 to 100, an increase of almost 25%.
I guess that that's a hard limit at the moment.
Okay, seems this crash is nailed down.
I did some testing and you are right Halconnen.
As soon as DF reaches 2Gig Memory usage it dies witha runtime error.
I set GRAPHICS:NO and did a 13x13 build while watching the memory usage - it went up to 1.8 Gig on embark and worked perefectly.
Same with GRAPHICS:YES hits 2 Gig after a few seconds and dies.
So: its a memory usage problem with graphics enabled
|I've had this issue occur even with GRAPHICS:NO. AFAIK the game may allocate you a different number of Z-levels depending on the location you're embarking on, which of course affects memory usage, so that's another factor that can affect whether you get this crash.|
bicker x 2 (reporter)
|I found this issue to ocour with or without graphics being on as well|
bicker x 2 (reporter)
|This seems to again ocour with vertion 02|
|This isn't so much a crash as it is your system not having enough memory. What I mean by that is that I doubt this will ever be fixed - the fix is that your embark size is directly proportionate to your memory size. Too much and it crashes.|
Intel(R) Core(TM)2 Duo CPU T7100 @ 1.8 GHz 1.0 GHz
2 GB Ram
32-Bit Operating System
- I did my testing on a pocket region with various embark sizes.
- you never specified what size region you did your tests on
- larger regions very probably require more memory.
Confirming the 2GB Limit: just wanted to embark on a 16x16 (before checking here) and it crashed the moment DF hit 2GB RAM Usage.
Worked in earlier versions, so I blame it on a) the increased amount of z-levels and features or b) an unwanted memory leak somewhere down the line.
edited on: 2010-04-10 09:00
on a personal note, I think you guys are at somewhat insane trying to embark on such a large area, I've never wanted or needed larger than 5x5.
However, I do agree that it shouldn't simply crash on you. If you've got the hardware to run it, go for it, have fun.
32-bit operating systems has 2 GB limit for accessed memory.
64-bit operating systems doesn't have that limit, but 32-bit applications (like Dwarf Fortress) are still limited by the 2 GB limit (even though you could have 3gb memory and 3gb virtual memory).
Two possibilities to go beyond 2GB:
- Release a 64-bit version separately.
- Run special 32-bit processes that hold parts of the data.
In the mean time, the game should stop at 95% of total memory, 95% of available memory or 95% of 2GB (whichever comes first) to prevent the locking up before the crash.
Of course, 32-bit processors can't run 64-bit applications, so to release a separate 64-bit program could cause confusion and issues for 32-bit users.
But, of course, it would be brilliant for 64-bit users.
With Visual Studio, if you specify /LARGEADDRESSAWARE on the link command line you can get 3 to 3.5 GB of address space on 32-bit windows.
See http://msdn.microsoft.com/en-us/library/wz223b1z%28v=VS.80%29.aspx [^]
(This page is for VS 2005, but the switch also works for VS 2008 and VS 2010)
|Are there any workarounds for this at the moment that would allow embarking on 16x16? I've tried turning graphics off but I still hit the memory threshold.|
I have only one question - why do you want to embark on a 16x16 area?
That being said, no - since DF is compiled as an x86 or 32 bit program, it has the maximum memory usage limit of 2gb as mentioned above, which caps your playing area to one in which that memory limit is not achieved.
Turn graphics off - turn off temperature, economy, invaders, set a low population cap - disable all cavern layers/find an embark site with very few underground layers - find a very flat embark site - find an embark site without running water.
Those tricks should lower your memory consumption.
|Isn't this the same as 0000002? (which Toady is fixing in .04?)|
edited on: 2010-04-14 23:04
I got this problem when I embarked on a 4x14 or so site.
I was trying to build a bridge between two continents, something I've really been dying to do for awhile now.
Apparently that's a bit too big.
EDIT: But you know, the game locks up like clockwork after only a few minutes of me not doing anything really.
I uh, have a save here if it's ever requested.
I'll start up a new fort on a different save.
edited on: 2010-04-15 12:35
@Langdon they're both related to excess memory usage, but I doubt the fix for 0000002 was one that allowed the program to break the 2 GB limit. More likely he optimized the site finder to use less memory. This one will probably require restricting the size of the embark area -- I don't see any way around that.
|@Jodoodwin: Reminds me of having to download a new executable for ETW when it came out, and that the only difference was something like that /LARGEADDRESSAWARE so it wouldn't crash and freeze mid-game for no reason.|
Sadly, ETW is still, even after being released over a year ago, far buggier and far more prone to crashing than Dwarf Fortress.
Chalk one up for Toady! He beats a multimillion dollar studio and dev team of dozens when it comes to squashing bugs.
Huh? How did this turn into something about ETW?
Anyways, for .08, I don't crash when I do a large area, but it does take a while for it to go through the embarkation screens and it is pretty laggy when I get to the spot sometimes. DF did hang a few times for one particular spot on the map I'm using when I was looking at a 16X13 area, but I eventually got through and I think that spot was just extrenely memory taxing for various reasons.
edited on: 2010-07-26 20:51
Sorry for double post, but:
I'm getting this with the maximum possible on 31.10 and getting the runtime error as well. When I have the taskmanager open, I often see the load going past 50% and nearly maxing out the CPU.
I don't know if it was the exact error, but its definetly a runtime error.
Sometimes it will pull through with a very large area (on the order of more than 10x10), sometimes not. It seems to help when the embark area is totally flat. It handles 16x5 or 16x6 (and vice versa) just fine however.
Edit: Just a little update, it seems to handle large embark zones better in .12, haven't tried maximum possible yet though.
Reminder sent to: Nimrod
Does this still occur in 31.18?
|I can confirm this issue still occurs on my system with 31.18|
I can confirm this in .25. I'm running a 64-bit windows 7 with a quad-core and 8 gigs of ram, but as DF hits the 2 gb line it crashes with the following error:
Problem Event Name: APPCRASH
Application Name: Dwarf Fortress.exe
Application Version: 0.0.0.0
Application Timestamp: 4d90764f
Fault Module Name: MSVCR100.dll
Fault Module Version: 10.0.30319.1
Fault Module Timestamp: 4ba1dbbe
Exception Code: 40000015
Exception Offset: 0008d635
OS Version: 6.1.7600.2.0.0.256.1
Locale ID: 1035
Additional Information 1: 53ab
Additional Information 2: 53ab78575e4e4ce741bf82bee235390b
Additional Information 3: 3b1c
Additional Information 4: 3b1c7dab8afb70c763830f7ffc3fdc79
Tried it once more to be sure but then df didn't even give me an error. Just crashed.
Making a separate 64-bit version would be sweet and I'd gladly pay good money for it, but I can imagine that the shuffle wouldn't be too straightforward for Toady. Still some sort of fail-switch to shut the game down gently if the memory allocation limit is reached would be a nice addition, especially if it came with a warning screen that "By the bye, you're trying to embark on a too large area as I cannot reserve enough memory for it."
To give this a little bump for testing:
I currently do not have a system available to properly test this on myself (my main computer is borked, going to take a while to replace), but for users running DF on 64-Bit windows and 4GB or more of RAM:
Have you tried setting the LAA Flag on the Dwarf Fortress exe to see if it still behaves and -does- allocate more than 2GB of RAM?
Toady clearly didn't set it, but also didn't do anything to keep the game from hard crashing when it reaches that limit, so it might do fine unless he uses the spare bit in the pointers as a flag.
Should make no difference on a 32-Bit system since it'd take registry changes for windows to actually react to that, but for 64-Bit windows it should then proceed to allow DF up to 4GB of memory. Which is twice what it gets now.
May not work, may work. It certainly worked for Skyrim before Bethesda went and set the flag themselves.
A program like the CFF Explorer, or the Skyrim/Fallout 3 LAA tools or whatever should be able to do the job.
Using this LAA tool i was able to get DF to hog up all availble ram(~3gb) and it should be able to use even more, but i didn't have anymore
It seems for 64bit users that just setting the LAA flag would work wonders for huge embarks.
proof pic: http://i.imgur.com/r6UNK.png [^]
|I can confirm this as well for version 0.34.11, but I don't understand it since I used to embark on 16x16 about a year ago on a machine with only 2GB of RAM and much inferior specs overall. Using the LAA on a 32 bit machine allows the process to run fully but the game is beyond laggy, with massive lag spikes every 10 seconds while the game is unpaused.|
It totally depends on the land features. I think you had very few z-levels in total when you made that 16*16 embark.
What version were you using when you made that embark?
|It was probably version 0.31.25|
Just to add an update, in 34.11, with no mods and no changes to any settings from download state, this limitation still exists. anything over 10x10 cannot be embarked upon, as soon as it hits about 2 gigs of ram it just crashes.
running the LAA tool to force Large Address Aware allows a 16x16 embark of 198 z-levels (that's a total of almost 117 million tiles) and upon finally drawing the map after embark was taking up 3.6 gigs of RAM. This means I wouldn't be able to actually *play* for very long before hitting the LAA limit of 4 gigs.
System specs are not at play here (quadcore i7, 24 gigs of ram, win 7 x64) , it's the simple fact that 32 bit programs can only use a maximum of 2 gigs of ram (without being linked to LargeAddressAware, or on 32bit systems) or 4 gigs of ram on 64 bit systems with Large Address Aware linked.
With all modern processors capable of running 64 bit or 32 bit mode, and a vast majority of pre-packaged systems going 64 bit, I would like to re-request that future versions be compiled for both x86 and x64. For those of us insane enough to have a Megaproject in mind that requires a full 16x16 embark, it would certainly be nice to be able to play the game to the full capabilities that it proclaims to have. (Why let us *choose* 16x16 if it's physically impossible?) =)
I did quite a bit more playing around. it is plausible to get a larger embark in smaller memory if you decrease the z-levels.
DF 34.11 with MayDay and dfhack, Large custom world, cut the cave levels down to 1, and embarked on a 16x16 area where a volcano meets the beginning of a stream. there's lots of cliff, but total z-levels is 110, and total RAM usage at embark is 1.8 gigs. This is within the realm of plausibility. If one were to embark on a flatter area, with no cave levels, it could very well be playable at 16x16 without even LAA. Further z-level reduction could be had from other world gen options.
For those questioning 'why 16x16' - it's the only way to guarantee, 100%, a certain 'feature'. And imagine the megaprojects!
|Not surprising, but behavior still exists in 40.0x|
|There's a tool out there that fixes it. Can't recall what it is off the top of my head, though.|
LAA Large Address Aware
Fair warning: If you do this you will get a LOT of pathing bugs if you embark on large sites. The most common sign is that when you try to build something in clearly pathable areas it will say "no access to materials". Other things include dwarves being stuck in one spot and slowly starving to death.
This can be resolved by saving and reloading.
I haven't reported this particular issue as a bug since the exe is being modified but I'm pretty sure it might pop up once Toady starts the 64-bit conversion so... conflicted.
v0.43.05: The 64-bit version removes the 2GB addressing limit; the limitations of the legacy 32-bit version are unlikely to change. It would, however, be nice if the game could proactively reject problematic embarks without crashing.
Any tangential bugs with the 64-bit version (pathing, etc.) should be reported as separate issues after confirming them against an official 64-bit release.
|2010-04-02 08:17||Nimrod||New Issue|
|2010-04-02 09:10||Kurik||Note Added: 0000194|
|2010-04-02 09:40||Nimrod||Note Added: 0000203|
|2010-04-02 09:46||Nimrod||Note Added: 0000207|
|2010-04-02 09:47||Nimrod||Tag Attached: embark|
|2010-04-02 09:47||Nimrod||Tag Attached: crash|
|2010-04-02 16:15||jbrown6982||Note Added: 0000320|
|2010-04-02 16:15||jbrown6982||Issue Monitored: jbrown6982|
|2010-04-02 16:43||DoctorZuber||Note Added: 0000333|
|2010-04-02 16:45||DoctorZuber||Note Edited: 0000333||View Revisions|
|2010-04-02 16:47||DoctorZuber||Note Edited: 0000333||View Revisions|
|2010-04-02 16:48||DoctorZuber||Note Edited: 0000333||View Revisions|
|2010-04-02 16:52||DoctorZuber||Note Edited: 0000333||View Revisions|
|2010-04-02 16:54||DoctorZuber||Note Edited: 0000333||View Revisions|
|2010-04-02 20:44||bbgun06||Note Added: 0000405|
|2010-04-02 20:44||bbgun06||Issue Monitored: bbgun06|
|2010-04-02 20:49||DoctorZuber||Note Added: 0000407|
|2010-04-02 20:49||DoctorZuber||Note Edited: 0000407||View Revisions|
|2010-04-02 23:29||Nimrod||Note Added: 0000438|
|2010-04-03 00:55||Halconnen||Note Added: 0000452|
|2010-04-03 02:52||Nimrod||Note Added: 0000465|
|2010-04-03 04:37||Kurik||Issue Monitored: Kurik|
|2010-04-03 04:46||Kurik||Note Added: 0000475|
|2010-04-03 11:56||Footkerchief||Relationship added||has duplicate 0000188|
|2010-04-03 12:01||Aquillion||Tag Attached: modding|
|2010-04-03 12:51||bicker x 2||Note Added: 0000627|
|2010-04-07 19:29||Footkerchief||Summary||CTD runtime error on hitting 'embark' in Fortress mod => Crash when embarking on large area|
|2010-04-07 19:30||Footkerchief||Relationship added||has duplicate 0000758|
|2010-04-08 14:15||Footkerchief||Relationship added||has duplicate 0000822|
|2010-04-09 10:08||bicker x 2||Note Added: 0002349|
|2010-04-09 10:14||bicker x 2||Issue Monitored: bicker x 2|
|2010-04-09 10:20||SirPenguin||Note Added: 0002351|
|2010-04-09 12:32||DoctorZuber||Note Added: 0002384|
|2010-04-10 03:06||bicker x 2||Note Added: 0002545|
|2010-04-10 08:53||PetePetePete||Note Added: 0002578|
|2010-04-10 08:55||bicker x 2||Note Deleted: 0002545|
|2010-04-10 09:00||DoctorZuber||Note Added: 0002580|
|2010-04-10 09:00||DoctorZuber||Note Edited: 0002580||View Revisions|
|2010-04-11 17:30||Greyhawk||Note Added: 0002913|
|2010-04-12 06:44||James.Denholm||Note Added: 0003034|
|2010-04-12 23:40||Footkerchief||Relationship added||has duplicate 0001065|
|2010-04-13 19:32||jgoodwin||Note Added: 0003365|
|2010-04-13 19:51||Arkaaito||Note Added: 0003369|
|2010-04-13 20:01||Arkaaito||Tag Attached: memory|
|2010-04-13 20:15||Arkaaito||Issue Monitored: Arkaaito|
|2010-04-14 19:15||Clothar||Issue Monitored: Clothar|
|2010-04-14 19:50||Logical2u||Note Added: 0003596|
|2010-04-14 22:32||Langdon||Note Added: 0003623|
|2010-04-14 23:03||Gauphastus||Note Added: 0003626|
|2010-04-14 23:04||Gauphastus||Note Edited: 0003626||View Revisions|
|2010-04-15 08:57||Footkerchief||Relationship added||has duplicate 0001194|
|2010-04-15 12:34||Footkerchief||Note Added: 0003743|
|2010-04-15 12:35||Footkerchief||Note Edited: 0003743||View Revisions|
|2010-04-15 17:15||jbrown6982||Tag Attached: RESOLVED|
|2010-04-15 17:50||Stregone||Issue Monitored: Stregone|
|2010-04-15 23:53||jgoodwin||Issue Monitored: jgoodwin|
|2010-04-25 14:10||Firehound||Note Added: 0005034|
|2010-04-26 17:26||Footkerchief||Summary||Crash when embarking on large area => Crash when embarking on large area, possibly dependent on graphics|
|2010-04-26 17:26||Footkerchief||Tag Detached: RESOLVED|
|2010-04-28 14:05||Footkerchief||Category||General => Technical|
|2010-04-29 17:50||Footkerchief||Relationship added||has duplicate 0001648|
|2010-05-12 16:58||cloth4r||Issue Monitored: cloth4r|
|2010-06-22 18:13||Footkerchief||Sticky Issue||No => Yes|
|2010-06-22 18:13||Footkerchief||Summary||Crash when embarking on large area, possibly dependent on graphics => Crash when embarking on large area|
|2010-06-22 18:13||Footkerchief||Relationship added||has duplicate 0002436|
|2010-06-25 09:28||hyndis||Note Added: 0009087|
|2010-06-29 07:38||Footkerchief||Category||Technical => Technical -- General|
|2010-06-30 09:36||smjjames||Note Added: 0009377|
|2010-07-16 04:08||Logical2u||Relationship added||parent of 0002715|
|2010-07-19 13:38||smjjames||Note Added: 0010669|
|2010-07-19 13:50||smjjames||Note Edited: 0010669||View Revisions|
|2010-07-26 20:51||smjjames||Note Edited: 0010669||View Revisions|
|2010-11-17 10:10||Footkerchief||Note Added: 0013960|
|2010-12-13 14:00||Footkerchief||Relationship added||has duplicate 0003814|
|2010-12-17 22:05||SirPenguin||Note Added: 0014619|
|2011-03-21 06:19||Logical2u||Relationship added||has duplicate 0004270|
|2011-06-10 07:21||Egodeus||Note Added: 0017968|
|2011-08-16 13:00||Footkerchief||Summary||Crash when embarking on large area => When embarking on large area, DF hits 2GB memory limit and crashes|
|2012-02-19 08:19||Footkerchief||Relationship added||parent of 0004833|
|2012-02-21 06:49||Footkerchief||Relationship added||has duplicate 0005284|
|2012-02-21 06:51||Footkerchief||Relationship replaced||related to 0002715|
|2012-02-21 06:52||Footkerchief||Relationship replaced||related to 0004833|
|2012-03-07 08:51||Buglist||Issue Monitored: Buglist|
|2012-03-12 10:08||Footkerchief||Relationship added||has duplicate 0005622|
|2012-03-12 10:10||Footkerchief||Relationship added||has duplicate 0003659|
|2012-03-12 12:19||Footkerchief||Relationship added||related to 0005283|
|2012-03-12 17:11||Logical2u||Issue Monitored: Logical2u|
|2012-03-23 09:03||Halconnen||Note Added: 0021678|
|2012-03-29 10:27||erbridge||Issue Monitored: erbridge|
|2012-04-04 06:00||Echo51||Note Added: 0022073|
|2012-09-01 08:18||MightyRavenDark||Note Added: 0023514|
|2012-09-01 11:12||Kipi||Note Added: 0023515|
|2012-09-01 11:13||Kipi||Issue Monitored: Kipi|
|2012-09-01 15:34||MightyRavenDark||Note Added: 0023517|
|2012-11-26 23:22||Rayanth||Note Added: 0023747|
|2012-11-26 23:27||Rayanth||Issue Monitored: Rayanth|
|2012-12-01 23:20||Rayanth||Note Added: 0023757|
|2014-07-11 19:42||SirPenguin||Note Added: 0025909|
|2014-07-16 05:28||Talvieno||Note Added: 0026618|
|2014-08-13 14:07||handofcode||Note Added: 0028991|
|2014-08-13 14:09||handofcode||Note Edited: 0028991||View Revisions|
|2014-08-13 14:09||handofcode||Note Deleted: 0028991|
|2014-08-14 06:27||ptb_ptb||Note Added: 0029016|
|2014-08-15 14:00||handofcode||Note Added: 0029109|
|2014-09-11 21:11||chuzzum||Issue Monitored: chuzzum|
|2016-12-15 14:46||Loci||Note Added: 0036086|
|2018-01-11 21:03||Loci||Sticky Issue||Yes => No|
|2018-01-11 21:21||BlueManedHawk||Tag Attached: Windows|
|2018-01-13 10:27||lethosor||Tag Detached: Windows|
|Copyright © 2000 - 2010 MantisBT Group|