Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0000332Dwarf FortressDwarf Mode -- Stockpilespublic2010-04-03 17:272010-07-26 07:17
Assigned ToToady One 
PrioritynormalSeverityminorReproducibilityhave not tried
PlatformOSOS Version
Product Version0.31.01 
Target VersionFixed in Version0.31.09 
Summary0000332: Raw files parsed in strict alphabetic order in Linux but not Windows, causing ID mismatches for saves moved between systems
DescriptionUsually, on embark, bituminous coal and lignite are considered to be an "Economic" stone, in terms of the stone stockpile categorization.

However, my current fort (about 3 years in) has suddenly decided that they should be in the "Other" category. The same fort was OK previously.

They are still red ("don't use as misc stone") in the 'z'->Stone screen.

This is possibly related to 0000219 as I seem to have lost the ability to make coke from those two stone types too.
TagsSave Included, SDL-only
Attached Files

- Relationships
has duplicate 0000219resolvedFootkerchief Smelting reaction ingredients/products change during gameplay, possibly Wine-specific 

-  Notes
Zet (reporter)
2010-04-26 07:22

I can confirm this. It has happened to me in two different forts now. Bituminous coal and lignite are red in the stones screen by default, but they don't list the "make coke" ability, and the smelters do not get the "make coke" menu option, even though loads of the resource are available.
DoctorZuber (reporter)
2010-04-26 07:25

check (z) stone and see if it has changed to green (which it can do if you find a lot). I'm wondering if this also moves it in the stockpile now as well.
Footkerchief (manager)
2010-04-26 10:31

oliver and Zet: what OS are you using?
oliver (reporter)
2010-04-26 15:52

In my case this was the same save that was transferred from XP to Linux/Wine and suffered from the disappearing reaction problem as described in 0000219, so it's quite likely related.
CptFastbreak (reporter)
2010-04-28 09:16
edited on: 2010-04-28 09:16

Cross-posting this from 0000219, as I've been told that it's an occurence of this bug.

OS: Windows XP Professional 32Bit

I've had the same occurence with Bituminous Coal and Lignite:
- They're still red in the z stone screen.
- They don't list the 'make coke' ability.
- They are in 'ordinary stone' in the stockpile menu.
- It's possible that making coke still worked for some time as I only noticed this when my repeating orders got cancelled.
- Now I can add the jobs via manager, but it states 'Urist McSmelter cancels Make Coke from Lignite: needs rock' and cancels.
- I got a few reoccurring crashes afterwards that seemed to stop when I cancelled the manager coke orders, so ordering coke might eventually lead to crashes. I'm not positive on this one though.
- I can still make Pig Iron and Steel from charcoal coke.
- Saving and reloading did not help, smelter is not in a burrow, save was never transferred.

Footkerchief (manager)
2010-04-28 12:57

CptFastbreak: do you have anything interesting in your errorlog?

Also, it might be helpful to upload that save to [^]
CptFastbreak (reporter)
2010-04-28 16:13

Here's the save file: [^]

The errorlog doesn't contain anything related to the smelter or to coke, just a bunch of "path fail" and "loop path fail" mostly related to the construction of buildings and the fleeing from invaders.
Eskarion (reporter)
2010-06-05 03:07

I encountered this bug after transfering a savegame made with 31.04 - legacy version to 31.05 - sdl-version (tried linux as well as win). Using the save with the 31.05 - legacy version works fine.
I also created a new world with the 31.05 sdl - version (win) and here the reactions listed in the z-stone menu are correct, although most of the ores now show the ability to make sharp blades while Obsidian isn't listed at all.
Phancy Physicist (reporter)
2010-06-13 01:49

I also encounter this bug. I used wine before and loaded my save to linux and now I can't make coal. Interestingly, if I save and load into wine (new and old windows versions) the save game works fine. When I go back to linux and it is broken again. So i guess there is a linux-only bug here.
Phancy Physicist (reporter)
2010-06-13 23:59

So one way to work a round this is to use links
"ln -s path_to_linux_df/data/save/ path_to_win_df/data/save"
then just open with wine queue up coal creation then exit open in linux and the coal creation will start up.

Not a great solution but works for now.
jeti0 (reporter)
2010-06-14 12:14

Actually, not only the coke creation tasks are missing from the smelters. All make <alloy> from <bars/ore> tasks are missing, too.

This happens when trying to use Wine generated world on native Linux DF and vice versa.
oliver (reporter)
2010-06-28 01:49
edited on: 2010-06-28 01:55

I have managed to work out how to reproduce this. Someone may want to reopen 0000219 with this info, since it's more directly relevant there.

As suggested in a comment on 0000219, it is to do with the order in which the raws are loaded.

I reproduced this on Linux by the following steps:

* Create a new region, embark
* Verify that bituminous coal has the right reactions.
* Save the region.
* Run 'ls -lU | grep inorganic' in data/save/regionN/raw/objects. -U provides an "unsorted" list of files that will reflect the order that the OS-level directory-scanning APIs return. In my case I get this order:


* Do something to inorganic_stone_mineral.txt (which contains the bituminous coal definition) so it is in a different position in the order. It will be very OS and filesystem specific what you need to do here. In my case, on my Linux install, renaming it to "inorganic_test.txt" worked, producing this order:


* Load the saved region, check reactions, note that bituminous coal is missing reactions and is considered an "other" stone in the stockpile lists.

* Save
* rename the raw file back to its original position
* Reload
* The reactions are fixed.

I'm guessing there is an ID stored internally somewhere in the region save that is just an index into the stone types loaded from the raws, and that index can end up pointing somewhere different depending on the order the raws are loaded. So moving between Wine/Linux/Windows etc, or just bad luck with what the filesystem does under the covers, has a chance of changing the ordering of the raws and causing chaos. Fun times!

Possible fix: sort the raw filenames before loading them, since that should be a stable order regardless of what the filesystem does. But that will break old saves :(

(Might be worth alerting Toady to this one, since it has the potential to cause all sorts of savegame problems - backups that don't work, can't transfer saves between different systems, savegames mysteriously going bad for no apparent reason, etc)

uggi (reporter)
2010-06-29 14:53

I'm having this bug as well. I started the game under Wine. Later on I switched to Windows, mainly because the game ran so much faster under Windows. I moved the game files to Windows partition, and also upgraded the game a few times. I think I started with 31.03, running 31.08 now.

Then I noticed I can't make coke from bituminous coal. I can use the manager noble to add the job, but the dwarfs just cancel it, saying "..cancels Make coke from bituminous coal: Needs melanite."

I tried running the game under Wine again. I first copied the game files back to the Linux partition. Now the smelter shows "Make coke from bituminous coal" and "Make coke from lignite" options, which were not available under Windows.

However, as the game files have been moved from one partition to another and then copied back again, I'm not sure what the original order was when I started the game. The Linux partition is in ext3 format, and Windows partition is in NTFS format. This is the order inorganic* files are now listed in the Linux partition:

Now if I run the native Linux version with the exact same save files, linked from the Wine directory so they should be listed in the same order, I'm again unable to create coke.

Also if I try running the game straight from the Windows partition under Wine, I'm unable to create coke again. The inorganic* files are listed alphabetically in that partition.

In short: the only way to create coke now is to run the game under Wine from the Linux partition.
uggi (reporter)
2010-07-01 07:55

If I rename those files, for example inorganic_stone_mineral.txt to inorganic_00_stone_mineral.txt (so it comes first in alphabetic order) I can then create coke in Windows too.

But, looking at the stone data in the z-stone-menu reveals that everything else is borked too. I could for example make bismuth bronze bars from cinnabar or brimstone. Everything in that screen changes depending on the order I rename those files.

So there's like ..umm.. 720 (?) different combinations how the inorganic* files can be ordered? I can't find the correct order, so I guess I'm better off starting a new game anyway.
Footkerchief (manager)
2010-07-01 08:07

oliver -- does that mean 0000219 can be reproduced without using Wine? If so, I'll reopen it. Thanks for investigating, by the way.
oliver (reporter)
2010-07-01 18:14

Yes, I believe that both this bug and 0000219 have the same cause, a change in the load order of raws. That seems to be most commonly triggered by moving saves to/from Wine, but it could trigger in other ways too.
oliver (reporter)
2010-07-02 05:05

Having dug into libgraphics a bit:

- POSIX systems (Linux/OSX) use glob(), which sorts results
- Win32 uses FindFirstFile()/FindNextFile(), which does not sort results
m4st0r (reporter)
2010-07-02 05:52
edited on: 2010-07-02 06:52

I'm having strange reaction results without changing the OS. Linux only, no wine, but instead of bituminous coal magnetite is used and lignite reaction has vanished. Haven't changed the raw's either. Some days ago everything worked fine.
Chromite now has the "Make coke from Lignite" tag.

Could a kernel update change the sort results and produce this behavior? That's the only major system-change that happened in the last couple of days.

numBone (reporter)
2010-07-02 09:43

I've been running in Windows from a shared drive on a Linux server. When I moved it to a shared drive on a Windows server, it seems to have caused this same problem. So it seems it still can be caused just by the servers having a different OS, even though I've been executing DF from the same workstation.
oliver (reporter)
2010-07-02 19:24

There's a patch to libgraphics that makes the sorting of filenames explicit (and case-insensitive) at [^]

Only tested on Linux. I can't build/test on Win32 or OSX.
Footkerchief (manager)
2010-07-02 19:47

Reminder sent to: Baughn

Hey Baughn -- sounds like this is a fix you can add yourself. oliver posted a patch at 0000332:0009551
Baughn (manager)
2010-07-03 05:03

Toady says this shouldn't matter, but that something is obviously busted.

It'll get fixed at a deeper level than just sorting the files.
Toady One (administrator)
2010-07-03 05:21

Yeah, I can get this to reproduce strictly within one version on Windows by changing the order of the files. However, the raw loader reports that the game is sorting them in with the proper, original order (the regions store the tokens of the individual objects, so file order should never matter). It seems there's some auxiliary information somewhere that's reacting badly to how the files are set up. My guess is that it is loading in the files, pre-calculating some reaction information, and then only doing the token sort afterward, but I have to check the order. If that's what it is, I just need to delay the pre-calc and everything should be fine all around. I can hope to preserve save compatibility (and get it working where it wasn't before), but no guarantees of course.
Toady One (administrator)
2010-07-08 03:59

Going to mark this one of for 0.31.09. I fixed the erroneous pre-calcs I found, they would cause it, and I can no longer reproduce it after the fix. Hopefully it'll be good now.

- Issue History
Date Modified Username Field Change
2010-04-03 17:27 oliver New Issue
2010-04-26 07:22 Zet Note Added: 0005113
2010-04-26 07:25 DoctorZuber Note Added: 0005114
2010-04-26 10:31 Footkerchief Note Added: 0005138
2010-04-26 10:46 Footkerchief Relationship added child of 0000219
2010-04-26 15:52 oliver Note Added: 0005196
2010-04-28 09:16 CptFastbreak Note Added: 0005539
2010-04-28 09:16 CptFastbreak Note Edited: 0005539 View Revisions
2010-04-28 09:28 CptFastbreak Issue Monitored: CptFastbreak
2010-04-28 12:57 Footkerchief Note Added: 0005568
2010-04-28 16:13 CptFastbreak Note Added: 0005603
2010-05-07 10:33 Footkerchief Relationship replaced related to 0000219
2010-05-16 23:52 Footkerchief Summary Coal / lignite mysteriously move to "other" stone type => Bituminous coal / lignite mysteriously move to "other" stone type
2010-05-16 23:54 Footkerchief Tag Attached: Save Included
2010-06-05 03:07 Eskarion Note Added: 0007711
2010-06-13 01:49 Phancy Physicist Note Added: 0008257
2010-06-13 23:59 Phancy Physicist Note Added: 0008391
2010-06-14 12:09 jeti0 Issue Monitored: jeti0
2010-06-14 12:14 jeti0 Note Added: 0008420
2010-06-28 01:49 oliver Note Added: 0009206
2010-06-28 01:51 oliver Note Edited: 0009206 View Revisions
2010-06-28 01:55 oliver Note Edited: 0009206 View Revisions
2010-06-28 02:48 oliver Issue Monitored: oliver
2010-06-29 14:53 uggi Note Added: 0009325
2010-07-01 07:55 uggi Note Added: 0009456
2010-07-01 08:07 Footkerchief Note Added: 0009458
2010-07-01 08:08 Footkerchief Summary Bituminous coal / lignite mysteriously move to "other" stone type => Bituminous coal / lignite mysteriously move to "other" stone type, due to raw files not being parsed in strict alphabetic order
2010-07-01 18:14 oliver Note Added: 0009484
2010-07-01 20:35 Footkerchief Relationship replaced has duplicate 0000219
2010-07-01 20:36 Footkerchief Summary Bituminous coal / lignite mysteriously move to "other" stone type, due to raw files not being parsed in strict alphabetic order => Raw files not parsed in strict alphabetic order in Linux, causing ID mismatches for saves moved between systems
2010-07-01 20:36 Footkerchief Sticky Issue No => Yes
2010-07-02 05:05 oliver Note Added: 0009509
2010-07-02 05:24 Footkerchief Summary Raw files not parsed in strict alphabetic order in Linux, causing ID mismatches for saves moved between systems => Raw files parsed in strict alphabetic order in Linux but not Windows, causing ID mismatches for saves moved between systems
2010-07-02 05:52 m4st0r Note Added: 0009511
2010-07-02 06:52 m4st0r Note Edited: 0009511 View Revisions
2010-07-02 08:31 jeti0 Issue End Monitor: jeti0
2010-07-02 09:43 numBone Note Added: 0009527
2010-07-02 19:24 oliver Note Added: 0009551
2010-07-02 19:45 Footkerchief Tag Attached: SDL-only
2010-07-02 19:47 Footkerchief Issue Monitored: Baughn
2010-07-02 19:47 Footkerchief Note Added: 0009554
2010-07-03 05:03 Baughn Note Added: 0009568
2010-07-03 05:21 Toady One Note Added: 0009571
2010-07-03 05:21 Toady One Assigned To => Toady One
2010-07-03 05:21 Toady One Status new => acknowledged
2010-07-05 05:07 uggi Issue Monitored: uggi
2010-07-08 03:59 Toady One Note Added: 0009774
2010-07-08 03:59 Toady One Status acknowledged => resolved
2010-07-08 03:59 Toady One Fixed in Version => 0.31.09
2010-07-08 03:59 Toady One Resolution open => fixed
2010-07-09 21:06 oliver Issue End Monitor: oliver
2010-07-12 12:53 Footkerchief Sticky Issue Yes => No
2010-07-26 07:17 uggi Issue End Monitor: uggi

Copyright © 2000 - 2010 MantisBT Group
Powered by Mantis Bugtracker