Dwarf Fortress Bug Tracker - Dwarf Fortress
View Issue Details
0005000Dwarf FortressAdventure Mode -- Movementpublic2011-12-29 14:482014-09-18 14:40
normalmajorhave not tried
LinuxUbuntuUbuntu 10.04.1
0005000: Lag/slowness in Adv Mode town (especially near keeps?)
I've got an adventure mode game where movement on local screens takes 13 seconds per move. During the 13 second pause, one of my CPUs is maxxed at 100%. Running strace shows that it is in a futex. Here is a portion of the strace output while is was stuck:

read(14, "MemTotal: 8129332 kB\nMemF"..., 1024) = 1024
close(14) = 0
munmap(0xf771a000, 4096) = 0
sched_yield() = 0
sched_yield() = 0
sched_yield() = 0
clock_gettime(CLOCK_MONOTONIC, {10921, 230666759}) = 0
clock_gettime(CLOCK_MONOTONIC, {10921, 230700655}) = 0
clock_gettime(CLOCK_MONOTONIC, {10921, 230732836}) = 0
sched_yield() = 0
gettimeofday({1325196848, 812373}, NULL) = 0
gettimeofday({1325196848, 812407}, NULL) = 0
read(6, 0xb14f8a8, 4096) = -1 EAGAIN (Resource temporarily unavailable)
select(7, [6], NULL, NULL, {0, 0}) = 0 (Timeout)
gettimeofday({1325196848, 812551}, NULL) = 0
gettimeofday({1325196848, 812584}, NULL) = 0
futex(0xb0356f8, FUTEX_WAIT_PRIVATE, 0, NULL

I've saved the game and restarted and had the same problem. I've even rebooted my computer and it still happens. It pauses whenever I do a move or make a targeted (A)ttack. I can look at my (i)nventory with no problem, and can (l)ook around at things. It seems like it is only freezing up when I do something that involves time passing, or maybe moving other creatures. While it is frozen nothing updates on the screen, but when it unfreezes, I can see drops of rain falling and stacked creatures flashing.

It first started when i had restored from a save when I'd just woken up at dawn after sleeping in a fort's keep. I was able to move just fine at that point and I checked my companions and had 5 of them. Then I left the keep and checked companions and suddenly only had one peasant. All of my soldier companions had disappeared from the list. I searched the fort and didn't find them, so I recruited 5 more soldiers and while I was heading for the gate to leave the fort things suddenly started freezing for 10-15 seconds every move. Then it cleared up and I was able to get away from the fort and start fast travel, which worked fine. I moved a short distance to a quest location and left fast travel and things were freezing every move once again. I went through the entire fight with a quest boss and his gang with a pause after every action. Usually when I move the screen shows the move before it freezes, but occasionally it freezes first, then shows the move when it unfreezes. When I would make a targeted attack, it would let me select my attack normally and then it would freeze, and when it unfroze it would show that tick's combat results. Doing a, c, d, i, k, l, r, z, and ? all work fine with no freezeup. Doing any move, or doing '.' to wait will cause the freeze, as well as doing e, g, p, s, which go through any menus fine, but freeze up as soon as I return to the map view.
Save is at http://dffd.wimbli.com/file.php?id=5305 [^]
related to 0007154resolved Toady One Maxed-out animal populations cause lag in dwarf/goblin sites 
has duplicate 0005281resolved Footkerchief Massive Lag outside Keep in Town 
has duplicate 0005389resolved Footkerchief Massive slowdown in a certain town 
has duplicate 0005410resolved Knight Otu Huge Lag spikes in some areas of towns, lots of abandoned housing, almost impossible to travel in area. 
has duplicate 0007191resolved Footkerchief extreme lag in towns 
related to 0006623confirmed Footkerchief Game freezes after each move 
Issue History
2011-12-29 14:48khearnNew Issue
2011-12-29 15:03khearnIssue Monitored: khearn
2011-12-29 15:11khearnNote Added: 0019266
2011-12-29 20:48khearnNote Added: 0019267
2012-02-22 06:53FootkerchiefRelationship addedhas duplicate 0005281
2012-02-22 06:54FootkerchiefRelationship addedhas duplicate 0005389
2012-02-22 06:54FootkerchiefNote Added: 0020455
2012-02-22 06:55FootkerchiefNote Edited: 0020455bug_revision_view_page.php?bugnote_id=0020455#r7599
2012-02-22 06:55FootkerchiefSummaryAdventure movement slows to a crawl => Lag/slowness in Adv Mode town (especially near keeps?)
2012-02-23 02:53Knight OtuRelationship addedhas duplicate 0005410
2012-02-23 05:14malvadoNote Added: 0020533
2012-02-23 05:55FootkerchiefNote Added: 0020537
2012-02-23 08:13runiqIssue Monitored: runiq
2012-02-25 03:02RosomackNote Added: 0020645
2012-02-25 03:02RosomackNote Edited: 0020645bug_revision_view_page.php?bugnote_id=0020645#r7680
2012-02-26 23:21Khym ChanurIssue Monitored: Khym Chanur
2012-02-27 06:11MopsyNote Added: 0020793
2012-02-27 06:16MopsyNote Edited: 0020793bug_revision_view_page.php?bugnote_id=0020793#r7739
2012-02-27 06:20MopsyIssue Monitored: Mopsy
2012-03-04 14:55MopsyNote Edited: 0020793bug_revision_view_page.php?bugnote_id=0020793#r7851
2012-04-13 12:45FootkerchiefIssue Monitored: malvado
2012-04-13 12:45FootkerchiefNote Added: 0022251
2012-04-13 12:45FootkerchiefTag Attached: AWAITING UPDATE
2012-04-13 22:25khearnNote Added: 0022254
2012-04-13 22:34FootkerchiefNote Added: 0022255
2012-04-14 06:41MopsyNote Added: 0022258
2012-04-14 06:44MopsyNote Edited: 0022258bug_revision_view_page.php?bugnote_id=0022258#r8285
2012-04-15 05:19MopsyNote Edited: 0022258bug_revision_view_page.php?bugnote_id=0022258#r8291
2012-04-15 16:24malvadoNote Added: 0022274
2012-06-05 14:37goronNote Added: 0022860
2012-06-05 14:38goronNote Edited: 0022860bug_revision_view_page.php?bugnote_id=0022860#r8503
2012-06-05 14:42goronIssue Monitored: goron
2013-09-23 09:14FootkerchiefTag Detached: AWAITING UPDATE
2014-07-08 07:49FootkerchiefRelationship addedrelated to 0006623
2014-07-12 14:35FootkerchiefRelationship addedrelated to 0007154
2014-07-13 00:15FootkerchiefRelationship addedhas duplicate 0007191
2014-07-13 06:09BloaxNote Added: 0026114
2014-07-13 09:32KennelTag Attached: lag
2014-07-14 16:42smjjamesNote Added: 0026388
2014-07-14 16:45smjjamesNote Edited: 0026388bug_revision_view_page.php?bugnote_id=0026388#r9831
2014-07-14 16:48smjjamesNote Edited: 0026388bug_revision_view_page.php?bugnote_id=0026388#r9832
2014-07-14 16:50smjjamesNote Edited: 0026388bug_revision_view_page.php?bugnote_id=0026388#r9833
2014-09-18 14:40MopsyNote Added: 0030247

2011-12-29 15:11   
I just copied the save to my laptop, which is running Unbuntu 10.04.03 LTS and I get the same behavior on that machine as well. I'll try to test it on my Windows XP machine at home this evening.
2011-12-29 20:48   
I just tried it on Windows XP and I get the freezups there, too. So this looks to be very reproduceable. I was expecting it would be a linux-specific thing, but it looks like it happens everywhere. That's good, because it means it's a problem with the save, not with the specific OS build. Should be easier to debug.
2012-02-22 06:54   
(edited on: 2012-02-22 06:55)
More saves:
pialamode, from 0005281: http://dffd.wimbli.com/file.php?id=5594 [^]
Rosomack, from 0005389: http://dffd.wimbli.com/file.php?id=5653 [^]

2012-02-23 05:14   
I reported a similar bug on 34.02 , although since it also involved Sewers I thought it would not be related directly to this bug.
Included in that bug report is a save with the area involved.

http://www.bay12games.com/dwarves/mantisbt/view.php?id=5410 [^]
2012-02-23 05:55   
malvado: all of the underground stuctures are connected, and you may have been close to a keep without knowing it. I've personally experienced keep-related lag when approaching the keep via both surface roads and sewers.
2012-02-25 03:02   
Ok, I don't know if it's related, but I managed to go capture a strange happening upon entering the keep courtyard - many fight messages, but I didn't see any of this. Later when I went into the keep I found some poor bastard's mutilated corpse. I also had these messages appear close to the keep where I experienced the massive slowdown.

http://tinypic.com/r/u0hm0/5 [^]

2012-02-27 06:11   
(edited on: 2012-03-04 14:55)
An example of similar extreme lag and high CPU use occurring near a necromancer's tower: http://dffd.wimbli.com/file.php?id=5713 [^]

Walking less than a hundred tiles north should be enough to trigger the bug. I have not observed any strange events or behaviors beyond the lag and CPU use.

Edit: I looked up the tower in question ("Whimsshin") in the world gen data and discovered that it's inhabited by no less than 6882 living humans (and a paltry 74 zombies). That's more than the nearby capital and almost 17% of the total human population. The lag doesn't seem all that mysterious now. But how did all those people end up in the tower? They aren't listed as prisoners.

2012-04-13 12:45   
Reminder sent to: khearn, malvado, Mopsy

Is this bug still present in 0.34.07+?
2012-04-13 22:25   
I can't tell if it's still present in 0.34.07. The save is from 0.31.25, so I can't try it with 0.34.07.
2012-04-13 22:34   
I didn't mean that particular save. I meant, have you observed this bug in recent versions? Especially in worlds generated in recent versions?
2012-04-14 06:41   
(edited on: 2012-04-15 05:19)
The problem remains the same when I load the save I put on DFFD in 0.34.7 and walk north. Traveling away from the tower, saving, quitting and restarting DF, reloading the save and then returning to the tower also produces the same result. When your adventurer approaches within a certain distance from the tower, the game abruptly starts to freeze with 100% CPU use for at least a minute for every in-game step of time. CPU use remains at 85% or so while in-game time is standing still (e.g. while you are 'l'ooking at things).

When I took the same adventurer to a significantly less populated (33 living, 92 undead) necro tower in the same world, there was no lag problem. It wouldn't surprise me if the lag is entirely due to the demands placed on the game engine by a large number of sentient creatures (with their equipment, historical records, etc.) clustering in a small area. I suppose you know better than me whether almost seven thousand people living together in a tower counts as a bug in itself. The world in question was generated by 0.34.2, I haven't been playing in any world generated by later versions.

Edit: Generally, the lag between steps in adventure mode seems to depend strongly upon the number of creatures in the vicinity. Lag is low in the wilderness, higher in hamlets and higher still around (or below) town markets and keeps. In case that wasn't clear, I'll add that the lag drops again once you move away from the crowded area.

2012-04-15 16:24   
The issue is less now in Towns, although as Mopsy points out theres a problem with some areas such as Necro Towers where Lag spikes appears randomly.

Theres some other areas as well I discovered that also had this problem such as when you swim, however it can be hard to catch when the game stops when you are swimming, due to no animation and that you have no sense of movement in some areas.
2012-06-05 14:37   
(edited on: 2012-06-05 14:38)
i have lags in a town in a world generated with 0.34.10 on Windows

2014-07-13 06:09   
Currently the game lags to all hell because EVERYONE (which usually means over 80 different entities) are CONSTANTLY talking to eachother.

If this constant stream of chatter was diffused over a longer period of time this problem wouldn't be as severe as it is now.
2014-07-14 16:42   
(edited on: 2014-07-14 16:50)
Heres a case of extreme lag with 40.03, I'm on the surface above a generated fort and the lag is horrible. Annoyingly, I can't fast travel until I move away from nearby obstacles..... which are underground....

http://dffd.wimbli.com/file.php?id=8972 [^]

Edit: I am using Maydays graphics pack, otherwise no changes. Also, when I waited until it was night, the lag was gone.

2014-09-18 14:40   
I've uploaded a save where it takes me several seconds to move a single step in a human keep:

http://dffd.wimbli.com/file.php?id=9731 [^]

The town outside the keep, known as "Pantbronze", has 1,967 human citizens and 217 cats. The cats are very active, running all over the place and trying to get into buildings. The world was generated by 40.10 and the save last updated by 40.12, but when the save is loaded in 40.13, the lag remains the same.

If anyone has the patience to go up there, this save also exhibits bug 0008303.