Minecraft killed my Git
Yes, that’s exactly what happened. But it’s not as simple as it might sound.
– Dad, Minecraft is not working, I can’t go into my world.
– Have you tried turning it off and on again?
– Yes, I did.
– Yeah, sure. You guys always do something with the server. Sorry, don’t have time now – go and play something else.
These kids, they always break things. You just look away, and here it comes – Huge herd of cows and terrifying Stack Overflow Error. And then, you have to fix everything, restart server manually, remove broken Minecraft instance, fix it somewhere else. Uff. Kids nowadays, it wasn’t like that in the past. If you broke something, you had to learn how to fix it.
So, the next day I have started my daily routine with GTD. Just for your information, I use Sublime Text and Sublime Merge to maintain my GTD lists. Sharing (between machines) is done via Git server hosted at the same machine as Minecraft server is. Using Git was a natural and perfect solution to maintain my text based lists of Questions and Problems of Life, the Universe, and Everything.
I have cleaned up few lists, committed changes, and here comes git push.
> git push ... error: remote unpack failed: unable to create temporary object directory
WTF?!
Quick session with stackoverflow and I have found ~1000 ideas, completely not suitable for me, each. Based on my experience, whenever I have this kind of issues – like things not being able to write to tmp – I use df/du combo.
> df /dev/sda1 239259268 233469248 0 100% / /dev/sdb1 120147264 120102144 45120 100% /media/backup-1 /dev/sdc1 120147264 120102144 45120 100% /media/backup-2
WTF?
> du -s -h * ... 137G media ... 221G var
OK, I get it, something went not quite as expected.
> du -s -h /var/* ... 220G minecraft
and then
> du -s -h /var/minecraft/* 64G backup-creative 94G backup-creative-public 61G backup-survival
So, here you are, my completely lost storage. It turned out that it’s no good, after all, when everything works as expected, and doesn’t crash :) Our server was running for year and a half, slowly creating heavier and heavier backups. Whenever Minecraft world grew a little bit, so did backup. Eventually, it turned out my initial calculations
let number_of_days = volume_size / server_size
were completely misleading as I didn’t assume that server grows over time ;)
In fact, the whole situation triggered yet another issue
[05:57:37] [Server thread/ERROR]: Encountered an unexpected exception java.lang.NullPointerException: Cannot invoke "com.google.gson.JsonArray.iterator()" because "$$1" is null at agd.f(SourceFile:121) ~[server-1.18.1.jar:?] at aci.z(SourceFile:84) ~[server-1.18.1.jar:?] at aci.<init>(SourceFile:23) ~[server-1.18.1.jar:?] at acj.e(SourceFile:174) ~[server-1.18.1.jar:?] at net.minecraft.server.MinecraftServer.w(SourceFile:657) ~[server-1.18.1.jar:?] at net.minecraft.server.MinecraftServer.a(SourceFile:270) ~[server-1.18.1.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?]
which was a result of broken server files. At some point, when system run out of space, something was not correctly stored and server simply died. Fortunately, I had a backup – tones of it ;)
Anyways, everything eventually got back to normal. Next week I will probably forget this mess, and next year I will hear a dreadful complain, again
– Dad, Minecraft is not working, I can’t go into my world.
Kids, you can’t leave them alone for a one minute. They always break things.