Why is Linux swallowing my processes?

[KrazyHorse]

I'm running a fairly big job (~11 hours on a decent machine). It's crapped out on me after 7 hours one time and 5 hours another. It's writing formatted data to a big giant file (should be ~4 gigs when done)

I redirected output (not the data stuff...just some program messages) and error to a log file. Now, this log file should probably only be in the kb when complete. Since computers seem to write to files in chunks that size or larger (sometimes 32 kb, sometimes more) I don't expect the log file to contain anything until run is complete. Or, if it conks out unexpectedly, it should finish writing whatever was in buffer to the file and I should be able to read what the error was.

Unfortunately, it didn't do this. The log file is empty. The data file finishes in the middle of a line...which means to me that it's the end of last complete chunk being written from buffer, not the actual break point.

Other than the root user killing my process or some idiot turning off the machine (it's in another office, and I'm running this **** in background remotely) what the **** could be causing this?

[Kuciwalker]

Can ext3 or whatever filesystem you're using handle >2GB or >4GB files?

[Kuciwalker]

Hm, ext3 supports at least 16GB... you might be using FAT though, which would be the problem.

[KrazyHorse]

I doubt I'm using FAT. This is a departmental machine. It's never seen anything other than linux on it.

Is there a variable which stores filesystem format information?

[Kuciwalker]

On unix? No clue.

Linux machines often use FAT for interoperability.

[Kuciwalker]

If you have any sort of graphical file browser you could try using that, right-clicking on the drive...

[KrazyHorse]

Plus, wouldn't this cause the job to die at the same point every time?

I've had it happen both under and over 2GB (significant difference)

And I'd imagine that it would cause process to die cleanly with error and buffers written out

[KrazyHorse]

Originally posted by Kuciwalker
If you have any sort of graphical file browser you could try using that, right-clicking on the drive...

Don't have direct access to machine right now. Am lying in bed 5 miles away feeling pissed off.

[Kuciwalker]

Originally posted by KrazyHorse
Plus, wouldn't this cause the job to die at the same point every time?

I've had it happen both under and over 2GB (significant difference)

Then it's probably not the filesystem.

And I'd imagine that it would cause process to die cleanly with error and buffers written out

I have no idea what happens when you actually try to make a file larger than the filesystem allows.

[KrazyHorse]

If this attempt dies too I'm going to hunt down this machine and reprogram it with a hatchet.

[Jon Miller]

I have had issues with things ending for no reason because I ran out of space. Basically my home directory was significantly limited and things would end because they would run out of space to operate in. They would end at different points, depending on what else was going on in memory.

JM

[Koyaanisqatsi]

Add some buffer flushes during execution...as least that should help you see what's going on.

[KrazyHorse]

I'm supposed to have mounds of free space on this machine.

How can I check in terminal window?

[KrazyHorse]

Originally posted by Koyaanisqatsi
Add some buffer flushes during execution...as least that should help you see what's going on.

? Don't know how to do this.

[Koyaanisqatsi]

quota -v