What does preallocating help reduce fragments when downloading torrents

Unfortunately, the problem is still present on some embedded platforms (NMT players and some NAS'es). Setting preallocation flag to 2 doesn't help - downloaded video files are still terribly fragmented, so that they are unwatchable (playback constantly stutters). It is definitely fragmentation, because after copying (not moving!) file elsewhere on the same disk it is played seamlessly. Looks like on those platforms posix_fallocate doesn't actually allocated disk space, just reserves it. This issue is very common in our local NMT community, everyone using T suffers it a lot. The solution is obvious - add another option (something like preallocation=3) to pad all files with zeroes before downloading. This will slow down beginning of download, but will simplify life for a lot of people. The appropriate patch is extremely simple, something like that:

if( preallocationMode == TR_PREALLOCATE_FILL ) { uint8_t buf[ 4 * ]; memset( buf, 0, sizeof( buf ) ); while ( length > 0 ) { const size_t thisPass = MIN( length, sizeof( buf ) ); write( fd, buf, thisPass ); length -= thisPass; } }

which should be located in preallocateFileFull function in libtransmission/fdlimit.c after successfully opening the file (and, of course, declaring constant TR_PREALLOCATE_FILL in transmission.h; sorry for not providing complete diff). Please consider including this feature into Transmission. Thanks!

Hey guys!

Again this problem manifests itself on my NAS as well, the Readynas NV+. Downloaded torrents are heavily fragmented. I've used sandeens script to investigate further, and obviously there is something fishy with the posix_fallocate one some platforms.

First of all, preallocating 16 megs does give me only one extent, but increasing the size for the preallocate call to Mb, a typical filesize for my torrents, gives me this:

readynas:~/dev-test$ ll total 64k drwxr-xr-x 2 xxx users 16k Jun 3 . drwx 14 xxx users 16k Jun 3 .. -rwxr-xr-x 1 xxx users 12k Jun 3 www.cronistalascolonias.com.ar -rw-r--r-- 1 xxx users Jun 3 test.c readynas:~/dev-test$ cat test.c #define _XOPEN_SOURCE #include <stdlib.h> #include <fcntl.h> #include <string.h> #include <stdio.h> void main(void) { int fd, ret; fd = open("mytestfile", O_CREAT|O_RDWR, S_IRWXU); if (fd < 0) { perror("file open failed"); exit(1); } ret = posix_fallocate(fd, 0, **); if (ret) { printf("posix_fallocate failed: %s", strerror(ret)); close(fd); exit(1); } close(fd); } readynas:~/dev-test$ gcc test.c test.c: In function `main': test.c warning: return type of `main' is not `int' readynas:~/dev-test$ ./www.cronistalascolonias.com.ar readynas:~/dev-test$ ll total M drwxr-xr-x 2 xxx users 16k Jun 3 . drwx 14 xxx users 16k Jun 3 .. -rwxr-xr-x 1 xxx users 12k Jun 3 www.cronistalascolonias.com.ar -rwx 1 xxx users M Jun 3 mytestfile -rw-r--r-- 1 xxx users Jun 3 test.c readynas:~/dev-test$ sudo filefrag mytestfile mytestfile: extents found, perfection would be 1 extent readynas:~/dev-test$

The platform is Debian-Sarge on a SPARC cpu. (Netgear Readynas NV+). I'd say this definitely calls for an explicit zero-fill option.

Wish

Replying to wishyou:

First of all, preallocating 16 megs does give me only one extent, but increasing the size for the preallocate call to Mb, a typical filesize for my torrents, gives me this:

readynas:~/dev-test$ ./www.cronistalascolonias.com.ar readynas:~/dev-test$ ll total M drwxr-xr-x 2 xxx users 16k Jun 3 . drwx 14 xxx users 16k Jun 3 .. -rwxr-xr-x 1 xxx users 12k Jun 3 www.cronistalascolonias.com.ar -rwx 1 xxx users M Jun 3 mytestfile -rw-r--r-- 1 xxx users Jun 3 test.c readynas:~/dev-test$ sudo filefrag mytestfile mytestfile: extents found, perfection would be 1 extent

That's ok. Nothing about posix_fallocate guarantees any kind of contiguity; it really only guarantees that you won't run out of space after it succeeds. However, it does give the filesystem a better -chance- at getting contiguous allocations. Depending on your filesystem, where free space is, etc, it could conceivably be very bad. But I'd be surprised if it's worse than nothing at all - as another test, on that same filesystem, remove that "mytestfile" and download a M torrent, see what fragmentation looks like there.

But let's not confuse the issue too much here; the real crux of why this bug was reopened, I think, is that posix_fallocate isn't getting configured in, or called, on some platforms I guess.

I reopen this bug because it is not fixed. I just downloaded Ubuntu CD (MB) and filefrag reports:

ubuntudesktop-iiso: extents found, perfection would be 6 extents

That's 75 fragments/MB.

My system: Transmission , Ubuntu amd64, Ext3

When I start downloading, "ls -lh" and Nautilus reports the file to have full size (MB) from the start. But the available disk space is not immediately taken and declines slowly throughout the download.

This is a very severe issue for me (the only one I have with otherwise brilliant Transmission program), because I have a very slow notebook disk. When I download ISO image with Transmission, I can't burn it to CD/DVD, because the disk is not fast enough to read it and burning stops frequently. When I download a movie (especially HD), the movie player can't play it, because the disk is not fast enough to read it and playback stutters a lot. Simply put, I always have to copy downloaded files to another location, because they are unusable without that.

I don't know much about preallocation methods and such stuff. I have a simple request - please make it work. I will gladly sacrifice loads of disk space in advance, but make the resulting files usable. It may not be the default behaviour (if you deem it not appropriate), but add an option to make it possible. Thank you.