Thread: [Genext2fs-devel] genext2fs: not enough memory for filesystem

Hi, I am using genext2fs 1.4.1 to generate an ext2 file system image for my
SD card.

I am trying to create a 2GB image to fill the whole SD card with the
following command:

"genext2fs -b 2000000 -d rootfs/ image.ext2"

but I always get the following error:

genext2fs: not enough memory for filesystem


Is there any limitation in the image size or something? I have 3GB of RAM in
my computer.

Thank you.

On 18/06/09 14:34, javier Martin wrote:
> Hi, I am using genext2fs 1.4.1 to generate an ext2 file system image for my
> SD card.
> 
> I am trying to create a 2GB image to fill the whole SD card with the
> following command:
> 
> "genext2fs -b 2000000 -d rootfs/ image.ext2"

If you are creating an image with free space, you'll probably want free inodes 
as well. This command will produce only enough inodes for what's in rootfs/.

> 
> but I always get the following error:
> 
> genext2fs: not enough memory for filesystem

Does, say, 1 million blocks work? (It may take a while.)

> 
> 
> Is there any limitation in the image size or something?

I never tried to create an image that large. Perhaps a revision 0 ext2 
filesystem has such limits, but this is an error from calloc() anyway.

> I have 3GB of RAM in my computer.

Can you cat /proc/meminfo and see what it says about free memory, swap etc?

Finn

> 
> Thank you.

2009/6/19 Finn Thain <ft...@te...>

>
> On 18/06/09 14:34, javier Martin wrote:
> > Hi, I am using genext2fs 1.4.1 to generate an ext2 file system image for
> my
> > SD card.
> >
> > I am trying to create a 2GB image to fill the whole SD card with the
> > following command:
> >
> > "genext2fs -b 2000000 -d rootfs/ image.ext2"
>
> If you are creating an image with free space, you'll probably want free
> inodes
> as well. This command will produce only enough inodes for what's in
> rootfs/.
>

Ah, you are right! I have changed to this command:

"genext2fs  -b 2000000 -d rootfs/ -N 20000 image.ext2"

This way I create 20000 inodes which is 1% of total blocks and seems a good
value. However I still got the same error:

genext2fs: not enough memory for filesystem


> >
> > but I always get the following error:
> >
> > genext2fs: not enough memory for filesystem
>
> Does, say, 1 million blocks work? (It may take a while.)
>

No, it does not say so, only the "not enough memory" message.


>
> >
> >
> > Is there any limitation in the image size or something?
>
> I never tried to create an image that large. Perhaps a revision 0 ext2
> filesystem has such limits, but this is an error from calloc() anyway.
>

According to kernel documentation filesystem limit is

 <http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt#314>	Filesystem
block size:     1kB        2kB        4kB        8kB
 <http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt#315>
 <http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt#316>	File
size limit:          16GB      256GB     2048GB     2048GB
 <http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt#317>	Filesystem
size limit:  2047GB     8192GB    16384GB    32768GB


http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt


> > I have 3GB of RAM in my computer.
>
> Can you cat /proc/meminfo and see what it says about free memory, swap etc?
>

Of course,

javier@acuario:/ cat /proc/meminfo
MemTotal:      3096932 kB
MemFree:        118332 kB
Buffers:         34220 kB
Cached:         336576 kB
SwapCached:          0 kB
Active:        1722356 kB
Inactive:       247188 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:           11920 kB
Writeback:           0 kB
AnonPages:     1598748 kB
Mapped:         697484 kB
Slab:           277296 kB
SReclaimable:   250052 kB
SUnreclaim:      27244 kB
PageTables:      24072 kB
NFS_Unstable:       20 kB
Bounce:              0 kB
CommitLimit:   1548464 kB
Committed_AS:  2338748 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     71676 kB
VmallocChunk: 34359665747 kB

However, it doesn't mean nothing for me because I don't know how much memory
does genext2fs try to use.

What I can tell you is that running the same command in a 6GB RAM system, it
works:

javier@escorpion:~$ cat /proc/meminfo
MemTotal:      6122840 kB
MemFree:        885828 kB
Buffers:       1283880 kB
Cached:        1274528 kB
SwapCached:          0 kB
Active:        2501212 kB
Inactive:      1728624 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:               8 kB
Writeback:           0 kB
AnonPages:     1671116 kB
Mapped:         143688 kB
Slab:           926312 kB
SReclaimable:   870020 kB
SUnreclaim:      56292 kB
PageTables:      27260 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3061420 kB
Committed_AS:  2613956 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     57700 kB
VmallocChunk: 34359679995 kB


> Finn
>
> >
> > Thank you.
>
>

I have found a temporal (or may be not) solution which is create the minimum
size file system for the image and then using dd + resize2fs for getting a
2GB image.

Nevertheless, it would be nice if genext2fs could do this directly.

Thanks.

Hi Javier,

On Fri, 2009-06-19 at 11:29 +0200, javier Martin wrote:

> 
> 
> 
> 2009/6/19 Finn Thain <ft...@te...>
> 
>         
>         On 18/06/09 14:34, javier Martin wrote:
>         > Hi, I am using genext2fs 1.4.1 to generate an ext2 file
>         system image for my
>         > SD card.
>         >
>         > I am trying to create a 2GB image to fill the whole SD card
>         with the
>         > following command:
>         >
>         > "genext2fs -b 2000000 -d rootfs/ image.ext2"
>         
>         
>         
>         If you are creating an image with free space, you'll probably
>         want free inodes
>         as well. This command will produce only enough inodes for
>         what's in rootfs/.
>         
>         
> 
> Ah, you are right! I have changed to this command:
> 
> "genext2fs  -b 2000000 -d rootfs/ -N 20000 image.ext2"
> 
> This way I create 20000 inodes which is 1% of total blocks and seems a
> good value. However I still got the same error:
> 
> genext2fs: not enough memory for filesystem


That's because genext2fs tries to allocate the full filesystem in
memory, and apparently you don't have a kernel that can allocate 2Gb per
process.


>         >
>         > but I always get the following error:
>         >
>         > genext2fs: not enough memory for filesystem
>         
>         
>         
>         Does, say, 1 million blocks work? (It may take a while.)
>         
>         
> 
> No, it does not say so, only the "not enough memory" message.
>  
> 
>         
>         >
>         >
>         > Is there any limitation in the image size or something?
>         
>         
>         
>         I never tried to create an image that large. Perhaps a
>         revision 0 ext2
>         filesystem has such limits, but this is an error from calloc()
>         anyway.
>         
>         
> 
> According to kernel documentation filesystem limit is 
> 
> 
> 
> 	Filesystem block size:     1kB        2kB        4kB        8kB
> 
> 
> 	File size limit:          16GB      256GB     2048GB     2048GB
> 
> 	Filesystem size limit:  2047GB     8192GB    16384GB    32768GB
> 
> 
> http://www.mjmwired.net/kernel/Documentation/filesystems/ext2.txt 
> 
> 
>         
>         > I have 3GB of RAM in my computer.
>         
>         
>         
>         Can you cat /proc/meminfo and see what it says about free
>         memory, swap etc?
>         
> 
> Of course,
> 
> javier@acuario:/ cat /proc/meminfo
> MemTotal:      3096932 kB
> MemFree:        118332 kB
> Buffers:         34220 kB
> Cached:         336576 kB
> SwapCached:          0 kB
> Active:        1722356 kB
> Inactive:       247188 kB
> SwapTotal:           0 kB
> SwapFree:            0 kB
> Dirty:           11920 kB
> Writeback:           0 kB
> AnonPages:     1598748 kB
> Mapped:         697484 kB
> Slab:           277296 kB
> SReclaimable:   250052 kB
> SUnreclaim:      27244 kB
> PageTables:      24072 kB
> NFS_Unstable:       20 kB
> Bounce:              0 kB
> CommitLimit:   1548464 kB
> Committed_AS:  2338748 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:     71676 kB
> VmallocChunk: 34359665747 kB
>  
> However, it doesn't mean nothing for me because I don't know how much
> memory does genext2fs try to use.
> 
> What I can tell you is that running the same command in a 6GB RAM
> system, it works:
> 
> javier@escorpion:~$ cat /proc/meminfo 
> MemTotal:      6122840 kB
> MemFree:        885828 kB
> Buffers:       1283880 kB
> Cached:        1274528 kB
> SwapCached:          0 kB
> Active:        2501212 kB
> Inactive:      1728624 kB
> SwapTotal:           0 kB
> SwapFree:            0 kB
> Dirty:               8 kB
> Writeback:           0 kB
> AnonPages:     1671116 kB
> Mapped:         143688 kB
> Slab:           926312 kB
> SReclaimable:   870020 kB
> SUnreclaim:      56292 kB
> PageTables:      27260 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:   3061420 kB
> Committed_AS:  2613956 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:     57700 kB
> VmallocChunk: 34359679995 kB
> 
> 
>         
>         
>         Finn
>         
>         >
>         > Thank you.
>         
> 
> 
> 
> I have found a temporal (or may be not) solution which is create the
> minimum size file system for the image and then using dd + resize2fs
> for getting a 2GB image.
> 
> Nevertheless, it would be nice if genext2fs could do this directly.


Yes, that'd be nice, but it would probably need quite a restructuring of
genext2fs.
In fact the problem is that right now genext2fs probably can't write a
4Gb filesystem.
To make it work, it would need at least:
- to mmap() the filesystem on-disk instead of allocate it in memory
- to make the endianness fixes on the fly at each read/write operation
instead of once when reading and once when writing
- perhaps to change some types to avoid the 32bits (4Gb) limit

Honestly I'm not that motivated to do that work, so don't hold your
breath.

    Xav

On 19/06/09 11:29, javier Martin wrote:
> 2009/6/19 Finn Thain <ft...@te...>

...
> >
> > Does, say, 1 million blocks work? (It may take a while.)
> >
> 
> No, it does not say so, only the "not enough memory" message.
> 

Makes sense...

...
> >
> > Can you cat /proc/meminfo and see what it says about free memory, swap etc?
> >
> 
> Of course,
> 
> javier@acuario:/ cat /proc/meminfo
> MemTotal:      3096932 kB
> MemFree:        118332 kB
> Buffers:         34220 kB
> Cached:         336576 kB
> SwapCached:          0 kB
> Active:        1722356 kB
> Inactive:       247188 kB
> SwapTotal:           0 kB
> SwapFree:            0 kB
> Dirty:           11920 kB
> Writeback:           0 kB
> AnonPages:     1598748 kB
> Mapped:         697484 kB
> Slab:           277296 kB
> SReclaimable:   250052 kB
> SUnreclaim:      27244 kB
> PageTables:      24072 kB
> NFS_Unstable:       20 kB
> Bounce:              0 kB
> CommitLimit:   1548464 kB
> Committed_AS:  2338748 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:     71676 kB
> VmallocChunk: 34359665747 kB
> 
> However, it doesn't mean nothing for me because I don't know how much memory
> does genext2fs try to use.

genext2fs will try to allocate the full 2 gig as Xavier says.

So given no swap, only a little free memory, and so much active memory, I 
don't think this can work. So I'd try adding a swap file if you don't have 
spare partition. Google has this:

http://blog.mypapit.net/2007/07/how-to-add-linux-swap-file-if-you-dont-have-swap-partition.html

> > What I can tell you is that running the same command in a 6GB RAM system, it
> works:
> 
> javier@escorpion:~$ cat /proc/meminfo
> MemTotal:      6122840 kB
> MemFree:        885828 kB

This is not enough in itself...

> Buffers:       1283880 kB
> Cached:        1274528 kB

..but these will give up enough memory for genext2fs as needed.

You can also use the free(1) command to get a less complicated summary.

Finn

> SwapCached:          0 kB
> Active:        2501212 kB
> Inactive:      1728624 kB
> SwapTotal:           0 kB
> SwapFree:            0 kB
> Dirty:               8 kB
> Writeback:           0 kB
> AnonPages:     1671116 kB
> Mapped:         143688 kB
> Slab:           926312 kB
> SReclaimable:   870020 kB
> SUnreclaim:      56292 kB
> PageTables:      27260 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> CommitLimit:   3061420 kB
> Committed_AS:  2613956 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:     57700 kB
> VmallocChunk: 34359679995 kB
> 
> 
> > Finn
> >
> > >
> > > Thank you.
> >
> >
> 
> I have found a temporal (or may be not) solution which is create the minimum
> size file system for the image and then using dd + resize2fs for getting a
> 2GB image.
> 
> Nevertheless, it would be nice if genext2fs could do this directly.
> 
> Thanks.