logfs-devel Mailing List for A Log Structured File System for Linux
Status: Pre-Alpha
Brought to you by:
ppadala
Menu
▾
▴
logfs-devel — LFS Developement Mailing List
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(15) |
Sep
(1) |
Oct
(1) |
Nov
(2) |
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2007 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
|
| 2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: linux l. <lin...@ya...> - 2017-04-22 19:08:32
|
|
From: Sami L. <sam...@ik...> - 2012-05-04 22:54:53
|
Hi, It seems the logfs filesystem needs some more sanity checks. I have a corrupted (intentionally, using the Berserker toolkit) 10 MiB ext2 filesystem which is not recognized as ext2 and which logfs tries to mount with mount -t auto, leading to a crash on a vanilla 3.3.4 running in kvm x86-64. The corrupted ext2 filesystem actually differs from an excellently working, clean ext2 filesystem by a single bit: ------------------------------------------------------------ $ diff -u <(hd testimg.ext2) <(hd testimg.ext2.89.min) --- /dev/fd/63 2012-05-05 01:24:33.184116801 +0300 +++ /dev/fd/62 2012-05-05 01:24:33.184116801 +0300 @@ -3,7 +3,7 @@ 00000400 00 0a 00 00 00 28 00 00 00 02 00 00 1d 23 00 00 |.....(.......#..| 00000410 9d 04 00 00 01 00 00 00 00 00 00 00 00 00 00 00 |................| 00000420 00 20 00 00 00 20 00 00 00 05 00 00 fc 12 ac 48 |. ... .........H| -00000430 66 4f a4 4f 00 00 26 00 53 ef 01 00 01 00 00 00 |fO.O..&.S.......| +00000430 66 4f a4 4f 00 00 26 00 53 ed 01 00 01 00 00 00 |fO.O..&.S.......| 00000440 66 4f a4 4f 00 4e ed 00 00 00 00 00 01 00 00 00 |fO.O.N..........| 00000450 00 00 00 00 0b 00 00 00 80 00 00 00 38 00 00 00 |............8...| 00000460 02 00 00 00 01 00 00 00 17 ca c1 08 7d a3 42 47 |............}.BG| ------------------------------------------------------------ You can get the offending filesystem image at http://sli.dy.fi/~sliedes/berserker/testcases/testimg.ext2.89.crash-logfs and the crash can be reproduced simply by mounting the filesystem with mount $path_to_filesystem /mnt -o errors=continue Sami ------------------------------------------------------------ REISERFS warning (device vdb): super-6506 reiserfs_getopt: bad value "continue" for option "errors" EXT3-fs (vdb): error: can't find ext3 filesystem on dev vdb. EXT2-fs (vdb): error: can't find an ext2 filesystem on dev vdb. EXT4-fs (vdb): VFS: Can't find ext4 filesystem cramfs: wrong magic SQUASHFS error: Can't find a SQUASHFS superblock on vdb VFS: Can't find a Minix filesystem V1 | V2 | V3 on device vdb. FAT-fs (vdb): bogus number of reserved sectors FAT-fs (vdb): Can't find a valid FAT filesystem FAT-fs (vdb): bogus number of reserved sectors FAT-fs (vdb): Can't find a valid FAT filesystem BFS-fs: bfs_fill_super(): No BFS filesystem on vdb (magic=00000000) hfs: unable to parse mount options hfs: unable to parse mount options. vxfs: WRONG superblock magic VFS: unable to find oldfs superblock on device vdb VFS: could not find a valid V7 on vdb. HPFS: Bad magic ... probably not HPFS NTFS-fs error (device vdb): read_ntfs_boot_sector(): Primary boot sector is invalid. NTFS-fs error (device vdb): read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting without trying to recover. NTFS-fs error (device vdb): ntfs_fill_super(): Not an NTFS volume. UFS-fs: Invalid option: "errors=continue" or missing value wrong mount options INFO: trying to register non-static key. the code is fine but needs lockdep annotation. turning off the locking correctness validator. Pid: 1419, comm: mount Not tainted 3.3.4 #2 Call Trace: [<ffffffff81074d35>] __lock_acquire+0x8b5/0x1ba0 [<ffffffff8107660d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff8172b41b>] ? _raw_spin_unlock_irq+0x2b/0x40 [<ffffffff8172a072>] ? wait_for_common+0x122/0x160 [<ffffffff8105e140>] ? try_to_wake_up+0x160/0x160 [<ffffffff812b421d>] ? logfs_get_wblocks+0x3d/0x90 [<ffffffff81077821>] lock_acquire+0xa1/0x140 [<ffffffff812b421d>] ? logfs_get_wblocks+0x3d/0x90 [<ffffffff81728a4a>] mutex_lock_nested+0x3a/0x2f0 [<ffffffff812b421d>] ? logfs_get_wblocks+0x3d/0x90 [<ffffffff812b421d>] logfs_get_wblocks+0x3d/0x90 [<ffffffff812b0ec8>] logfs_sync_fs+0x18/0x40 [<ffffffff8111896e>] __sync_filesystem+0x5e/0x90 [<ffffffff811189f2>] sync_filesystem+0x32/0x60 [<ffffffff812b85a7>] logfs_kill_sb+0x27/0xe0 [<ffffffff810eedd5>] deactivate_locked_super+0x45/0x80 [<ffffffff812b8f85>] logfs_mount+0x255/0x710 [<ffffffff810f06ab>] mount_fs+0x1b/0xd0 [<ffffffff8110a0fd>] vfs_kern_mount+0x6d/0x110 [<ffffffff8172b636>] ? _raw_read_unlock+0x26/0x30 [<ffffffff8110a21f>] do_kern_mount+0x4f/0x100 [<ffffffff8110b88a>] do_mount+0x53a/0x840 [<ffffffff810bafc2>] ? __get_free_pages+0x12/0x50 [<ffffffff8110b1d5>] ? copy_mount_options+0x35/0x170 [<ffffffff811374ef>] compat_sys_mount+0xdf/0x260 [<ffffffff8172d531>] sysenter_dispatch+0x7/0x2a [<ffffffff8156295e>] ? trace_hardirqs_on_thunk+0x3a/0x3f BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffff81728adb>] mutex_lock_nested+0xcb/0x2f0 PGD 6a52067 PUD 5657067 PMD 0 Oops: 0002 [#1] CPU 0 Pid: 1419, comm: mount Not tainted 3.3.4 #2 Bochs Bochs RIP: 0010:[<ffffffff81728adb>] [<ffffffff81728adb>] mutex_lock_nested+0xcb/0x2f0 RSP: 0018:ffff880005639c08 EFLAGS: 00010046 RAX: 0000000000000000 RBX: ffff880006a3ce10 RCX: 0000000000000002 RDX: 00000000ffffffff RSI: ffff880005639c28 RDI: ffff880006a3ce10 RBP: ffff880005639c78 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000003 R12: ffff880006a5a040 R13: 0000000000000246 R14: ffff880005639c28 R15: ffff880006a3ce50 FS: 0000000000000000(0000) GS:ffffffff81c1d000(0063) knlGS:00000000f7562750 CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b CR2: 0000000000000000 CR3: 0000000006a7e000 CR4: 00000000000006b0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process mount (pid: 1419, threadinfo ffff880005638000, task ffff880006a5a040) Stack: ffffffff812b421d ffff880005639bb8 0000000000001383 ffff8800069dc800 ffff880006a3ce50 0000000000000000 1111111111111111 ffff880005639c28 ffff880005639c40 ffff8800069dc800 0000000000000000 0000000000000400 Call Trace: [<ffffffff812b421d>] ? logfs_get_wblocks+0x3d/0x90 [<ffffffff812b421d>] logfs_get_wblocks+0x3d/0x90 [<ffffffff812b0ec8>] logfs_sync_fs+0x18/0x40 [<ffffffff8111896e>] __sync_filesystem+0x5e/0x90 [<ffffffff811189f2>] sync_filesystem+0x32/0x60 [<ffffffff812b85a7>] logfs_kill_sb+0x27/0xe0 [<ffffffff810eedd5>] deactivate_locked_super+0x45/0x80 [<ffffffff812b8f85>] logfs_mount+0x255/0x710 [<ffffffff810f06ab>] mount_fs+0x1b/0xd0 [<ffffffff8110a0fd>] vfs_kern_mount+0x6d/0x110 [<ffffffff8172b636>] ? _raw_read_unlock+0x26/0x30 [<ffffffff8110a21f>] do_kern_mount+0x4f/0x100 [<ffffffff8110b88a>] do_mount+0x53a/0x840 [<ffffffff810bafc2>] ? __get_free_pages+0x12/0x50 [<ffffffff8110b1d5>] ? copy_mount_options+0x35/0x170 [<ffffffff811374ef>] compat_sys_mount+0xdf/0x260 [<ffffffff8172d531>] sysenter_dispatch+0x7/0x2a [<ffffffff8156295e>] ? trace_hardirqs_on_thunk+0x3a/0x3f Code: f6 e8 aa 8e 94 ff 49 8b 54 24 08 4c 89 f6 48 89 df e8 fa 8f 94 ff 48 8b 43 48 ba ff ff ff ff 4c 89 73 48 4c 89 7d b0 48 89 45 b8 <4c> 89 30 89 d0 4c 89 65 c0 87 03 83 f8 01 0f 84 f9 00 00 00 48 RIP [<ffffffff81728adb>] mutex_lock_nested+0xcb/0x2f0 RSP <ffff880005639c08> CR2: 0000000000000000 ---[ end trace 30712c04b91b8fd0 ]--- Kernel panic - not syncing: Fatal exception Rebooting in 1 seconds.. ------------------------------------------------------------ |
|
From: Jim M. <Jim...@wd...> - 2011-11-14 23:30:58
|
Hi I'm interested in seeing how garbage collection works on block devices. I've mounted logfs on a block device (HDD) and made a 40GB partition. I wrote ~20GB of files in the partition, and then re-wrote to 20GB. DF shows me that the entire device is consumed. I'm trying to find a way to trigger GC and get some of the used capacity back. [root@localhost logfs]# df /mnt/logfs Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 40316280 37216368 1051912 98% / [root@localhost logfs]# ls -l total 17629004 -rw-r--r--. 1 root root 6543004600 Nov 11 06:58 qemp1 -rw-r--r--. 1 root root 6543004600 Nov 11 06:43 qemp2 -rw-r--r--. 1 root root 1635751150 Nov 11 08:14 remp1 -rw-r--r--. 1 root root 1635751150 Nov 11 08:15 remp2 -rw-r--r--. 1 root root 1635751150 Nov 11 08:16 remp3 -rw-r--r--. 1 root root 32715023 Nov 10 05:43 semp4 -rw-rw-r--. 1 jim jim 64149 Nov 9 15:54 temp1 -rw-r--r--. 1 root root 6453196 Nov 10 05:42 temp11 -rw-r--r--. 1 root root 6453196 Nov 10 03:24 temp12 -rw-r--r--. 1 root root 6453196 Nov 10 05:42 temp13 -rw-r--r--. 1 root root 6453196 Nov 10 05:42 temp14 -rw-r--r--. 1 root root 64149 Nov 9 17:46 temp2 -rw-r--r--. 1 root root 64149 Nov 9 17:46 temp3 -rw-r--r--. 1 root root 64149 Nov 9 17:46 temp4 Any recommendations are appreciated. Jim |
|
From: Tianbing X. <xut...@gm...> - 2007-05-06 20:47:21
|
Hi,
I create several files and then access ,write them.
here is clean test result.
TESTING...
TESTING: argc no. 2
TESTING: argv ./lfs_cleanerd
TESTING: argv /dev/hda7
TESTING: argv /dev/hda7
optind 1
name=rootfs,mnt=/,type=rootfs
name=/dev/root,mnt=/,type=ext3
name=/dev,mnt=/dev,type=tmpfs
name=/proc,mnt=/proc,type=proc
name=/sys,mnt=/sys,type=sysfs
name=/proc/bus/usb,mnt=/proc/bus/usb,type=usbfs
name=devpts,mnt=/dev/pts,type=devpts
name=/dev/hda1,mnt=/boot,type=ext3
name=tmpfs,mnt=/dev/shm,type=tmpfs
name=/dev/hda5,mnt=/home,type=ext3
name=automount(pid2587),mnt=/net,type=autofs
name=/dev/hda7,mnt=/mnt/lfs_partition,type=lfs
TESTING getmntinfo mntname /mnt/lfs_partition
lfs_cleanerd[3463]: Cleaner starting on filesystem /dev/hda7
Testing : fs_name is /dev/hda7, mntname is /mnt/lfs_partition
ifile size = 160
1 inode daddr 225
2 inode daddr 24380
3 inode daddr 24377
4 inode daddr 237
seguse table size = 1168
lfs_cleanerd[3463]: segments clean 9455 segments dirty 9456
segment = 1, live bytes = 21513
segment = 2, live bytes = 21519
segment = 3, live bytes = 21525
segment = 4, live bytes = 21531
segment = 5, live bytes = 21538
segment = 6, live bytes = 21544
segment = 7, live bytes = 21550
segment = 8, live bytes = 21556
segment = 9, live bytes = 21562
segment = 10, live bytes = 21568
segment = 11, live bytes = 21574
segment = 12, live bytes = 21580
segment = 13, live bytes = 21586
segment = 14, live bytes = 21592
segment = 15, live bytes = 21598
segment = 16, live bytes = 21604
segment = 17, live bytes = 21610
segment = 18, live bytes = 21616
segment = 19, live bytes = 21622
segment = 20, live bytes = 21628
segment = 21, live bytes = 21634
segment = 22, live bytes = 21640
segment = 23, live bytes = 21646
segment = 24, live bytes = 21652
segment = 25, live bytes = 21658
segment = 26, live bytes = 21664
segment = 27, live bytes = 21670
segment = 28, live bytes = 21676
segment = 29, live bytes = 21682
segment = 30, live bytes = 21688
segment = 31, live bytes = 21694
segment = 32, live bytes = 21700
segment = 33, live bytes = 21706
segment = 34, live bytes = 21712
segment = 35, live bytes = 21718
segment = 36, live bytes = 21724
segment = 37, live bytes = 21730
segment = 38, live bytes = 21736
segment = 39, live bytes = 21742
segment = 40, live bytes = 21748
segment = 41, live bytes = 21754
segment = 42, live bytes = 21760
segment = 43, live bytes = 460
segment = 44, live bytes = 466
segment = 45, live bytes = 472
segment = 46, live bytes = 478
segment = 47, live bytes = 484
segment = 48, live bytes = 317952
TESTING: partition name /dev/hda7, mount device /mnt/lfs_partition
TESTING: ifile_fd: 6
Testing : clean loop
max_free_segs = 0
lfs_cleanerd[3463]: Cleaner Running at Sun May 6 16:20:38 2007
(9455 of 0 segments available, avail = 9458, bfree = 9457)
Testing: clean fs
lfs_cleanerd[3463]: Entering choose_segments
lfs_cleanerd[3463]: magic 0x10203FF version 0x1
lfs_cleanerd[3463]: nseg 48
Testing : nsegs is 48
0...lfs_cleanerd[3463]: Segment 0 nbytes=21513 flags=D nsums=21515
lastmod: Thu Jan 1 00:58:36 1970
lfs_cleanerd[3463]: choosing segment 0
1...lfs_cleanerd[3463]: Segment 1 nbytes=21519 flags=C nsums=21521
lastmod: Thu Jan 1 00:58:42 1970
2...lfs_cleanerd[3463]: Segment 2 nbytes=21525 flags=D nsums=21527
lastmod: Thu Jan 1 00:58:48 1970
lfs_cleanerd[3463]: choosing segment 2
3...lfs_cleanerd[3463]: Segment 3 nbytes=21531 flags=CA nsums=21534
lastmod: Thu Jan 1 00:58:55 1970
4...lfs_cleanerd[3463]: Segment 4 nbytes=21538 flags=DA nsums=21540
lastmod: Thu Jan 1 00:59:01 1970
5...lfs_cleanerd[3463]: Segment 5 nbytes=21544 flags=CA nsums=21546
lastmod: Thu Jan 1 00:59:07 1970
6...lfs_cleanerd[3463]: Segment 6 nbytes=21550 flags=DA nsums=21552
lastmod: Thu Jan 1 00:59:13 1970
7...lfs_cleanerd[3463]: Segment 7 nbytes=21556 flags=CA nsums=21558
lastmod: Thu Jan 1 00:59:19 1970
8...lfs_cleanerd[3463]: Segment 8 nbytes=21562 flags=DA nsums=21564
lastmod: Thu Jan 1 00:59:25 1970
9...lfs_cleanerd[3463]: Segment 9 nbytes=21568 flags=CA nsums=21570
lastmod: Thu Jan 1 00:59:31 1970
10...lfs_cleanerd[3463]: Segment 10 nbytes=21574 flags=DA nsums=21576
lastmod: Thu Jan 1 00:59:37 1970
11...lfs_cleanerd[3463]: Segment 11 nbytes=21580 flags=CA nsums=21582
lastmod: Thu Jan 1 00:59:43 1970
12...lfs_cleanerd[3463]: Segment 12 nbytes=21586 flags=DA nsums=21588
lastmod: Thu Jan 1 00:59:49 1970
13...lfs_cleanerd[3463]: Segment 13 nbytes=21592 flags=CA nsums=21594
lastmod: Thu Jan 1 00:59:55 1970
14...lfs_cleanerd[3463]: Segment 14 nbytes=21598 flags=DA nsums=21600
lastmod: Thu Jan 1 01:00:01 1970
15...lfs_cleanerd[3463]: Segment 15 nbytes=21604 flags=CA nsums=21606
lastmod: Thu Jan 1 01:00:07 1970
16...lfs_cleanerd[3463]: Segment 16 nbytes=21610 flags=DA nsums=21612
lastmod: Thu Jan 1 01:00:13 1970
17...lfs_cleanerd[3463]: Segment 17 nbytes=21616 flags=CA nsums=21618
lastmod: Thu Jan 1 01:00:19 1970
18...lfs_cleanerd[3463]: Segment 18 nbytes=21622 flags=DA nsums=21624
lastmod: Thu Jan 1 01:00:25 1970
19...lfs_cleanerd[3463]: Segment 19 nbytes=21628 flags=CA nsums=21630
lastmod: Thu Jan 1 01:00:31 1970
20...lfs_cleanerd[3463]: Segment 20 nbytes=21634 flags=DA nsums=21636
lastmod: Thu Jan 1 01:00:37 1970
21...lfs_cleanerd[3463]: Segment 21 nbytes=21640 flags=CA nsums=21642
lastmod: Thu Jan 1 01:00:43 1970
22...lfs_cleanerd[3463]: Segment 22 nbytes=21646 flags=DA nsums=21648
lastmod: Thu Jan 1 01:00:49 1970
23...lfs_cleanerd[3463]: Segment 23 nbytes=21652 flags=CA nsums=21654
lastmod: Thu Jan 1 01:00:55 1970
24...lfs_cleanerd[3463]: Segment 24 nbytes=21658 flags=DA nsums=21660
lastmod: Thu Jan 1 01:01:01 1970
25...lfs_cleanerd[3463]: Segment 25 nbytes=21664 flags=CA nsums=21666
lastmod: Thu Jan 1 01:01:07 1970
26...lfs_cleanerd[3463]: Segment 26 nbytes=21670 flags=DA nsums=21672
lastmod: Thu Jan 1 01:01:13 1970
27...lfs_cleanerd[3463]: Segment 27 nbytes=21676 flags=CA nsums=21678
lastmod: Thu Jan 1 01:01:19 1970
28...lfs_cleanerd[3463]: Segment 28 nbytes=21682 flags=DA nsums=21684
lastmod: Thu Jan 1 01:01:25 1970
29...lfs_cleanerd[3463]: Segment 29 nbytes=21688 flags=CA nsums=21690
lastmod: Thu Jan 1 01:01:31 1970
30...lfs_cleanerd[3463]: Segment 30 nbytes=21694 flags=DA nsums=21696
lastmod: Thu Jan 1 01:01:37 1970
31...lfs_cleanerd[3463]: Segment 31 nbytes=21700 flags=CA nsums=21702
lastmod: Thu Jan 1 01:01:43 1970
32...lfs_cleanerd[3463]: Segment 32 nbytes=21706 flags=DA nsums=21708
lastmod: Thu Jan 1 01:01:49 1970
33...lfs_cleanerd[3463]: Segment 33 nbytes=21712 flags=CA nsums=21714
lastmod: Thu Jan 1 01:01:55 1970
34...lfs_cleanerd[3463]: Segment 34 nbytes=21718 flags=DA nsums=21720
lastmod: Thu Jan 1 01:02:01 1970
35...lfs_cleanerd[3463]: Segment 35 nbytes=21724 flags=CA nsums=21726
lastmod: Thu Jan 1 01:02:07 1970
36...lfs_cleanerd[3463]: Segment 36 nbytes=21730 flags=DA nsums=21732
lastmod: Thu Jan 1 01:02:13 1970
37...lfs_cleanerd[3463]: Segment 37 nbytes=21736 flags=CA nsums=21738
lastmod: Thu Jan 1 01:02:19 1970
38...lfs_cleanerd[3463]: Segment 38 nbytes=21742 flags=DA nsums=21744
lastmod: Thu Jan 1 01:02:25 1970
39...lfs_cleanerd[3463]: Segment 39 nbytes=21748 flags=CA nsums=21750
lastmod: Thu Jan 1 01:02:31 1970
40...lfs_cleanerd[3463]: Segment 40 nbytes=21754 flags=DA nsums=21756
lastmod: Thu Jan 1 01:02:37 1970
41...lfs_cleanerd[3463]: Segment 41 nbytes=21760 flags=CA nsums=21762
lastmod: Thu Jan 1 01:02:43 1970
42...lfs_cleanerd[3463]: Segment 42 nbytes=460 flags=CA nsums=462 lastmod:
Wed Dec 31 19:07:43 1969
43...lfs_cleanerd[3463]: Segment 43 nbytes=466 flags=DA nsums=468 lastmod:
Wed Dec 31 19:07:49 1969
44...lfs_cleanerd[3463]: Segment 44 nbytes=472 flags=CA nsums=474 lastmod:
Wed Dec 31 19:07:55 1969
45...lfs_cleanerd[3463]: Segment 45 nbytes=478 flags=DA nsums=480 lastmod:
Wed Dec 31 19:08:01 1969
46...lfs_cleanerd[3463]: Segment 46 nbytes=484 flags=D nsums=486 lastmod:
Wed Dec 31 19:08:07 1969
lfs_cleanerd[3463]: choosing segment 46
47...lfs_cleanerd[3463]: Segment 47 nbytes=317952 flags=DA nsums=1
lastmod: Sun May 6 16:02:04 2007
Testing : nsegs is 3
lfs_cleanerd[3463]: Sorting...
lfs_cleanerd[3463]: segment 46 age 1178482351 contains 484 priority 0
lfs_cleanerd[3463]: segment 2 age 1178461310 contains 21525 priority 0
lfs_cleanerd[3463]: segment 0 age 1178461322 contains 21513 priority 0
lfs_cleanerd[3463]: Returning 3 segments
TESTING: total number of choosed segments: 3
TESTING: debug=3
TESTING: empty segments number 0
TESTING: number of nsegs: 1
lfs_cleanerd[3463]: clean_fs: found 3 segments to clean in
/mnt/lfs_partition
TESTING: found 3 segments to clean.
TESTING: options: 0
TESTING: count segments
lfs_cleanerd[3463]: Cleaning segment 46 (of 1 choices)
lfs_cleanerd[3463]: adding segment 46: contains 484 bytes
lfs_cleanerd[3463]: mmap_segment seg_daddr: 1511424 seg_size:
32768
lfs_cleanerd[3463]: segment buffer at: 0x8052a48 seg_addr 0x171000
Segmentation fault
I found that print the last error informantion is here
if(debug > 1)
702 syslog(LOG_DEBUG, "\tsegment buffer at: %p\tseg_addr
0x%llx",
703 seg_buf, (long long)seg_addr);
in int lfs_segmapv(FS_INFO *fsp, int segnum, caddr_t seg_buf,
676 BLOCK_INFO **blocks, int *bcount)
of cleanerd.c
Anyone knows how to solve that ?
Thank you.
|
|
From: VC <gu...@tw...> - 2007-05-04 00:51:03
|
Could you please give me a detailed step-by-step for how to get LFS to be compatible with the 2.6.20 kernel? Vito |
|
From: Tianbing X. <xut...@gm...> - 2007-04-29 01:03:12
|
HI: I am trying to implemnt lfs in 2.6.20. But seens member named 'i_blksize' has been removed since 2.6.20, so there is compile error: /home/lfs/kernel/inode.c: In function 'lfs_new_inode': /home/lfs/kernel/inode.c:435: error: 'struct inode' has no member named 'i_blksize' Anyone knows how to solve this problem? |
|
From: Sameer R. <sra...@bi...> - 2006-02-24 08:53:46
|
Hello, I am a graduate student at SUNY Binghamton and have started to work on = my Masters thesis. I plan to work (and experiment) with various cleaning = policies. I read a comment about the cleaner on Pradeep's blog quoting = that the cleaner wasnt a part of this LFS. As far as understanding the = concepts, I have read some papers regarding LFS. Can someone guide me = with catching up with the code already written? Which source files do I = need to focus on (in what order) so that it would speed up things of = getting me upto the mark. I would be glad to share all the work I do as = a part of this thesis. Any suggestions related to this will really help. Sameer Railkar ------------ If I knew what I was doing, I would not call it "Research" -- Albert Einstein |
|
From: Pradeep P. <pp...@ee...> - 2005-11-17 18:44:19
|
> I am interesting in LFS too. After reading several papers about > LFS, I got a question. LFS splits the disk into equal-sized > segments(right?), in which files are stored. How about if a file is > larger than one segment? How does the file span across several > segments? I am not sure whether its inode can only contain the blocks > of the same segment or any block in other segments. An inode can contain blocks from any segment. Pradeep |
|
From: <xue...@ya...> - 2005-11-17 01:46:05
|
Hello,
I am interesting in LFS too. After reading several papers about LFS, I got a question. LFS splits the disk into equal-sized segments(right?), in which files are stored. How about if a file is larger than one segment? How does the file span across several segments? I am not sure whether its inode can only contain the blocks of the same segment or any block in other segments.
Thanks!
---------------------------------
无限容量雅虎相册，原图等大下载，超快速度，赶快抢注！ |
|
From: Pradeep P. <pp...@ee...> - 2005-10-10 22:14:34
|
Hello folks, How's everyone doing? Have been taking it easy after the non-stop working days, and I think I am refreshed enough to re-plug into the matrix. There are some interesting developments about another logfs almost concurrently developed by a group from Nippon. They recently posted a message to the vfs-devel list, which I have added to the end of this mail. I am in touch with the authors, and so far they seem to be interested in merging. I will have a look at their code when I get some time. Pradeep -------- Forwarded Message -------- From: am...@os... To: lin...@vg... Cc: pp...@ee..., ni...@os... Subject: NILFS: the New Implementation of a Log-structured File System Date: Mon, 26 Sep 2005 15:20:26 +0900 (JST) Hello, We are developing an LFS for Linux 2.6, named Nilfs (the New Implementation of a Log-structured File System). We implemented NILFS using modern technology, for example, file and inode blocks are managed by B-trees, internal data is processed in 64 bit wide word size. The NILFS also supports snapshots. The snapshots are automatically hold without any explicit requests, snapshot can be mounted as a read-only file system. The latest version is available on the NILFS WEB site http://www.nilfs.org/ Comments are welcome! Thank you! AMAGAI Yoshiji Nippon Telegraph and Telephone Corporation NTT Cyber Space Laboratories Open Source Software Computing Project Kernel Team ni...@os... |
|
From: Pradeep P. <pp...@ee...> - 2005-09-04 21:28:00
|
Hi, I received a couple of mails showing interest in LFS and its future, and I see that a bunch of people joined the mailing list. To update on the status of LFS, I have my prelims exam coming up in next few weeks, so I will be busy with it for a while. I will pickup LFS hopefully by end of september, and start adding snapshot cleaner etc. If you are interested in trying out LFS, please follow the instructions at http://logfs.sourceforge.net/compiling.html. The directory doc/ contains a few bits and pieces of documentation and you can always look at the code for more info. If you are running the cleaner run it with -q option, which runs the cleaner only once. For testing, have a look at the doc/cases file, for a list of test cases that are known to work. Use lfsprogs/lfsread to inspect the contents of any block in LFS. If you encounter any problems, let me know. -- Pradeep Padala http://ppadala.blogspot.com |
|
From: Pradeep P. <pp...@ee...> - 2005-08-30 17:34:13
|
Hello folks,
It's time to release an official version. I have uploaded the latest
and greatest of LFS to sourceforge website. lfs-0.1 version can be
downloaded from http://www.sf.net/projects/logfs. This contains all the
proposed features and should be stable enough. I still want to caution
anyone planning to test this to use a separate test machine or a vm.
I also have patches that integrate LFS into the kernel build system,
but haven't added them to the release as I don't think anybody really
wants to compile to LFS into the kernel. For the curious and brave, the
patches are in patches/ directory (*.diff). I have tested the
compilation with LFS, it works fine, but I haven't tested an in-kernel LFS.
Cheers !
--
Pradeep Padala
http://ppadala.blogspot.com
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-27 00:03:58
|
Hello,
For past two months, I have been working on developing a
log-structured file system (LFS) that supports snapshots as a part of
the Google Summer of Code (SoC) [1] program. The motivation and goals
for the project are described in my SoC proposal [2].
In the current state, LFS is quite usable (all the basic fs
operations are working), and a working cleaner is also available.
However, it's not yet ready for the heavy iron and I need to sort out a
few issues. I hope to include it in the mainline, and I will release a
set of patches later for reviewing.
Instructions on compiling and using the stable and development
versions from CVS are available at [3]. The bleeding edge version
includes basic snapshotting framework.
The project website is http://logfs.sourceforge.net, and a mailing
list [4] is available for discussion.
Comments are welcome.
Thanks,
--
Pradeep Padala
http://ppadala.blogspot.com
[1] http://code.google.com/summerofcode.html
[2] http://logfs.sourceforge.net/lfs.pdf
[3] http://logfs.sourceforge.net/compiling.html
[4] http://lists.sourceforge.net/lists/listinfo/logfs-devel
|
|
From: Marius E. <ma...@mo...> - 2005-08-24 03:38:58
|
yep. i think it's totally fine to make this a system daemon. we need
one anyway for the cleaner. maybe make the both of them "lfsd."
magic is best done in userland.
marius.
On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> Ok, I see it. You mean by using mount --bind. Yep, this is actually
> quite cool. We don't have to do any readdir/read_inode magic to support
> .snapshot. Only problem I see is that mount cannot be invoked by a
> normal user, and a daemon with root privileges should be given the
> responsibility of keeping track of these snapshot directories.
>=20
> Marius Eriksen wrote:
> > i believe in linux you can do things such as:
> >
> > alias /usr/foo/bar to /home/marius/foo
> >
> > and everything under /usr/foo/bar appears under /home/marius/foo.
> >
> > basically what the nullfs stackable filesystem does, but it works with
> > dentrycache magic.
> >
> > so, for example, let's say you were to support ~/.snapshots/hourly.2,
> > then you might have
> >
> > /mnt/hourly.2
> >
> > and alias
> >
> > /mnt/hourly.2/home/marius/ to /home/marius/.snapshots/hourly.2
> >
> > marius.
> >
> > On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >
> >>Umm, interesting. What exactly do you mean by namespace aliasing? An
> >>example would make things clearer for me.
> >>
> >>Marius Eriksen wrote:
> >>
> >>>i think that this should be purely a userland administrative thing.
> >>>
> >>>i.e., all that is needed is the ability to mount a particular
> >>>snapshot. (and of course, to make said snapshots).
> >>>
> >>>then, you can have a daemon that manages ~/.snapshot directories.
> >>>this should be especially doable with the namespace aliasing that
> >>>linux can do.
> >>>
> >>>marius.
> >>>
> >>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >>>
> >>>
> >>>>Marius Eriksen wrote:
> >>>>
> >>>>
> >>>>>how about making them mountable filesystems?
> >>>>>
> >>>>> mount -t lfs -osnapshot=3D"january 12, 13:15" /mnt/old
> >>>>
> >>>>Yep, thought about it, and is probably what I will have as the first
> >>>>version.
> >>>>
> >>>>In NetAPP's WAFL, for each dir .snapshot contains something like
> >>>>hourly.0/ hourly.1/ ... weekly.0/ ...
> >>>>Under these directories you can find snapshots for corresponding
> >>>>directories.
> >>>>
> >>>>This is certainly doable, but requires more careful thought, as we ma=
y
> >>>>have to alternate between IFILEs as the user cds back and forth, not
> >>>>trivial. Thoughts ?
> >>>>
> >>>>
> >>>>
> >>>>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>>Hi,
> >>>>>> The final piece in the puzzle is in. I have checked in the basic
> >>>>>>framework for doing snapshots. I haven't yet decided on the user
> >>>>>>intreface (may be .snapshots like NetAPP does), but you can access =
the
> >>>>>>snapshots using lfsread program.
> >>>>>> I want to explain a little bit about the design.
> >>>>>> I have toyed around with two different ways of implementing
> >>>>>>snapshots, theoritically actually same. To support any snapshots, o=
ne
> >>>>>>has to make a copy of the inode map (in LFS case IFILE) and fs
> >>>>>>modifications are done to the new IFILE only. Also, writes to exist=
ing
> >>>>>>blocks (both inode and data) should be re-directed to new blocks (t=
his
> >>>>>>is almost trivial in LFS).
> >>>>>> Now, how do we make a copy of inode map? We can simply make a new
> >>>>>>IFILE file, copy the contents and start using the new file or We ca=
n
> >>>>>>divide the IFILE into snapshots. For example, the snapshot index (s=
napi)
> >>>>>>can be used to encode within the inode number (say first two bits).=
So,
> >>>>>>the operations are roughly as follows
> >>>>>>
> >>>>>>1. Implementation using IFILE copy
> >>>>>> *) the on-disk/in-memory inode structure has snapi.
> >>>>>> *) user calls LFS_SNAP_CREATE ioctl
> >>>>>> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointer=
s
> >>>>>> *) when a file is written its snapi is compared to current snapi,=
if
> >>>>>> they differ new block is created.
> >>>>>> *) to access the snapshot just switch the IFILE. Temporarily
> >>>>>> switching IFILE back and forth is not that trivial though.
> >>>>>>
> >>>>>>2. Implementation using inode encoding
> >>>>>> *) For each inode, inode number is derived with a simple function
> >>>>>> like below
> >>>>>>
> >>>>>> static inline ino_t INUMBER(struct super_block *sb, ino_t ino=
)
> >>>>>> { __u32 snapi =3D CURR_SNAPI(sb);
> >>>>>>
> >>>>>> snapi =3D snapi << 30;
> >>>>>> return ino | snapi;
> >>>>>> }
> >>>>>>
> >>>>>> *) user calls LFS_SNAP_CREATE ioctl
> >>>>>> *) make a copy of existing inodes to correponding new inodes. Fo=
r
> >>>>>> example inode 1 is copied to 0x4001
> >>>>>> *) switch the root inode to the new inode.
> >>>>>> *) here comes the tricky part, the directory entries still conta=
in
> >>>>>> old inodes. so a readdir always ORs the inode number with cur=
rent
> >>>>>> snapi.
> >>>>>> *) to access the snapshot, just switch to the old root inode
> >>>>>>
> >>>>>>I have actually switched back and forth, but finally decided to sti=
ck
> >>>>>>with the first implementation as the dentry contents magic is not
> >>>>>>required. It actually doesn't matter other than at the level of cod=
ing
> >>>>>>complexity/clarity. I also kept a local copy of the second
> >>>>>>implementation in case I need it.
> >>>>>>
> >>>>>>Comments are welcome.
> >>>>>>
> >>>>>>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread c=
an be
> >>>>>>used to read the contents of inodes etc. and see the snapshot.
> >>>>>>
> >>>>>>--
> >>>>>>Pradeep Padala
> >>>>>>http://ppadala.blogspot.com
> >>
> >>
>=20
> --
> Pradeep Padala
> http://ppadala.blogspot.com
>=20
>
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-24 03:25:19
|
Ok, I see it. You mean by using mount --bind. Yep, this is actually
quite cool. We don't have to do any readdir/read_inode magic to support
.snapshot. Only problem I see is that mount cannot be invoked by a
normal user, and a daemon with root privileges should be given the
responsibility of keeping track of these snapshot directories.
Marius Eriksen wrote:
> i believe in linux you can do things such as:
>
> alias /usr/foo/bar to /home/marius/foo
>
> and everything under /usr/foo/bar appears under /home/marius/foo.
>
> basically what the nullfs stackable filesystem does, but it works with
> dentrycache magic.
>
> so, for example, let's say you were to support ~/.snapshots/hourly.2,
> then you might have
>
> /mnt/hourly.2
>
> and alias
>
> /mnt/hourly.2/home/marius/ to /home/marius/.snapshots/hourly.2
>
> marius.
>
> On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>
>>Umm, interesting. What exactly do you mean by namespace aliasing? An
>>example would make things clearer for me.
>>
>>Marius Eriksen wrote:
>>
>>>i think that this should be purely a userland administrative thing.
>>>
>>>i.e., all that is needed is the ability to mount a particular
>>>snapshot. (and of course, to make said snapshots).
>>>
>>>then, you can have a daemon that manages ~/.snapshot directories.
>>>this should be especially doable with the namespace aliasing that
>>>linux can do.
>>>
>>>marius.
>>>
>>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>>>
>>>
>>>>Marius Eriksen wrote:
>>>>
>>>>
>>>>>how about making them mountable filesystems?
>>>>>
>>>>> mount -t lfs -osnapshot="january 12, 13:15" /mnt/old
>>>>
>>>>Yep, thought about it, and is probably what I will have as the first
>>>>version.
>>>>
>>>>In NetAPP's WAFL, for each dir .snapshot contains something like
>>>>hourly.0/ hourly.1/ ... weekly.0/ ...
>>>>Under these directories you can find snapshots for corresponding
>>>>directories.
>>>>
>>>>This is certainly doable, but requires more careful thought, as we may
>>>>have to alternate between IFILEs as the user cds back and forth, not
>>>>trivial. Thoughts ?
>>>>
>>>>
>>>>
>>>>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Hi,
>>>>>> The final piece in the puzzle is in. I have checked in the basic
>>>>>>framework for doing snapshots. I haven't yet decided on the user
>>>>>>intreface (may be .snapshots like NetAPP does), but you can access the
>>>>>>snapshots using lfsread program.
>>>>>> I want to explain a little bit about the design.
>>>>>> I have toyed around with two different ways of implementing
>>>>>>snapshots, theoritically actually same. To support any snapshots, one
>>>>>>has to make a copy of the inode map (in LFS case IFILE) and fs
>>>>>>modifications are done to the new IFILE only. Also, writes to existing
>>>>>>blocks (both inode and data) should be re-directed to new blocks (this
>>>>>>is almost trivial in LFS).
>>>>>> Now, how do we make a copy of inode map? We can simply make a new
>>>>>>IFILE file, copy the contents and start using the new file or We can
>>>>>>divide the IFILE into snapshots. For example, the snapshot index (snapi)
>>>>>>can be used to encode within the inode number (say first two bits). So,
>>>>>>the operations are roughly as follows
>>>>>>
>>>>>>1. Implementation using IFILE copy
>>>>>> *) the on-disk/in-memory inode structure has snapi.
>>>>>> *) user calls LFS_SNAP_CREATE ioctl
>>>>>> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
>>>>>> *) when a file is written its snapi is compared to current snapi, if
>>>>>> they differ new block is created.
>>>>>> *) to access the snapshot just switch the IFILE. Temporarily
>>>>>> switching IFILE back and forth is not that trivial though.
>>>>>>
>>>>>>2. Implementation using inode encoding
>>>>>> *) For each inode, inode number is derived with a simple function
>>>>>> like below
>>>>>>
>>>>>> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
>>>>>> { __u32 snapi = CURR_SNAPI(sb);
>>>>>>
>>>>>> snapi = snapi << 30;
>>>>>> return ino | snapi;
>>>>>> }
>>>>>>
>>>>>> *) user calls LFS_SNAP_CREATE ioctl
>>>>>> *) make a copy of existing inodes to correponding new inodes. For
>>>>>> example inode 1 is copied to 0x4001
>>>>>> *) switch the root inode to the new inode.
>>>>>> *) here comes the tricky part, the directory entries still contain
>>>>>> old inodes. so a readdir always ORs the inode number with current
>>>>>> snapi.
>>>>>> *) to access the snapshot, just switch to the old root inode
>>>>>>
>>>>>>I have actually switched back and forth, but finally decided to stick
>>>>>>with the first implementation as the dentry contents magic is not
>>>>>>required. It actually doesn't matter other than at the level of coding
>>>>>>complexity/clarity. I also kept a local copy of the second
>>>>>>implementation in case I need it.
>>>>>>
>>>>>>Comments are welcome.
>>>>>>
>>>>>>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can be
>>>>>>used to read the contents of inodes etc. and see the snapshot.
>>>>>>
>>>>>>--
>>>>>>Pradeep Padala
>>>>>>http://ppadala.blogspot.com
>>
>>
--
Pradeep Padala
http://ppadala.blogspot.com
|
|
From: Marius E. <ma...@mo...> - 2005-08-24 00:26:51
|
i believe in linux you can do things such as:
alias /usr/foo/bar to /home/marius/foo
and everything under /usr/foo/bar appears under /home/marius/foo.
basically what the nullfs stackable filesystem does, but it works with
dentrycache magic.
so, for example, let's say you were to support ~/.snapshots/hourly.2,
then you might have
/mnt/hourly.2
and alias=20
/mnt/hourly.2/home/marius/ to /home/marius/.snapshots/hourly.2
marius.
On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> Umm, interesting. What exactly do you mean by namespace aliasing? An
> example would make things clearer for me.
>=20
> Marius Eriksen wrote:
> > i think that this should be purely a userland administrative thing.
> >
> > i.e., all that is needed is the ability to mount a particular
> > snapshot. (and of course, to make said snapshots).
> >
> > then, you can have a daemon that manages ~/.snapshot directories.
> > this should be especially doable with the namespace aliasing that
> > linux can do.
> >
> > marius.
> >
> > On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >
> >>Marius Eriksen wrote:
> >>
> >>>how about making them mountable filesystems?
> >>>
> >>> mount -t lfs -osnapshot=3D"january 12, 13:15" /mnt/old
> >>
> >>Yep, thought about it, and is probably what I will have as the first
> >>version.
> >>
> >>In NetAPP's WAFL, for each dir .snapshot contains something like
> >>hourly.0/ hourly.1/ ... weekly.0/ ...
> >>Under these directories you can find snapshots for corresponding
> >>directories.
> >>
> >>This is certainly doable, but requires more careful thought, as we may
> >>have to alternate between IFILEs as the user cds back and forth, not
> >>trivial. Thoughts ?
> >>
> >>
> >>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >>>
> >>>
> >>>>Hi,
> >>>> The final piece in the puzzle is in. I have checked in the basic
> >>>>framework for doing snapshots. I haven't yet decided on the user
> >>>>intreface (may be .snapshots like NetAPP does), but you can access th=
e
> >>>>snapshots using lfsread program.
> >>>> I want to explain a little bit about the design.
> >>>> I have toyed around with two different ways of implementing
> >>>>snapshots, theoritically actually same. To support any snapshots, one
> >>>>has to make a copy of the inode map (in LFS case IFILE) and fs
> >>>>modifications are done to the new IFILE only. Also, writes to existin=
g
> >>>>blocks (both inode and data) should be re-directed to new blocks (thi=
s
> >>>>is almost trivial in LFS).
> >>>> Now, how do we make a copy of inode map? We can simply make a new
> >>>>IFILE file, copy the contents and start using the new file or We can
> >>>>divide the IFILE into snapshots. For example, the snapshot index (sna=
pi)
> >>>>can be used to encode within the inode number (say first two bits). S=
o,
> >>>>the operations are roughly as follows
> >>>>
> >>>>1. Implementation using IFILE copy
> >>>> *) the on-disk/in-memory inode structure has snapi.
> >>>> *) user calls LFS_SNAP_CREATE ioctl
> >>>> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
> >>>> *) when a file is written its snapi is compared to current snapi, =
if
> >>>> they differ new block is created.
> >>>> *) to access the snapshot just switch the IFILE. Temporarily
> >>>> switching IFILE back and forth is not that trivial though.
> >>>>
> >>>>2. Implementation using inode encoding
> >>>> *) For each inode, inode number is derived with a simple function
> >>>> like below
> >>>>
> >>>> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
> >>>> { __u32 snapi =3D CURR_SNAPI(sb);
> >>>>
> >>>> snapi =3D snapi << 30;
> >>>> return ino | snapi;
> >>>> }
> >>>>
> >>>> *) user calls LFS_SNAP_CREATE ioctl
> >>>> *) make a copy of existing inodes to correponding new inodes. For
> >>>> example inode 1 is copied to 0x4001
> >>>> *) switch the root inode to the new inode.
> >>>> *) here comes the tricky part, the directory entries still contai=
n
> >>>> old inodes. so a readdir always ORs the inode number with curr=
ent
> >>>> snapi.
> >>>> *) to access the snapshot, just switch to the old root inode
> >>>>
> >>>>I have actually switched back and forth, but finally decided to stick
> >>>>with the first implementation as the dentry contents magic is not
> >>>>required. It actually doesn't matter other than at the level of codin=
g
> >>>>complexity/clarity. I also kept a local copy of the second
> >>>>implementation in case I need it.
> >>>>
> >>>>Comments are welcome.
> >>>>
> >>>>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can=
be
> >>>>used to read the contents of inodes etc. and see the snapshot.
> >>>>
> >>>>--
> >>>>Pradeep Padala
> >>>>http://ppadala.blogspot.com
>=20
>
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-24 00:22:52
|
Umm, interesting. What exactly do you mean by namespace aliasing? An
example would make things clearer for me.
Marius Eriksen wrote:
> i think that this should be purely a userland administrative thing.
>
> i.e., all that is needed is the ability to mount a particular
> snapshot. (and of course, to make said snapshots).
>
> then, you can have a daemon that manages ~/.snapshot directories.
> this should be especially doable with the namespace aliasing that
> linux can do.
>
> marius.
>
> On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>
>>Marius Eriksen wrote:
>>
>>>how about making them mountable filesystems?
>>>
>>> mount -t lfs -osnapshot="january 12, 13:15" /mnt/old
>>
>>Yep, thought about it, and is probably what I will have as the first
>>version.
>>
>>In NetAPP's WAFL, for each dir .snapshot contains something like
>>hourly.0/ hourly.1/ ... weekly.0/ ...
>>Under these directories you can find snapshots for corresponding
>>directories.
>>
>>This is certainly doable, but requires more careful thought, as we may
>>have to alternate between IFILEs as the user cds back and forth, not
>>trivial. Thoughts ?
>>
>>
>>>On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>>>
>>>
>>>>Hi,
>>>> The final piece in the puzzle is in. I have checked in the basic
>>>>framework for doing snapshots. I haven't yet decided on the user
>>>>intreface (may be .snapshots like NetAPP does), but you can access the
>>>>snapshots using lfsread program.
>>>> I want to explain a little bit about the design.
>>>> I have toyed around with two different ways of implementing
>>>>snapshots, theoritically actually same. To support any snapshots, one
>>>>has to make a copy of the inode map (in LFS case IFILE) and fs
>>>>modifications are done to the new IFILE only. Also, writes to existing
>>>>blocks (both inode and data) should be re-directed to new blocks (this
>>>>is almost trivial in LFS).
>>>> Now, how do we make a copy of inode map? We can simply make a new
>>>>IFILE file, copy the contents and start using the new file or We can
>>>>divide the IFILE into snapshots. For example, the snapshot index (snapi)
>>>>can be used to encode within the inode number (say first two bits). So,
>>>>the operations are roughly as follows
>>>>
>>>>1. Implementation using IFILE copy
>>>> *) the on-disk/in-memory inode structure has snapi.
>>>> *) user calls LFS_SNAP_CREATE ioctl
>>>> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
>>>> *) when a file is written its snapi is compared to current snapi, if
>>>> they differ new block is created.
>>>> *) to access the snapshot just switch the IFILE. Temporarily
>>>> switching IFILE back and forth is not that trivial though.
>>>>
>>>>2. Implementation using inode encoding
>>>> *) For each inode, inode number is derived with a simple function
>>>> like below
>>>>
>>>> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
>>>> { __u32 snapi = CURR_SNAPI(sb);
>>>>
>>>> snapi = snapi << 30;
>>>> return ino | snapi;
>>>> }
>>>>
>>>> *) user calls LFS_SNAP_CREATE ioctl
>>>> *) make a copy of existing inodes to correponding new inodes. For
>>>> example inode 1 is copied to 0x4001
>>>> *) switch the root inode to the new inode.
>>>> *) here comes the tricky part, the directory entries still contain
>>>> old inodes. so a readdir always ORs the inode number with current
>>>> snapi.
>>>> *) to access the snapshot, just switch to the old root inode
>>>>
>>>>I have actually switched back and forth, but finally decided to stick
>>>>with the first implementation as the dentry contents magic is not
>>>>required. It actually doesn't matter other than at the level of coding
>>>>complexity/clarity. I also kept a local copy of the second
>>>>implementation in case I need it.
>>>>
>>>>Comments are welcome.
>>>>
>>>>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can be
>>>>used to read the contents of inodes etc. and see the snapshot.
>>>>
>>>>--
>>>>Pradeep Padala
>>>>http://ppadala.blogspot.com
|
|
From: Marius E. <ma...@mo...> - 2005-08-24 00:09:57
|
i think that this should be purely a userland administrative thing.
i.e., all that is needed is the ability to mount a particular
snapshot. (and of course, to make said snapshots).
then, you can have a daemon that manages ~/.snapshot directories.=20
this should be especially doable with the namespace aliasing that
linux can do.
marius.
On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> Marius Eriksen wrote:
> > how about making them mountable filesystems?
> >
> > mount -t lfs -osnapshot=3D"january 12, 13:15" /mnt/old
>=20
> Yep, thought about it, and is probably what I will have as the first
> version.
>=20
> In NetAPP's WAFL, for each dir .snapshot contains something like
> hourly.0/ hourly.1/ ... weekly.0/ ...
> Under these directories you can find snapshots for corresponding
> directories.
>=20
> This is certainly doable, but requires more careful thought, as we may
> have to alternate between IFILEs as the user cds back and forth, not
> trivial. Thoughts ?
>=20
> > On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> >
> >>Hi,
> >> The final piece in the puzzle is in. I have checked in the basic
> >>framework for doing snapshots. I haven't yet decided on the user
> >>intreface (may be .snapshots like NetAPP does), but you can access the
> >>snapshots using lfsread program.
> >> I want to explain a little bit about the design.
> >> I have toyed around with two different ways of implementing
> >>snapshots, theoritically actually same. To support any snapshots, one
> >>has to make a copy of the inode map (in LFS case IFILE) and fs
> >>modifications are done to the new IFILE only. Also, writes to existing
> >>blocks (both inode and data) should be re-directed to new blocks (this
> >>is almost trivial in LFS).
> >> Now, how do we make a copy of inode map? We can simply make a new
> >>IFILE file, copy the contents and start using the new file or We can
> >>divide the IFILE into snapshots. For example, the snapshot index (snapi=
)
> >>can be used to encode within the inode number (say first two bits). So,
> >>the operations are roughly as follows
> >>
> >>1. Implementation using IFILE copy
> >> *) the on-disk/in-memory inode structure has snapi.
> >> *) user calls LFS_SNAP_CREATE ioctl
> >> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
> >> *) when a file is written its snapi is compared to current snapi, i=
f
> >> they differ new block is created.
> >> *) to access the snapshot just switch the IFILE. Temporarily
> >> switching IFILE back and forth is not that trivial though.
> >>
> >>2. Implementation using inode encoding
> >> *) For each inode, inode number is derived with a simple function
> >> like below
> >>
> >> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
> >> { __u32 snapi =3D CURR_SNAPI(sb);
> >>
> >> snapi =3D snapi << 30;
> >> return ino | snapi;
> >> }
> >>
> >> *) user calls LFS_SNAP_CREATE ioctl
> >> *) make a copy of existing inodes to correponding new inodes. For
> >> example inode 1 is copied to 0x4001
> >> *) switch the root inode to the new inode.
> >> *) here comes the tricky part, the directory entries still contain
> >> old inodes. so a readdir always ORs the inode number with curre=
nt
> >> snapi.
> >> *) to access the snapshot, just switch to the old root inode
> >>
> >>I have actually switched back and forth, but finally decided to stick
> >>with the first implementation as the dentry contents magic is not
> >>required. It actually doesn't matter other than at the level of coding
> >>complexity/clarity. I also kept a local copy of the second
> >>implementation in case I need it.
> >>
> >>Comments are welcome.
> >>
> >>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can b=
e
> >>used to read the contents of inodes etc. and see the snapshot.
> >>
> >>--
> >>Pradeep Padala
> >>http://ppadala.blogspot.com
> >>
> >>
> >>-------------------------------------------------------
> >>SF.Net email is Sponsored by the Better Software Conference & EXPO
> >>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Pract=
ices
> >>Agile & Plan-Driven Development * Managing Projects & Teams * Testing &=
QA
> >>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5=
sf
> >>_______________________________________________
> >>Logfs-devel mailing list
> >>Log...@li...
> >>https://lists.sourceforge.net/lists/listinfo/logfs-devel
>=20
>
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-23 23:42:17
|
Marius Eriksen wrote:
> how about making them mountable filesystems?
>
> mount -t lfs -osnapshot="january 12, 13:15" /mnt/old
Yep, thought about it, and is probably what I will have as the first
version.
In NetAPP's WAFL, for each dir .snapshot contains something like
hourly.0/ hourly.1/ ... weekly.0/ ...
Under these directories you can find snapshots for corresponding
directories.
This is certainly doable, but requires more careful thought, as we may
have to alternate between IFILEs as the user cds back and forth, not
trivial. Thoughts ?
> On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
>
>>Hi,
>> The final piece in the puzzle is in. I have checked in the basic
>>framework for doing snapshots. I haven't yet decided on the user
>>intreface (may be .snapshots like NetAPP does), but you can access the
>>snapshots using lfsread program.
>> I want to explain a little bit about the design.
>> I have toyed around with two different ways of implementing
>>snapshots, theoritically actually same. To support any snapshots, one
>>has to make a copy of the inode map (in LFS case IFILE) and fs
>>modifications are done to the new IFILE only. Also, writes to existing
>>blocks (both inode and data) should be re-directed to new blocks (this
>>is almost trivial in LFS).
>> Now, how do we make a copy of inode map? We can simply make a new
>>IFILE file, copy the contents and start using the new file or We can
>>divide the IFILE into snapshots. For example, the snapshot index (snapi)
>>can be used to encode within the inode number (say first two bits). So,
>>the operations are roughly as follows
>>
>>1. Implementation using IFILE copy
>> *) the on-disk/in-memory inode structure has snapi.
>> *) user calls LFS_SNAP_CREATE ioctl
>> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
>> *) when a file is written its snapi is compared to current snapi, if
>> they differ new block is created.
>> *) to access the snapshot just switch the IFILE. Temporarily
>> switching IFILE back and forth is not that trivial though.
>>
>>2. Implementation using inode encoding
>> *) For each inode, inode number is derived with a simple function
>> like below
>>
>> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
>> { __u32 snapi = CURR_SNAPI(sb);
>>
>> snapi = snapi << 30;
>> return ino | snapi;
>> }
>>
>> *) user calls LFS_SNAP_CREATE ioctl
>> *) make a copy of existing inodes to correponding new inodes. For
>> example inode 1 is copied to 0x4001
>> *) switch the root inode to the new inode.
>> *) here comes the tricky part, the directory entries still contain
>> old inodes. so a readdir always ORs the inode number with current
>> snapi.
>> *) to access the snapshot, just switch to the old root inode
>>
>>I have actually switched back and forth, but finally decided to stick
>>with the first implementation as the dentry contents magic is not
>>required. It actually doesn't matter other than at the level of coding
>>complexity/clarity. I also kept a local copy of the second
>>implementation in case I need it.
>>
>>Comments are welcome.
>>
>>lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can be
>>used to read the contents of inodes etc. and see the snapshot.
>>
>>--
>>Pradeep Padala
>>http://ppadala.blogspot.com
>>
>>
>>-------------------------------------------------------
>>SF.Net email is Sponsored by the Better Software Conference & EXPO
>>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
>>Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
>>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
>>_______________________________________________
>>Logfs-devel mailing list
>>Log...@li...
>>https://lists.sourceforge.net/lists/listinfo/logfs-devel
|
|
From: Marius E. <ma...@mo...> - 2005-08-23 23:12:20
|
how about making them mountable filesystems?
mount -t lfs -osnapshot=3D"january 12, 13:15" /mnt/old
marius.
On 8/23/05, Pradeep Padala <pp...@ee...> wrote:
> Hi,
> The final piece in the puzzle is in. I have checked in the basic
> framework for doing snapshots. I haven't yet decided on the user
> intreface (may be .snapshots like NetAPP does), but you can access the
> snapshots using lfsread program.
> I want to explain a little bit about the design.
> I have toyed around with two different ways of implementing
> snapshots, theoritically actually same. To support any snapshots, one
> has to make a copy of the inode map (in LFS case IFILE) and fs
> modifications are done to the new IFILE only. Also, writes to existing
> blocks (both inode and data) should be re-directed to new blocks (this
> is almost trivial in LFS).
> Now, how do we make a copy of inode map? We can simply make a new
> IFILE file, copy the contents and start using the new file or We can
> divide the IFILE into snapshots. For example, the snapshot index (snapi)
> can be used to encode within the inode number (say first two bits). So,
> the operations are roughly as follows
>=20
> 1. Implementation using IFILE copy
> *) the on-disk/in-memory inode structure has snapi.
> *) user calls LFS_SNAP_CREATE ioctl
> *) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
> *) when a file is written its snapi is compared to current snapi, if
> they differ new block is created.
> *) to access the snapshot just switch the IFILE. Temporarily
> switching IFILE back and forth is not that trivial though.
>=20
> 2. Implementation using inode encoding
> *) For each inode, inode number is derived with a simple function
> like below
>=20
> static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
> { __u32 snapi =3D CURR_SNAPI(sb);
>=20
> snapi =3D snapi << 30;
> return ino | snapi;
> }
>=20
> *) user calls LFS_SNAP_CREATE ioctl
> *) make a copy of existing inodes to correponding new inodes. For
> example inode 1 is copied to 0x4001
> *) switch the root inode to the new inode.
> *) here comes the tricky part, the directory entries still contain
> old inodes. so a readdir always ORs the inode number with current
> snapi.
> *) to access the snapshot, just switch to the old root inode
>=20
> I have actually switched back and forth, but finally decided to stick
> with the first implementation as the dentry contents magic is not
> required. It actually doesn't matter other than at the level of coding
> complexity/clarity. I also kept a local copy of the second
> implementation in case I need it.
>=20
> Comments are welcome.
>=20
> lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can be
> used to read the contents of inodes etc. and see the snapshot.
>=20
> --
> Pradeep Padala
> http://ppadala.blogspot.com
>=20
>=20
> -------------------------------------------------------
> SF.Net email is Sponsored by the Better Software Conference & EXPO
> September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practic=
es
> Agile & Plan-Driven Development * Managing Projects & Teams * Testing & Q=
A
> Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
> _______________________________________________
> Logfs-devel mailing list
> Log...@li...
> https://lists.sourceforge.net/lists/listinfo/logfs-devel
>=20
>
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-23 20:57:10
|
lfsprogs/runtests.sh (originally from Justin Piszcz ref:http://linuxgazette.net/102/piszcz.html) is modified to fit LFS needs. The first test is running fine on LFS, if anyone is interested. Others may or may not work. -- Pradeep Padala http://ppadala.blogspot.com |
|
From: Pradeep P. <pp...@ee...> - 2005-08-23 20:54:44
|
Hi,
The final piece in the puzzle is in. I have checked in the basic
framework for doing snapshots. I haven't yet decided on the user
intreface (may be .snapshots like NetAPP does), but you can access the
snapshots using lfsread program.
I want to explain a little bit about the design.
I have toyed around with two different ways of implementing
snapshots, theoritically actually same. To support any snapshots, one
has to make a copy of the inode map (in LFS case IFILE) and fs
modifications are done to the new IFILE only. Also, writes to existing
blocks (both inode and data) should be re-directed to new blocks (this
is almost trivial in LFS).
Now, how do we make a copy of inode map? We can simply make a new
IFILE file, copy the contents and start using the new file or We can
divide the IFILE into snapshots. For example, the snapshot index (snapi)
can be used to encode within the inode number (say first two bits). So,
the operations are roughly as follows
1. Implementation using IFILE copy
*) the on-disk/in-memory inode structure has snapi.
*) user calls LFS_SNAP_CREATE ioctl
*) make a copy of IFILE, ++snapi, set the new IFILE inode pointers
*) when a file is written its snapi is compared to current snapi, if
they differ new block is created.
*) to access the snapshot just switch the IFILE. Temporarily
switching IFILE back and forth is not that trivial though.
2. Implementation using inode encoding
*) For each inode, inode number is derived with a simple function
like below
static inline ino_t INUMBER(struct super_block *sb, ino_t ino)
{ __u32 snapi = CURR_SNAPI(sb);
snapi = snapi << 30;
return ino | snapi;
}
*) user calls LFS_SNAP_CREATE ioctl
*) make a copy of existing inodes to correponding new inodes. For
example inode 1 is copied to 0x4001
*) switch the root inode to the new inode.
*) here comes the tricky part, the directory entries still contain
old inodes. so a readdir always ORs the inode number with current
snapi.
*) to access the snapshot, just switch to the old root inode
I have actually switched back and forth, but finally decided to stick
with the first implementation as the dentry contents magic is not
required. It actually doesn't matter other than at the level of coding
complexity/clarity. I also kept a local copy of the second
implementation in case I need it.
Comments are welcome.
lfsprogs/lfssnap.c can be used to make snapshot. lfsprogs/lfsread can be
used to read the contents of inodes etc. and see the snapshot.
--
Pradeep Padala
http://ppadala.blogspot.com
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-18 22:01:42
|
Hi,
I have run a very basic read/write test, and the comparison between
ext2 and LFS is at http://logfs.sourceforg.net/perf.html. I have
disabled segusage table maintainance as it was causing Oops. One thing
to note that with bigger segment size, the write performance is closer
to ext2 performance. Note that the scales are a little different for the
different segment size graphs. I am still meddling with OOo to bring all
the graphs to the same scale.
The reasons for poor performance, I think is due to the lock
contention. Right now, locking is coarse-grained and needs to be refined.
Don't read too much into the graphs though. I was just happy to see
LFS not crash during these tests :-) I have also tried IOZone, but the
results are quite random and an strace revealed that it's doing a lot of
things, and LFS is not yet ready for the heavy iron.
The script used for testing is lfsprogs/bench.pl. I have got
Justin's (Author of benchmarking article
http://linuxgazette.net/102/piszcz.html) script, and will be doing more
tests with that.
Cheers,
--
Pradeep Padala
http://ppadala.blogspot.com
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-18 18:49:34
|
Hi,
I have checked in a functional cleaner into the main trunk. The
cleaner is connected to the kernel side using ioctls, and a lot of
functionality needed by the cleaner is added to the kernel.
I have tested only the main flow, and the amount of functionality
added to the kernel might have made the kernel module unstable.
If you want to try the cleaner, run it with the -q option to run it
only once.
Detailed Changelog below
*) lfsprogs
- segsum, segusage entries added for the first few inodes
- i_ino added, needed by the cleaner to identify the inode to
which an inode block belongs.
*) Cleaner
- cost-benefit algorithm working
- finfo, seguse entries read and updated correctly.
- reading, and writing through ioctls is functional
*) Kernel-side
- seguse.c added (seguse table functionality)
- segsum.c added (segment summary, finfo and friends
functionality)
- ioctl.c added (ioctl functionality)
--
Pradeep Padala
http://ppadala.blogspot.com
|
|
From: Pradeep P. <pp...@ee...> - 2005-08-18 18:38:11
|
Hi,
I have checked in a functional cleaner into the main trunk. The
cleaner is connected to the kernel side using ioctls, and a lot of
functionality needed by the cleaner is added to the kernel.
I have tested only the main flow, and the amount of functionality
added to the kernel might have made the kernel module unstable.
If you want to try the cleaner, run it with the -q option to run it
only once.
Detailed Changelog below
*) lfsprogs
- segsum, segusage entries added for the first few inodes
- i_ino added, needed by the cleaner to identify the inode to
which an inode block belongs.
*) Cleaner
- cost-benefit algorithm working
- finfo, seguse entries read and updated correctly.
- reading, and writing through ioctls is functional
*) Kernel-side
- seguse.c added (seguse table functionality)
- segsum.c added (segment summary, finfo and friends
functionality)
- ioctl.c added (ioctl functionality)
--
Pradeep Padala
http://ppadala.blogspot.com
|