Re: [Next3-users] Kernel bug..
Brought to you by:
amir73il
Menu
▾
▴
Re: [Next3-users] Kernel bug..
|
From: Amir G. <ami...@gm...> - 2012-04-27 11:25:27
|
Hi Kim, I see what the problem is. next3 version 1.0.13-6 had some bugs with regards to snapshot size > 4TB, which where resolved by this commit: https://github.com/amir73il/ext4-snapshots/commit/87a6639f2f440685d5a550b0ce759ad39e511e58 You need to upgrade to latest version 1.0.13-8 from Aug 2, 2011. Unfortunately, I seem to have neglected the next3 releases since I started releasing ext4-snapshots and this is the reason you did not get the latest next3 version when you installed it from the web site. I also did write any coherent documentation how to get the code from github and build the latests version. The Makefile used to create a release is in the next3-utils repository under patch directory: https://github.com/amir73il/next3-utils/blob/master/patch/Makefile If you like I can walk you through creating a new next3_snapshot.module.patch ('make next3_snapshot KERNEL=2.6.32') or you can wait until I clear some time to do it myself (and keep reminding me about it). Amir. On Fri, Apr 27, 2012 at 12:46 PM, Kim Hagen <kh...@mu...>wrote: > Hi Amir, > > The crash happens every day we create a snapshot it has happened for 3 > days straight. > > The size of our volume is 10TB. > > /dev/mapper/BackupData-BackupData 9.9T 478G 8.9T 5% > --- Logical volume --- > LV Name /dev/BackupData/BackupData > VG Name BackupData > LV UUID jndfk9-GYHb-6aqu-3MKV-fLml-OFd4-3qHJNg > LV Write Access read/write > LV Status available > # open 1 > LV Size 10.00 TiB > Current LE 2621440 > Segments 1 > Allocation inherit > Read ahead sectors auto > - currently set to 256 > Block device 253:1 > > > We have used this volume for 5 days, first 2 days were for testing > (without rsync) we also installed next3 at the same time we started using > the volume. > > The output of the fsck... (i had already removed the snapshot yesterday > and removed the cron job that takes the snapshots): > > # fsck.next3 -nf -C0 /dev/mapper/BackupData-BackupData > e2fsck 1.41.14-next3-1.0.13-1 (22-Feb-2011) > Pass 1: Checking inodes, blocks, and sizes > Pass 2: Checking directory structure > > Pass 3: Checking directory connectivity > > Pass 4: Checking reference counts > > Pass 5: Checking group summary information > > BackupData: 519307/671088640 files (3.7% non-contiguous), > 167287708/2684354560 blocks > > This is what is in the logs... (sorry for the length): > > Apr 26 07:00:02 nl-gouda-07 kernel: [71796.800005] snapshot: > next3_snapshot_enable() called with snapshot file (ino=21061634) not on list > Apr 26 07:00:02 nl-gouda-07 kernel: [71796.858621] snapshot: snapshot (36) > created > Apr 26 07:00:02 nl-gouda-07 kernel: [71797.094522] snapshot: snapshot (36) > activated > Apr 26 07:00:02 nl-gouda-07 kernel: [71797.096916] snapshot: snapshot (36) > has been taken > Apr 26 07:16:47 nl-gouda-07 kernel: [72802.630978] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:16:48 nl-gouda-07 kernel: [72803.712051] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:16:48 nl-gouda-07 kernel: [72803.773532] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:16:56 nl-gouda-07 kernel: [72810.923815] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:18:17 nl-gouda-07 kernel: [72892.729274] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:55 nl-gouda-07 kernel: [73530.588936] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:56 nl-gouda-07 kernel: [73530.839443] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:56 nl-gouda-07 kernel: [73531.517396] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:57 nl-gouda-07 kernel: [73532.444935] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:57 nl-gouda-07 kernel: [73532.491089] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:57 nl-gouda-07 kernel: [73532.758984] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:58 nl-gouda-07 kernel: [73533.479672] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:28:58 nl-gouda-07 kernel: [73533.747038] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:29:00 nl-gouda-07 kernel: [73535.182968] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:41:59 nl-gouda-07 kernel: [74314.708945] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:42:13 nl-gouda-07 kernel: [74328.253262] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.359183] snapshot: > next3_snapshot_get_undo_access(b:0/67162) failed! h_ref=1, err=-5 > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.359233] > next3_try_to_allocate_with_rsv: aborting transaction: IO failure in > __next3_journal_get_undo_access > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.359283] NEXT3-fs error (device > dm-1) in next3_new_blocks: IO failure > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.361688] NEXT3-fs error (device > dm-1) in next3_reserve_inode_write: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.363617] NEXT3-fs error (device > dm-1) in next3_dirty_inode: IO failure > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.365512] NEXT3-fs error (device > dm-1) in next3_inode_list_add: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.367397] NEXT3-fs error (device > dm-1) in next3_write_begin: IO failure > Apr 26 07:43:40 nl-gouda-07 rsyncd[19142]: rsync: write failed on > "/de-duisburg-02/file.pdf.restore" (in BackupData): Input/output error (5) > Apr 26 07:43:40 nl-gouda-07 rsyncd[19142]: rsync error: error in file IO > (code 11) at receiver.c(302) [receiver=3.0.7] > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.471732] snapshot: > next3_snapshot_get_undo_access(b:0/67162) failed! h_ref=2, err=-5 > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.471781] > __next3_free_blocks_sb_inode: aborting transaction: IO failure in > __next3_journal_get_undo_access > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.471831] NEXT3-fs error (device > dm-1) in __next3_free_blocks_sb_inode: IO failure > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.474006] NEXT3-fs error (device > dm-1) in next3_reserve_inode_write: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.475918] NEXT3-fs error (device > dm-1) in next3_truncate: IO failure > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.477827] NEXT3-fs error (device > dm-1) in next3_reserve_inode_write: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.479708] NEXT3-fs error (device > dm-1) in next3_inode_list_del: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.481625] NEXT3-fs error (device > dm-1) in next3_reserve_inode_write: Readonly filesystem > Apr 26 07:43:40 nl-gouda-07 kernel: [74415.483537] NEXT3-fs error (device > dm-1) in next3_delete_inode: IO failure > Apr 26 07:43:40 nl-gouda-07 rsyncd[19142]: rsync: connection unexpectedly > closed (1491 bytes received so far) [generator] > Apr 26 07:43:40 nl-gouda-07 rsyncd[19142]: rsync error: error in rsync > protocol data stream (code 12) at io.c(601) [generator=3.0.7] > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638221] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638233] attempt to access > beyond end of device > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638237] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638240] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638273] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638319] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.638380] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:46:59 nl-gouda-07 kernel: [74614.640734] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.254942] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.254953] attempt to access > beyond end of device > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.254956] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.254963] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.254995] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.255041] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.255089] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.257408] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563027] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563037] attempt to access > beyond end of device > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563041] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563044] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563077] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563123] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.563171] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:00 nl-gouda-07 kernel: [74615.566488] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839271] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839285] attempt to access > beyond end of device > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839288] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839293] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839326] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839372] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.839421] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:01 nl-gouda-07 kernel: [74615.843553] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:03 nl-gouda-07 rsyslogd: [origin software="rsyslogd" > swVersion="4.6.4" x-pid="1841" x-info="http://www.rsyslog.com"] rsyslogd > was HUPed, type 'lightweight'. > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302879] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302888] attempt to access > beyond end of device > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302892] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302896] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302928] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.302975] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.303023] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:05 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/......etc.Forum.pdf.fkslcK" (in BackupData) failed: > Input/output error (5) > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.305387] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523632] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523641] attempt to access > beyond end of device > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523645] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523649] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523681] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523727] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.523776] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:05 nl-gouda-07 kernel: [74620.525947] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:05 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/....etc.pdf.9RoRu9" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355528] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355537] attempt to access > beyond end of device > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355541] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355545] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355577] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355632] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.355681] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.358001] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:06 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.fUXwdB" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451537] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451545] attempt to access > beyond end of device > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451548] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451552] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451583] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451630] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.451678] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:06 nl-gouda-07 kernel: [74621.453915] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:06 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.pmPwd3" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899588] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899598] attempt to access > beyond end of device > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899601] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899605] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899637] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899683] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.899731] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:07 nl-gouda-07 kernel: [74621.903637] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:07 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.zWyoww" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571597] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571607] attempt to access > beyond end of device > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571610] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571614] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571646] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571692] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.571740] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.574126] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:07 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.ZeByM1" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.739001] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.739009] attempt to access > beyond end of device > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.739013] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.739017] snapshot: failed to > read COW bitmap #67168 of snapshot (36) > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.745017] snapshot: > next3_snapshot_get_write_access(b:1/67168) failed! h_ref=1, err=-5 > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.745064] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.745112] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:07 nl-gouda-07 kernel: [74622.747357] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:07 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.1uWVwx" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289794] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289802] attempt to access > beyond end of device > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289805] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289809] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289841] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289888] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.289936] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:08 nl-gouda-07 kernel: [74623.292109] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:08 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.dwNJO4" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547394] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547405] attempt to access > beyond end of device > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547409] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547413] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547445] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547491] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.547540] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:10 nl-gouda-07 kernel: [74625.550711] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:10 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.xjB0JI" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131379] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131389] attempt to access > beyond end of device > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131393] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131397] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131429] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131476] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.131533] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:13 nl-gouda-07 kernel: [74628.134510] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:13 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.loyFbu" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386694] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386703] attempt to access > beyond end of device > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386707] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386711] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386743] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386789] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.386837] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:15 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.L8Mocm" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:15 nl-gouda-07 kernel: [74630.389213] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584440] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584452] attempt to access > beyond end of device > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584455] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584460] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584492] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584539] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.584587] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:20 nl-gouda-07 kernel: [74635.587525] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:20 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.DjEhlt" (in BackupData) failed: Input/output > error (5) > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203074] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203085] attempt to access > beyond end of device > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203088] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203093] snapshot: failed to > read COW bitmap #67169 of snapshot (36) > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203126] snapshot: > next3_snapshot_get_write_access(b:1/67169) failed! h_ref=1, err=-5 > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203172] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.203221] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:47:23 nl-gouda-07 kernel: [74638.207136] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:47:23 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.BdBO6H" (in BackupData) failed: Input/output > error (5) > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278241] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278249] attempt to access > beyond end of device > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278253] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278257] snapshot: failed to > read COW bitmap #67171 of snapshot (36) > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278289] snapshot: > next3_snapshot_get_write_access(b:1/67171) failed! h_ref=1, err=-5 > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278335] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.278383] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:48:57 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.ls.Lj38Ce" (in BackupData) failed: Input/output > error (5) > Apr 26 07:48:57 nl-gouda-07 kernel: [74732.281008] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589644] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589654] attempt to access > beyond end of device > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589657] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589661] snapshot: failed to > read COW bitmap #67174 of snapshot (36) > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589694] snapshot: > next3_snapshot_get_write_access(b:1/67174) failed! h_ref=1, err=-5 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589740] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.589788] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.592176] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:10 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc (in BackupData) failed: Input/output error (5) > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603200] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603207] attempt to access > beyond end of device > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603210] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603214] snapshot: failed to > read COW bitmap #67174 of snapshot (36) > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603246] snapshot: > next3_snapshot_get_write_access(b:1/67174) failed! h_ref=1, err=-5 > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603298] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.603347] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:50:10 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.ls.NiNvWQ" (in BackupData) failed: Input/output > error (5) > Apr 26 07:50:10 nl-gouda-07 kernel: [74805.605316] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565081] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565091] attempt to access > beyond end of device > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565094] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565098] snapshot: failed to > read COW bitmap #67173 of snapshot (36) > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565131] snapshot: > next3_snapshot_get_write_access(b:1/67173) failed! h_ref=1, err=-5 > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565177] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.565225] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:50:43 nl-gouda-07 kernel: [74838.567610] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:43 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.pdf.9NvFJ9" (in BackupData) failed: Input/output > error (5) > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.797944] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.797952] attempt to access > beyond end of device > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.797955] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.797959] snapshot: failed to > read COW bitmap #67173 of snapshot (36) > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.797991] snapshot: > next3_snapshot_get_write_access(b:1/67173) failed! h_ref=1, err=-5 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.798037] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.798085] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.800157] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:44 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc failed: Input/output error (5) > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802769] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802777] attempt to access > beyond end of device > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802780] dm-1: rw=0, > want=23732295768, limit=21474836480 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802784] snapshot: failed to > read COW bitmap #67173 of snapshot (36) > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802816] snapshot: > next3_snapshot_get_write_access(b:1/67173) failed! h_ref=1, err=-5 > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802862] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.802910] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 07:50:44 nl-gouda-07 kernel: [74838.805242] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 07:50:44 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc.ppt.hcFvdt" (in BackupData) failed: Input/output > error (5) > Apr 26 08:01:42 nl-gouda-07 kernel: [75497.760927] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:06:20 nl-gouda-07 kernel: [75774.795683] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:08:33 nl-gouda-07 kernel: [75907.799568] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:08:58 nl-gouda-07 kernel: [75933.145117] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:05 nl-gouda-07 kernel: [75939.889330] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:12 nl-gouda-07 kernel: [75947.223080] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:25 nl-gouda-07 kernel: [75960.531557] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:43 nl-gouda-07 kernel: [75978.034688] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:50 nl-gouda-07 kernel: [75985.114594] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:50 nl-gouda-07 kernel: [75985.296441] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497242] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497254] ------------[ cut here > ]------------ > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497275] WARNING: at > /usr/src/next3-1.0.13/next3/snapshot.c:486 > next3_snapshot_test_cow_bitmap+0x208/0x415 [next3]() > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497278] Hardware name: ProLiant > DL380 G5 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497279] Modules linked in: > hcpdriver(P) cpufreq_conservative cpufreq_powersave cpufreq_userspace > cpufreq_stats dm_round_robin dm_multipath scsi_dh loop radeon ttm drm_kms > _helper drm i2c_algo_bit i2c_core i5000_edac snd_pcm snd_timer snd > soundcore snd_page_alloc ipmi_si pcspkr psmouse ipmi_msghandler edac_core > shpchp hpilo serio_raw hpwdt i5k_amb pci_hotplug joydev rng_core evdev c > ontainer button processor ext4 jbd2 crc16 dm_mod raid10 raid456 > async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx > raid1 raid0 multipath linear md_mod ext3 next3 mbcache jbd ses enclosure > sd_m > od crc_t10dif usbhid hid sg sr_mod cdrom ata_generic hpsa thermal ata_piix > uhci_hcd mptspi scsi_transport_spi mptsas cciss libata bnx2 mptscsih > mptbase ehci_hcd scsi_transport_sas scsi_mod thermal_sys usbcore nls_ > base [last unloaded: scsi_wait_scan] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497345] Pid: 27670, comm: rsync > Tainted: P 2.6.32-5-amd64 #1 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497347] Call Trace: > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497355] [<ffffffffa018fd2b>] ? > next3_snapshot_test_cow_bitmap+0x208/0x415 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497362] [<ffffffffa018fd2b>] ? > next3_snapshot_test_cow_bitmap+0x208/0x415 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497368] [<ffffffff8104df9c>] ? > warn_slowpath_common+0x77/0xa3 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497375] [<ffffffffa018fd2b>] ? > next3_snapshot_test_cow_bitmap+0x208/0x415 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497381] [<ffffffffa01693c9>] ? > do_get_write_access+0x3e3/0x441 [jbd] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497389] [<ffffffffa01907ba>] ? > next3_snapshot_test_and_cow+0x2ac/0x616 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497397] [<ffffffffa018f336>] ? > __next3_journal_get_write_access_inode+0x40/0x8f [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497404] [<ffffffffa017e33d>] ? > next3_new_inode+0x4f5/0x9f1 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497409] [<ffffffffa0169ba0>] ? > journal_start+0x96/0xc6 [jbd] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497416] [<ffffffffa01874a4>] ? > next3_create+0x96/0x113 [next3] > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497421] [<ffffffff810f6448>] ? > generic_permission+0xe/0x8a > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497424] [<ffffffff810f7f44>] ? > vfs_create+0x6d/0x89 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497427] [<ffffffff810f8e6d>] ? > do_filp_open+0x31e/0x94b > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497432] [<ffffffff810ed563>] ? > do_sys_open+0x55/0xfc > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497436] [<ffffffff81010b42>] ? > system_call_fastpath+0x16/0x1b > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497438] ---[ end trace > 90d840e343cd0075 ]--- > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497441] snapshot: failed to > read COW bitmap #75360 of snapshot (36) > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497474] snapshot: > next3_snapshot_get_write_access(b:1/75360) failed! h_ref=1, err=-5 > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497520] next3_new_inode: > aborting transaction: IO failure in __next3_journal_get_write_access_inode > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.497568] NEXT3-fs error (device > dm-1) in next3_new_inode: IO failure > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.499968] NEXT3-fs error (device > dm-1) in next3_create: IO failure > Apr 26 08:09:52 nl-gouda-07 rsyncd[27657]: rsync: mkstemp > "/de-duisburg-02/.....etc failed: Input/output error (5) > Apr 26 08:09:52 nl-gouda-07 kernel: [75987.723775] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:09:53 nl-gouda-07 kernel: [75987.809208] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:10:03 nl-gouda-07 kernel: [75998.282167] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.925187] NEXT3-fs warning > (device dm-1): next3_block_to_path: block < 0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970342] ------------[ cut here > ]------------ > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970376] kernel BUG at > /usr/src/next3-1.0.13/next3/snapshot.c:1063! > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970405] invalid opcode: 0000 > [#1] SMP > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970434] last sysfs file: > /sys/devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:01.0/0000:0e:00.0/host3/port-3:0/end_device-3:0/target3:0:0/3:0:0:1/state > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970504] CPU 1 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970526] Modules linked in: > hcpdriver(P) cpufreq_conservative cpufreq_powersave cpufreq_userspace > cpufreq_stats dm_round_robin dm_multipath scsi_dh loop radeon ttm drm_kms > _helper drm i2c_algo_bit i2c_core i5000_edac snd_pcm snd_timer snd > soundcore snd_page_alloc ipmi_si pcspkr psmouse ipmi_msghandler edac_core > shpchp hpilo serio_raw hpwdt i5k_amb pci_hotplug joydev rng_core evdev c > ontainer button processor ext4 jbd2 crc16 dm_mod raid10 raid456 > async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx > raid1 raid0 multipath linear md_mod ext3 next3 mbcache jbd ses enclosure > sd_m > od crc_t10dif usbhid hid sg sr_mod cdrom ata_generic hpsa thermal ata_piix > uhci_hcd mptspi scsi_transport_spi mptsas cciss libata bnx2 mptscsih > mptbase ehci_hcd scsi_transport_sas scsi_mod thermal_sys usbcore nls_ > base [last unloaded: scsi_wait_scan] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.970974] Pid: 27670, comm: rsync > Tainted: P W 2.6.32-5-amd64 #1 ProLiant DL380 G5 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971020] RIP: > 0010:[<ffffffffa0190335>] [<ffffffffa0190335>] > next3_snapshot_test_and_move+0x8b/0x264 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971081] RSP: > 0018:ffff88012c541888 EFLAGS: 00010202 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971108] RAX: ffff88012b839000 > RBX: 0000000000000000 RCX: 0000000082200883 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971138] RDX: ffff88010e39cba8 > RSI: ffff8801289fc6f0 RDI: ffffffffa0197af0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971168] RBP: ffff88010e39cba8 > R08: 0000000000000001 R09: 0000000000000001 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971198] R10: 0000000000000004 > R11: dead000000100100 R12: 0000000000000001 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971228] R13: ffff8801289fc6f0 > R14: ffff88010e39cba8 R15: 0000000000000001 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971258] FS: > 00007f82d6344700(0000) GS:ffff880005440000(0000) knlGS:0000000000000000 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971304] CS: 0010 DS: 0000 ES: > 0000 CR0: 000000008005003b > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971331] CR2: 00007fff137bd428 > CR3: 000000012c534000 CR4: 00000000000006e0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971361] DR0: 0000000000000000 > DR1: 0000000000000000 DR2: 0000000000000000 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971391] DR3: 0000000000000000 > DR6: 00000000ffff0ff0 DR7: 0000000000000400 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971422] Process rsync (pid: > 27670, threadinfo ffff88012c540000, task ffff88012b918000) > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971475] Stack: > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971499] ffffffff81065058 > 0000000000001000 0000000082200883 ffffffffa0197af0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971533] <0> ffff88012b838000 > ffff88012c5419e8 0000000000000000 ffff880075670e00 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971585] <0> ffff880075670070 > ffff88012c5419e8 ffff88010e39cba8 0000000000000004 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971653] Call Trace: > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971679] [<ffffffff81065058>] ? > wake_bit_function+0x0/0x23 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971713] [<ffffffffa0181cf3>] ? > next3_get_blocks_handle+0xc06/0x1127 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971762] [<ffffffffa01825f3>] ? > next3_getblk+0x53/0x172 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971793] [<ffffffff8110da89>] ? > __find_get_block+0x176/0x186 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971824] [<ffffffff810168f3>] ? > read_tsc+0xa/0x20 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971855] [<ffffffffa018fd09>] ? > next3_snapshot_test_cow_bitmap+0x1e6/0x415 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971904] [<ffffffffa01693c9>] ? > do_get_write_access+0x3e3/0x441 [jbd] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971936] [<ffffffff812fbf38>] ? > __wait_on_bit+0x62/0x70 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.971967] [<ffffffffa01907ba>] ? > next3_snapshot_test_and_cow+0x2ac/0x616 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972019] [<ffffffffa018f336>] ? > __next3_journal_get_write_access_inode+0x40/0x8f [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972069] [<ffffffffa017e33d>] ? > next3_new_inode+0x4f5/0x9f1 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972101] [<ffffffffa0169ba0>] ? > journal_start+0x96/0xc6 [jbd] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972133] [<ffffffffa01874a4>] ? > next3_create+0x96/0x113 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972163] [<ffffffff810f6448>] ? > generic_permission+0xe/0x8a > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972192] [<ffffffff810f7f44>] ? > vfs_create+0x6d/0x89 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972220] [<ffffffff810f8e6d>] ? > do_filp_open+0x31e/0x94b > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972250] [<ffffffff810e6f20>] ? > __slab_alloc+0xd3/0x39b > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972278] [<ffffffff810f9502>] ? > getname+0x23/0x1a0 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972306] [<ffffffff810ed563>] ? > do_sys_open+0x55/0xfc > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972334] [<ffffffff81010b42>] ? > system_call_fastpath+0x16/0x1b > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972361] Code: 1e 44 89 0c 24 48 > 8b 54 24 20 45 31 c0 4c 8b 4c 24 10 48 8b 7c 24 18 48 89 e9 e8 02 f1 ff ff > 41 f6 45 14 08 75 05 4c 39 f5 75 04 <0f> 0b eb fe 4c 89 ef e8 8 > 3 f1 ff ff 31 c0 48 85 ed 74 68 0f b7 > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972563] RIP > [<ffffffffa0190335>] next3_snapshot_test_and_move+0x8b/0x264 [next3] > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972613] RSP <ffff88012c541888> > Apr 26 08:10:05 nl-gouda-07 kernel: [75999.972909] ---[ end trace > 90d840e343cd0076 ]--- > Apr 26 08:10:05 nl-gouda-07 rsyncd[27657]: rsync: connection unexpectedly > closed (318 bytes received so far) [generator] > Apr 26 08:10:05 nl-gouda-07 rsyncd[27657]: rsync error: error in rsync > protocol data stream (code 12) at io.c(601) [generator=3.0.7] > > > Regards, > > Kim. > > > > From: Amir Goldstein <ami...@gm...> > To: Kim Hagen <kh...@mu...>, > Cc: nex...@li... > Date: 04/26/2012 06:56 PM > Subject: Re: [Next3-users] Kernel bug.. > ------------------------------ > > > > Hi Kim, > > Did you see any other snapshot related errors/warning in the kernel log > prior to this crash? > > Is this crash reproducible? > > What is the size of the volume/fs you are using? > > Have you been using this volume for a long time with snapshots? > > Can you please run fsck.next3 -nf to see if there are snapshot consistency > errors on your fs. > > Thanks, > Amir. > > On Thu, Apr 26, 2012 at 3:06 PM, Kim Hagen <*kh...@mu...*<kh...@mu...>> > wrote: > Hi Amir, > > The kernel version is: > root@nl-gouda-07:~# uname -a > Linux nl-gouda-07 2.6.32-5-amd64 #1 SMP Thu Mar 22 17:26:33 UTC 2012 > x86_64 GNU/Linux > > next3 version is: > root@nl-gouda-07:~# next3 version > snapshot-version = next3 snapshot v1.0.13-6 (2-May-2010) > > > Kim > > > > > From: Amir Goldstein <*ami...@gm...* <ami...@gm...>> > To: Kim Hagen <*kh...@mu...*<kh...@mu...>>, > > Cc: *nex...@li...*<nex...@li...> > Date: 04/26/2012 12:30 PM > Subject: Re: [Next3-users] Kernel bug.. > ------------------------------ > > > > Hi Kim, > > Can you please specify kernel version (uname -a) and next3 version by > Running 'next3 version ' command > > Thanks, > Amir > > Sent from my private persona, enabled by Cellrox > > On Apr 26, 2012 12:45 PM, "Kim Hagen" <*kh...@mu...*<kh...@mu...>> > wrote: > Hi, > > Im experiencing a kernel bug on making a snapshot, I really like to use > next3 so i hope it can be fixed. > >From what I can see rsync has something to do with it. > > I am using Debian Squeeze on a ProLiant DL380 G5, > the file system is on top of an LVM volume on a multipath MSA2012sa > device. > > > [75999.970342] ------------[ cut here ]------------ > [75999.970376] kernel BUG at /usr/src/next3-1.0.13/next3/snapshot.c:1063! > [75999.970405] invalid opcode: 0000 [#1] SMP > [75999.970434] last sysfs file: > /sys/devices/pci0000:00/0000:00:02.0/0000:09:00.0/0000:0a:01.0/0000:0e:00.0/host3/port-3:0/end_device-3:0/target3:0:0/3:0:0:1/state > [75999.970504] CPU 1 > [75999.970526] Modules linked in: hcpdriver(P) cpufreq_conservative > cpufreq_powersave cpufreq_userspace cpufreq_stats dm_round_robin > dm_multipath scsi_dh loop radeon ttm drm_kms_helper drm i2c_algo_bit > i2c_core i5000_edac snd_pcm snd_timer snd soundcore snd_page_alloc ipmi_si > pcspkr psmouse ipmi_msghandler edac_core shpchp hpilo serio_raw hpwdt > i5k_amb pci_hotplug joydev rng_core evdev container button processor ext4 > jbd2 crc16 dm_mod raid10 raid456 async_raid6_recov async_pq raid6_pq > async_xor xor async_memcpy async_tx raid1 raid0 multipath linear md_mod > ext3 next3 mbcache jbd ses enclosure sd_mod crc_t10dif usbhid hid sg sr_mod > cdrom ata_generic hpsa thermal ata_piix uhci_hcd mptspi scsi_transport_spi > mptsas cciss libata bnx2 mptscsih mptbase ehci_hcd scsi_transport_sas > scsi_mod thermal_sys usbcore nls_base [last unloaded: scsi_wait_scan] > [75999.970974] Pid: 27670, comm: rsync Tainted: P W 2.6.32-5-amd64 > #1 ProLiant DL380 G5 > [75999.971020] RIP: 0010:[<ffffffffa0190335>] [<ffffffffa0190335>] > next3_snapshot_test_and_move+0x8b/0x264 [next3] > [75999.971081] RSP: 0018:ffff88012c541888 EFLAGS: 00010202 > [75999.971108] RAX: ffff88012b839000 RBX: 0000000000000000 RCX: > 0000000082200883 > [75999.971138] RDX: ffff88010e39cba8 RSI: ffff8801289fc6f0 RDI: > ffffffffa0197af0 > [75999.971168] RBP: ffff88010e39cba8 R08: 0000000000000001 R09: > 0000000000000001 > [75999.971198] R10: 0000000000000004 R11: dead000000100100 R12: > 0000000000000001 > [75999.971228] R13: ffff8801289fc6f0 R14: ffff88010e39cba8 R15: > 0000000000000001 > [75999.971258] FS: 00007f82d6344700(0000) GS:ffff880005440000(0000) > knlGS:0000000000000000 > [75999.971304] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b > [75999.971331] CR2: 00007fff137bd428 CR3: 000000012c534000 CR4: > 00000000000006e0 > [75999.971361] DR0: 0000000000000000 DR1: 0000000000000000 DR2: > 0000000000000000 > [75999.971391] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: > 0000000000000400 > [75999.971422] Process rsync (pid: 27670, threadinfo ffff88012c540000, > task ffff88012b918000) > [75999.971475] Stack: > [75999.971499] ffffffff81065058 0000000000001000 0000000082200883 > ffffffffa0197af0 > [75999.971533] <0> ffff88012b838000 ffff88012c5419e8 0000000000000000 > ffff880075670e00 > [75999.971585] <0> ffff880075670070 ffff88012c5419e8 ffff88010e39cba8 > 0000000000000004 > [75999.971653] Call Trace: > [75999.971679] [<ffffffff81065058>] ? wake_bit_function+0x0/0x23 > [75999.971713] [<ffffffffa0181cf3>] ? > next3_get_blocks_handle+0xc06/0x1127 [next3] > [75999.971762] [<ffffffffa01825f3>] ? next3_getblk+0x53/0x172 [next3] > [75999.971793] [<ffffffff8110da89>] ? __find_get_block+0x176/0x186 > [75999.971824] [<ffffffff810168f3>] ? read_tsc+0xa/0x20 > [75999.971855] [<ffffffffa018fd09>] ? > next3_snapshot_test_cow_bitmap+0x1e6/0x415 [next3] > [75999.971904] [<ffffffffa01693c9>] ? do_get_write_access+0x3e3/0x441 > [jbd] > [75999.971936] [<ffffffff812fbf38>] ? __wait_on_bit+0x62/0x70 > [75999.971967] [<ffffffffa01907ba>] ? > next3_snapshot_test_and_cow+0x2ac/0x616 [next3] > [75999.972019] [<ffffffffa018f336>] ? > __next3_journal_get_write_access_inode+0x40/0x8f [next3] > [75999.972069] [<ffffffffa017e33d>] ? next3_new_inode+0x4f5/0x9f1 [next3] > [75999.972101] [<ffffffffa0169ba0>] ? journal_start+0x96/0xc6 [jbd] > [75999.972133] [<ffffffffa01874a4>] ? next3_create+0x96/0x113 [next3] > [75999.972163] [<ffffffff810f6448>] ? generic_permission+0xe/0x8a > [75999.972192] [<ffffffff810f7f44>] ? vfs_create+0x6d/0x89 > [75999.972220] [<ffffffff810f8e6d>] ? do_filp_open+0x31e/0x94b > [75999.972250] [<ffffffff810e6f20>] ? __slab_alloc+0xd3/0x39b > [75999.972278] [<ffffffff810f9502>] ? getname+0x23/0x1a0 > [75999.972306] [<ffffffff810ed563>] ? do_sys_open+0x55/0xfc > [75999.972334] [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b > [75999.972361] Code: 1e 44 89 0c 24 48 8b 54 24 20 45 31 c0 4c 8b 4c 24 10 > 48 8b 7c 24 18 48 89 e9 e8 02 f1 ff ff 41 f6 45 14 08 75 05 4c 39 f5 75 04 > <0f> 0b eb fe 4c 89 ef e8 83 f1 ff ff 31 c0 48 85 ed 74 68 0f b7 > [75999.972563] RIP [<ffffffffa0190335>] > next3_snapshot_test_and_move+0x8b/0x264 [next3] > [75999.972613] RSP <ffff88012c541888> > [75999.972909] ---[ end trace 90d840e343cd0076 ]--- > > > Kind regards, > > Kim Hagen > > > > > > This message contains information that may be priviliged or confidential > and is intended only for use by the addressee(s). > You are not authorized to use or distribute this message or any part of it > in any form, unless you are the intended recipient. > If you receive this communication unintentionally or in error, please > inform the sender immediately by return e-mail and delete all copies of > this message. > > If you do not want to receive any e-mail from Multi Corporation and/or its > affiliates in the future, please send an e-mail to: *no...@mu...*<no...@mu...> > Multi Corporation B.V., Gouda, is registered with the trade register in > the Netherlands under no. 34237857. > > > ------------------------------------------------------------------------------------------------------------------------------- > > Dit bericht bevat informatie die vertrouwelijk kan zijn en is uitsluitend > bestemd voor de geadresseerde(n). > Het is u niet toegestaan dit bericht of delen ervan te gebruiken of te > verspreiden in welke vorm dan ook, tenzij u de beoogde ontvanger bent. > Indien u dit bericht abusievelijk of onterecht ontvangt, wordt u verzocht > de afzender direkt te informeren door het bericht te retourneren en alle > kopieen van dit bericht direkt te verwijderen. > > Indien u geen email meer wilt ontvangen van Multi Corporation of haar > groepsvennootschappen kunt u dit kenbaar maken door een email te sturen > naar : *no...@mu...* <no...@mu...> > Multi Corporation B.V., Gouda, staat ingeschreven in de Kamer van > Koophandel onder nr. 34237857. > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. *http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/*<http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/> > _______________________________________________ > Next3-users mailing list* > **Nex...@li...* <Nex...@li...>* > **https://lists.sourceforge.net/lists/listinfo/next3-users*<https://lists.sourceforge.net/lists/listinfo/next3-users> > > > > > > > This message contains information that may be priviliged or confidential > and is intended only for use by the addressee(s). > You are not authorized to use or distribute this message or any part of it > in any form, unless you are the intended recipient. > If you receive this communication unintentionally or in error, please > inform the sender immediately by return e-mail and delete all copies of > this message. > > If you do not want to receive any e-mail from Multi Corporation and/or its > affiliates in the future, please send an e-mail to: *no...@mu...*<no...@mu...> > Multi Corporation B.V., Gouda, is registered with the trade register in > the Netherlands under no. 34237857. > > > ------------------------------------------------------------------------------------------------------------------------------- > > Dit bericht bevat informatie die vertrouwelijk kan zijn en is uitsluitend > bestemd voor de geadresseerde(n). > Het is u niet toegestaan dit bericht of delen ervan te gebruiken of te > verspreiden in welke vorm dan ook, tenzij u de beoogde ontvanger bent. > Indien u dit bericht abusievelijk of onterecht ontvangt, wordt u verzocht > de afzender direkt te informeren door het bericht te retourneren en alle > kopieen van dit bericht direkt te verwijderen. > > Indien u geen email meer wilt ontvangen van Multi Corporation of haar > groepsvennootschappen kunt u dit kenbaar maken door een email te sturen > naar : *no...@mu...* <no...@mu...> > Multi Corporation B.V., Gouda, staat ingeschreven in de Kamer van > Koophandel onder nr. 34237857. > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. *http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/*<http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/> > _______________________________________________ > Next3-users mailing list* > **Nex...@li...* <Nex...@li...>* > **https://lists.sourceforge.net/lists/listinfo/next3-users*<https://lists.sourceforge.net/lists/listinfo/next3-users> > > > > > > > > This message contains information that may be priviliged or confidential > and is intended only for use by the addressee(s). > You are not authorized to use or distribute this message or any part of it > in any form, unless you are the intended recipient. > If you receive this communication unintentionally or in error, please > inform the sender immediately by return e-mail and delete all copies of > this message. > > If you do not want to receive any e-mail from Multi Corporation and/or its > affiliates in the future, please send an e-mail to: no...@mu... > Multi Corporation B.V., Gouda, is registered with the trade register in > the Netherlands under no. 34237857. > > > ------------------------------------------------------------------------------------------------------------------------------- > > Dit bericht bevat informatie die vertrouwelijk kan zijn en is uitsluitend > bestemd voor de geadresseerde(n). > Het is u niet toegestaan dit bericht of delen ervan te gebruiken of te > verspreiden in welke vorm dan ook, tenzij u de beoogde ontvanger bent. > Indien u dit bericht abusievelijk of onterecht ontvangt, wordt u verzocht > de afzender direkt te informeren door het bericht te retourneren en alle > kopieen van dit bericht direkt te verwijderen. > > Indien u geen email meer wilt ontvangen van Multi Corporation of haar > groepsvennootschappen kunt u dit kenbaar maken door een email te sturen > naar : no...@mu... > Multi Corporation B.V., Gouda, staat ingeschreven in de Kamer van > Koophandel onder nr. 34237857. > > |
