Re: [Next3-devel] ext4 snapshot merged to 3.1 kernel

[Amir G. <ami...@gm...>]

Hi Sergey,

I was wondering if you could help out in testing the latest 3.1 kernel module,
as you now have it running on Fedora-16.

Can you please try a full run of xfstest (latest official repo) on an
ext4 fs with snapshots.
There are some reports below from Greg about failing tests and there
is the old test 070
bug which does not seem to happen for everyone.

I also wanted to ask for a favor. I can no longer use the test server
I was using until recently,
so I was wondering if it would be possible for you to provide ssh
access to a test server,
for myself and Yongqiang, so we can run regression tests on it.
Yongqiang doesn't get the test 070 bug on his system, so if we see
that bug on the test server
it could be very helpful for debugging the problem.

Thanks,
Amir.

On Thu, Nov 3, 2011 at 5:06 AM, Yongqiang Yang <xia...@gm...> wrote:
> On Thu, Nov 3, 2011 at 3:22 AM, Amir Goldstein <ami...@gm...> wrote:
>> On Wed, Nov 2, 2011 at 9:00 PM, Greg Freemyer <gre...@gm...> wrote:
>>> On Wed, Nov 2, 2011 at 2:54 PM, Amir Goldstein <ami...@gm...> wrote:
>>>> On Wed, Nov 2, 2011 at 5:17 PM, Greg Freemyer <gre...@gm...> wrote:
>>>>> Amir,
>>>>>
>>>>> I am now running kernel 3.1.0 with the ext4dev-kmp compiled from home:next4.
>>>>
>>>> and this is the same kernel that worked fine with the kmp from filesystems repo?
>>>>
>>>
>>> No, that was 3.1.0-RC9
>>>
>>> So this is both a new kernel and newly compiled KMP.
>>
>> So maybe just to be sure that we didn't break anything with the merge,
>> you can try to install the KMP from fs repo.
>> I think it should run with kernel 3.1...
>>
>>>
>>>>>
>>>>> I tried xfstests "./check -g auto" from Aditya's git repo.
>>>>>
>>>>> It failed on both test 113 and 255.  That is on the first pass, so no
>>>>> snapshots were active.  I "assume" that the official ext4 in 3.1.0 doesn't
>>>>> fail, but I haven't tested it.
>>>>
>>>> can you send the detailed error reports of those tests.
>>>
>>> I don't find a lot of output for 113, but here's what I have:
>>>
>>>> cat 113.out.bad
>>> QA output created by 113
>>> brevity is wit...
>>>
>>> -----------------------------------------------
>>> aio-stress.1 : -s 120m
>>> -----------------------------------------------
>>> ./113: line 47: 32471 Segmentation fault      $here/ltp/aio-stress
>>
>> SEGFAULT usually means some IO error, so you should look at dmesg
>> for errors from ext4dev fs.
>>
>>> $_param $AIOSTRESS_AVOID -I $_count $_files >> $tmp.out 2>&1
>>>    aio-stress (count=1000) returned 0
>>> file size 120MB, record size 64KB, depth 64, ios per iteration 8
>>> max io_submit 8, buffer alignment set to 4KB
>>> threads 1 files 1 contexts 1 context offset 2MB verification off
>>>
>>>
>>>
>>> And for 255 it's
>>>
>>
>> I don't know much about this test, but I reckon Yongqiang knows a lot about it.
> There are falloc and punch hole tests in 255.  It should has nothing
> to do with snapshot.
>>
>> in general I don't like the fact that xfstest is not uptodate.
>> there have been some bugfixes in xfstest, especially in those 25? tests.
> Yeah. xfstests has a lot fixes in recent release, especially in fsx.
>>
>> Aditya, could  you rebase/merge your xfstests repo to recent xfstests.
>> This probably means changing the test number again...
>
> We'd better update to latest xfstests.  Three days ago, I updated my
> xfstests tree to latest one and found a bug in ext4 related to punch
> hole and the patch has been merged to upstream by Tytso and will
> appear in 3.2 releases.   I am not sure if it is related to this
> error.
>
> Yongqiang.
>>
>> Thanks,
>> Amir.
>>
>>
>>
>>
>>>> cat 255.out.bad
>>> QA output created by 255
>>>        1. into a hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        2. into allocated space
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        3. into unwritten space
>>> daa100df6e6711906b61c9ab5aa16032
>>>        4. hole -> data
>>> 0: [0..23]: hole
>>> 1: [24..31]: extent
>>> 2: [32..39]: hole
>>> cc63069677939f69a6e8f68cae6a6dac
>>>        5. hole -> unwritten
>>> daa100df6e6711906b61c9ab5aa16032
>>>        6. data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..39]: hole
>>> 1b3779878366498b28c702ef88c4a773
>>>        7. data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..31]: extent
>>> 3: [32..39]: hole
>>> 1b3779878366498b28c702ef88c4a773
>>>        8. unwritten -> hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        9. unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..31]: extent
>>> 3: [32..39]: hole
>>> cc63069677939f69a6e8f68cae6a6dac
>>>        10. hole -> data -> hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        11. data -> hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        12. unwritten -> data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> daa100df6e6711906b61c9ab5aa16032
>>>        13. data -> unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        14. data -> hole @ EOF
>>> 0: [0..23]: extent
>>> 1: [24..39]: hole
>>> e1f024eedd27ea6b1c3e9b841c850404
>>>        15. data -> hole @ 0
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        16. data -> cache cold ->hole
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        17. data -> hole in single block file
>>> 0: [0..7]: extent
>>> 13535fd4d496bf0b74bb2335aa4d1b31
>>>        1. into a hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        2. into allocated space
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        3. into unwritten space
>>> daa100df6e6711906b61c9ab5aa16032
>>>        4. hole -> data
>>> 0: [0..23]: hole
>>> 1: [24..31]: extent
>>> 2: [32..39]: hole
>>> cc63069677939f69a6e8f68cae6a6dac
>>>        5. hole -> unwritten
>>> daa100df6e6711906b61c9ab5aa16032
>>>        6. data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..39]: hole
>>> 1b3779878366498b28c702ef88c4a773
>>>        7. data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..31]: extent
>>> 3: [32..39]: hole
>>> 1b3779878366498b28c702ef88c4a773
>>>        8. unwritten -> hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        9. unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..31]: extent
>>> 3: [32..39]: hole
>>> cc63069677939f69a6e8f68cae6a6dac
>>>        10. hole -> data -> hole
>>> daa100df6e6711906b61c9ab5aa16032
>>>        11. data -> hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        12. unwritten -> data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> daa100df6e6711906b61c9ab5aa16032
>>>        13. data -> unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        14. data -> hole @ EOF
>>> 0: [0..23]: extent
>>> 1: [24..39]: hole
>>> e1f024eedd27ea6b1c3e9b841c850404
>>>        15. data -> hole @ 0
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        16. data -> cache cold ->hole
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        17. data -> hole in single block file
>>> 0: [0..7]: extent
>>> 13535fd4d496bf0b74bb2335aa4d1b31
>>>        1. into a hole
>>> 5a58e46082be047d0f13bee7974015b9
>>>        2. into allocated space
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        3. into unwritten space
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        4. hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        5. hole -> unwritten
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        6. data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        7. data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        8. unwritten -> hole
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        9. unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        10. hole -> data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        11. data -> hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        12. unwritten -> data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        13. data -> unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        14. data -> hole @ EOF
>>> 0: [0..23]: extent
>>> 1: [24..39]: hole
>>> e1f024eedd27ea6b1c3e9b841c850404
>>>        15. data -> hole @ 0
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        16. data -> cache cold ->hole
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        17. data -> hole in single block file
>>> 0: [0..7]: extent
>>> 13535fd4d496bf0b74bb2335aa4d1b31
>>>        1. into a hole
>>> 5a58e46082be047d0f13bee7974015b9
>>>        2. into allocated space
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        3. into unwritten space
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        4. hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        5. hole -> unwritten
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        6. data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        7. data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        8. unwritten -> hole
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        9. unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..23]: hole
>>> 2: [24..39]: extent
>>> cc58a7417c2d7763adc45b6fcd3fa024
>>>        10. hole -> data -> hole
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        11. data -> hole -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        12. unwritten -> data -> unwritten
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        13. data -> unwritten -> data
>>> 0: [0..7]: extent
>>> 1: [8..31]: hole
>>> 2: [32..39]: extent
>>> f6aeca13ec49e5b266cd1c913cd726e3
>>>        14. data -> hole @ EOF
>>> 0: [0..23]: extent
>>> 1: [24..39]: hole
>>> e1f024eedd27ea6b1c3e9b841c850404
>>>        15. data -> hole @ 0
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        16. data -> cache cold ->hole
>>> 0: [0..15]: hole
>>> 1: [16..39]: extent
>>> eecb7aa303d121835de05028751d301c
>>>        17. data -> hole in single block file
>>> 0: [0..7]: extent
>>> 13535fd4d496bf0b74bb2335aa4d1b31
>>>
>>>
>>>
>>> Greg
>>>
>>>>
>>>> Thanks,
>>>> Amir.
>>>>
>>>>>
>>>>> Greg
>>>>>
>>>>> On Mon, Oct 31, 2011 at 1:20 PM, Greg Freemyer <gre...@gm...>
>>>>> wrote:
>>>>>>
>>>>>> On Mon, Oct 31, 2011 at 1:04 PM, Amir Goldstein <ami...@gm...>
>>>>>> wrote:
>>>>>> > On Mon, Oct 31, 2011 at 6:28 PM, Amir Goldstein <ami...@gm...>
>>>>>> > wrote:
>>>>>> >> On Mon, Oct 31, 2011 at 6:03 PM, Greg Freemyer
>>>>>> >> <gre...@gm...> wrote:
>>>>>> >>> Amir,
>>>>>> >>>
>>>>>> >>> We currently have 2 KMP projects to be aware of:
>>>>>> >>>
>>>>>> >>> "filesystems/ext4dev-snapshots" is the released version which normal
>>>>>> >>> testers/people should be downloading from.  We need to do our best to
>>>>>> >>> keep it usable.
>>>>>> >>>
>>>>>> >>> "home:next4/ext4dev-snapshots" is the one that should only be used by
>>>>>> >>> the next4 team.
>>>>>> >>>
>>>>>> >>> I just uploaded your new patch to there and will see if it builds.
>>>>>> >>> (It was building last week with the old patch, but something happened
>>>>>> >>> to factory recently and our KMP no longer builds.
>>>>>> >>
>>>>>> >> That's my bad.
>>>>>> >> the patch ext4dev-trace.patch has an ugly ugly hack.
>>>>>> >> it has the absolute path of the OBS BUILD directory hardcoded in the
>>>>>> >> patch.
>>>>>> >> I just could find another way to make the KMP build properly.
>>>>>> >> Now OBS has changed the path from /usr/src/packages/BUILD
>>>>>> >> to /home/abuild/rpmbuild/BUILD.
>>>>>> >> I will try to fix that by fixing the hardcoded path.
>>>>>> >>
>>>>>> >
>>>>>> > So now the module builds for Factory, but not for Thumbleweed,
>>>>>> > which still builds under /usr/src/packages/BUILD :-(
>>>>>> > I guess we can live with that problem for now...
>>>>>> >
>>>>>> > Amir.
>>>>>>
>>>>>> Tumbleweed will be moving on in the not too distant future to 3.2-rcx
>>>>>> etc., so it is not a good target for us.
>>>>>>
>>>>>> Let's just worry about factory/12.1.  12.1 should stick with 3.1 for
>>>>>> the next 18 months.
>>>>>>
>>>>>> Factory will move on two at some point, so 12.1 is our real target.
>>>>>> OBS just doesn't have full 12.1 repo support yet.
>>>>>>
>>>>>> I'll try to test this build in the next day or so, then push it to the
>>>>>> filesystem repo.
>>>>>>
>>>>>> Greg
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Greg Freemyer
>>>>> Head of EDD Tape Extraction and Processing team
>>>>> Litigation Triage Solutions Specialist
>>>>> http://www.linkedin.com/in/gregfreemyer
>>>>> CNN/TruTV Aired Forensic Imaging Demo -
>>>>>
>>>>> http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retrieved/
>>>>>
>>>>> The Norcross Group
>>>>> The Intersection of Evidence & Technology
>>>>> http://www.norcrossgroup.com
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Greg Freemyer
>>> Head of EDD Tape Extraction and Processing team
>>> Litigation Triage Solutions Specialist
>>> http://www.linkedin.com/in/gregfreemyer
>>> CNN/TruTV Aired Forensic Imaging Demo -
>>>    http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retrieved/
>>>
>>> The Norcross Group
>>> The Intersection of Evidence & Technology
>>> http://www.norcrossgroup.com
>>>
>>
>
>
>
> --
> Best Wishes
> Yongqiang Yang
>