Re: [Scst-devel] Honoring SYNC requests in combination with ZFS

[Sietse v. Z. <si...@wi...>]

Forgot to mention that you absolutely need to disable any write cache the drive itself provides. Independent from what you do with SCST and ZFS. This is most often overlooked with software RAID, HW RAID cards will normally disable the drives write cache.

hdparm -W 0 /dev/sdx

-Sietse


________________________________________
From: Sietse van Zanen [si...@wi...]
Sent: Monday, March 03, 2014 12:12
To: Thijs Cramer
Cc: scs...@li...; Vladislav Bolkhovitin
Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS

If SCST would not honor requests when they arrive, I would be using other iSCSI target software, and you probably too. :)



There is really no sync logic in the SCSI command set and SCST processes SCSI commands. So SCST will never be aware of an initator trying to sync writes. It queues, processes and notifies initiators about command completion.



A sync write is the iniator waiting for successful command completion, which may have been sent by the target after data is  cached and not actually written to backend storage.



So sync writes all come down to the backend storage used. If you want to be absolutely sure data has been written to stable storage, use blockio, and make sure your block devices have write cache disabled (or BBU).



When using fileio you will at least have to disable nv_cache, or the Linux page cache will be used and writes are certifiable not synced (and cached in non BBU).



When using ZFS as the backend storage you will always have nv_cache set to 0, as ZFS has its own caching and also using the page cache will cause double caching. ZFS helps you out with this inherent problem by offering the sync=always setting. However ZFS is awfully slow when not using ZIL with sync=always.



When using any other filesystem, you will most likely have to set write_through and o_direct on the device in SCST.



-Sietse







________________________________
From: Thijs Cramer [thi...@gm...]
Sent: Sunday, March 02, 2014 14:06
To: Sietse van Zanen
Cc: Vladislav Bolkhovitin; scs...@li...
Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS

sync=always wil indeed garantuee synced writes on ZFS, but that is not what i mean.
What i want, is for SCST to honor sync requests when they arrive.
If nv_cache 0 does that, then sync=always is not needed anymore.


2014-03-02 13:58 GMT+01:00 Sietse van Zanen <si...@wi...<mailto:si...@wi...>>:
I don't believe so. ZFS has its own caching mechanism, so this setting will not have any effect, just like o_direct. The effect is achieved by the zfs set sync=always setting for the filesystem or zvol..

-Sietse

-----Original Message-----
From: Vladislav Bolkhovitin [mailto:vs...@vl...<mailto:vs...@vl...>]
Sent: Friday, February 28, 2014 22:50
To: Sietse van Zanen
Cc: Thijs Cramer; scs...@li...<mailto:scs...@li...>
Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS

Sietse van Zanen, on 02/28/2014 02:57 AM wrote:
> You have to do 3 things.
>
>
>
> 1. Set nv_cache to 0 on the scst side

You mean write_through in 1 here, I believe.

Vlad

> 2. zfs set sync=always for the filesystem or zvol you have the file on.
>
> 3. Use ZIL or your performance will be terrible, definitely with SATA
>
>
>
> -Sietse
>
>
>
> ________________________________
> From: Thijs Cramer [thi...@gm...<mailto:thi...@gm...>]
> Sent: Tuesday, February 25, 2014 08:59
> To: scs...@li...<mailto:scs...@li...>
> Subject: [Scst-devel] Honoring SYNC requests in combination with ZFS
>
> Guys,
>
> Is there a definitive answer on whether we can use the fileio handler in combination with ZFSonLinux?
> I'm seeing extreme performance statistics > 18.000 IOPS over iSCSI when using ZFSonLinx on 4*4TB drives with sync=standard. The disks are exported over iSCSI through a ZVOL and I've defined a fileio handler in SCST. The iSCSI volume is exported to an ESXi cluster.
>
> I think this means that SCST doesn't honor the SYNC requests from ESXi.
>
> I know the linux page cache is involved when using the fileio handler, but i thought sync requests would still be honored? Am i mistaking?
>
> - Thijs
>
> ----------------------------------------------------------------------
> -------- Flow-based real-time traffic analytics software. Cisco
> certified tool.
> Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> Customize your own dashboards, set traffic alerts and generate reports.
> Network behavioral analysis & security monitoring. All-in-one tool.
> http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.
> clktrk _______________________________________________
> Scst-devel mailing list
> https://lists.sourceforge.net/lists/listinfo/scst-devel
>



------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works.
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
_______________________________________________
Scst-devel mailing list
https://lists.sourceforge.net/lists/listinfo/scst-devel