Re: [Scst-devel] Honoring SYNC requests in combination with ZFS

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Not quite true, ZFS has it's own mechanism to circumvent these
characteristics. It will send it's own SYNC / CACHE FLUSH commands to the
drives when it's so called transaction timeout occurs. I'm not scared that
ZFS will not do this properly when using ZFSonLinux or SCST.

The problem is, that ZFS caches ALL writes when the application does not
periodically send SYNC requests. I know there are specific SCSI commands
for this purpose:
http://en.wikipedia.org/wiki/SCSI_command (SYNCHRONIZE CACHE) command 35
and 36 in the SCSI protocol.

I know ESXi uses these when necessary. This is what i wanted to with my
original question, does SCST honor these commands, and if not, how can i
make it do it anyway :)

- Thijs

2014-03-03 14:12 GMT+01:00 Sietse van Zanen <si...@wi...>:

> Forgot to mention that you absolutely need to disable any write cache the
> drive itself provides. Independent from what you do with SCST and ZFS. This
> is most often overlooked with software RAID, HW RAID cards will normally
> disable the drives write cache.
>
> hdparm -W 0 /dev/sdx
>
> -Sietse
>
>
> ________________________________________
> From: Sietse van Zanen [si...@wi...]
> Sent: Monday, March 03, 2014 12:12
> To: Thijs Cramer
> Cc: scs...@li...; Vladislav Bolkhovitin
> Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS
>
> If SCST would not honor requests when they arrive, I would be using other
> iSCSI target software, and you probably too. :)
>
>
>
> There is really no sync logic in the SCSI command set and SCST processes
> SCSI commands. So SCST will never be aware of an initator trying to sync
> writes. It queues, processes and notifies initiators about command
> completion.
>
>
>
> A sync write is the iniator waiting for successful command completion,
> which may have been sent by the target after data is  cached and not
> actually written to backend storage.
>
>
>
> So sync writes all come down to the backend storage used. If you want to
> be absolutely sure data has been written to stable storage, use blockio,
> and make sure your block devices have write cache disabled (or BBU).
>
>
>
> When using fileio you will at least have to disable nv_cache, or the Linux
> page cache will be used and writes are certifiable not synced (and cached
> in non BBU).
>
>
>
> When using ZFS as the backend storage you will always have nv_cache set to
> 0, as ZFS has its own caching and also using the page cache will cause
> double caching. ZFS helps you out with this inherent problem by offering
> the sync=always setting. However ZFS is awfully slow when not using ZIL
> with sync=always.
>
>
>
> When using any other filesystem, you will most likely have to set
> write_through and o_direct on the device in SCST.
>
>
>
> -Sietse
>
>
>
>
>
>
>
> ________________________________
> From: Thijs Cramer [thi...@gm...]
> Sent: Sunday, March 02, 2014 14:06
> To: Sietse van Zanen
> Cc: Vladislav Bolkhovitin; scs...@li...
> Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS
>
> sync=always wil indeed garantuee synced writes on ZFS, but that is not
> what i mean.
> What i want, is for SCST to honor sync requests when they arrive.
> If nv_cache 0 does that, then sync=always is not needed anymore.
>
>
> 2014-03-02 13:58 GMT+01:00 Sietse van Zanen <si...@wi...<mailto:
> si...@wi...>>:
> I don't believe so. ZFS has its own caching mechanism, so this setting
> will not have any effect, just like o_direct. The effect is achieved by the
> zfs set sync=always setting for the filesystem or zvol..
>
> -Sietse
>
> -----Original Message-----
> From: Vladislav Bolkhovitin [mailto:vs...@vl...<mailto:vs...@vl...>]
> Sent: Friday, February 28, 2014 22:50
> To: Sietse van Zanen
> Cc: Thijs Cramer; scs...@li...<mailto:
> scs...@li...>
> Subject: Re: [Scst-devel] Honoring SYNC requests in combination with ZFS
>
> Sietse van Zanen, on 02/28/2014 02:57 AM wrote:
> > You have to do 3 things.
> >
> >
> >
> > 1. Set nv_cache to 0 on the scst side
>
> You mean write_through in 1 here, I believe.
>
> Vlad
>
> > 2. zfs set sync=always for the filesystem or zvol you have the file on.
> >
> > 3. Use ZIL or your performance will be terrible, definitely with SATA
> >
> >
> >
> > -Sietse
> >
> >
> >
> > ________________________________
> > From: Thijs Cramer [thi...@gm...<mailto:thi...@gm...
> >]
> > Sent: Tuesday, February 25, 2014 08:59
> > To: scs...@li...<mailto:
> scs...@li...>
> > Subject: [Scst-devel] Honoring SYNC requests in combination with ZFS
> >
> > Guys,
> >
> > Is there a definitive answer on whether we can use the fileio handler in
> combination with ZFSonLinux?
> > I'm seeing extreme performance statistics > 18.000 IOPS over iSCSI when
> using ZFSonLinx on 4*4TB drives with sync=standard. The disks are exported
> over iSCSI through a ZVOL and I've defined a fileio handler in SCST. The
> iSCSI volume is exported to an ESXi cluster.
> >
> > I think this means that SCST doesn't honor the SYNC requests from ESXi.
> >
> > I know the linux page cache is involved when using the fileio handler,
> but i thought sync requests would still be honored? Am i mistaking?
> >
> > - Thijs
> >
> > ----------------------------------------------------------------------
> > -------- Flow-based real-time traffic analytics software. Cisco
> > certified tool.
> > Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
> > Customize your own dashboards, set traffic alerts and generate reports.
> > Network behavioral analysis & security monitoring. All-in-one tool.
> > http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.
> > clktrk _______________________________________________
> > Scst-devel mailing list
> > https://lists.sourceforge.net/lists/listinfo/scst-devel
> >
>
>
>
>
> ------------------------------------------------------------------------------
> Subversion Kills Productivity. Get off Subversion & Make the Move to
> Perforce.
> With Perforce, you get hassle-free workflows. Merge that actually works.
> Faster operations. Version large binaries.  Built-in WAN optimization and
> the
> freedom to use Git, Perforce or both. Make the move to Perforce.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk
> _______________________________________________
> Scst-devel mailing list
> https://lists.sourceforge.net/lists/listinfo/scst-devel
>