From: Bart V. A. <bva...@ac...> - 2012-06-29 11:03:43
|
On 06/29/12 01:46, Vladislav Bolkhovitin wrote: > Bart Van Assche, on 06/26/2012 03:23 AM wrote: >> Although it is possible with the current Linux kernel to perform >> zero-copying, zero-copy I/O doesn't fit well with the SCST command >> engine. What we need is an API that allows to perform asynchronous >> zero-copying. What the Linux kernel offers is an API for synchronous >> zero-copying. Hence the infrastructure to reserve a command thread for >> execution of a command. Without that infrastructure it could e.g. happen >> that zero-copy reading for two different SCSI commands is started on the >> context of the same SCST command thread. That could easily trigger a >> deadlock, e.g. if both commands try to read the same pages from the page >> cache (zero-copy reading now locks the pages that will be read before >> reading starts). Some people are working on patches to make asynchronous >> I/O possible from inside the kernel but unfortunately these patches >> aren't ready yet. > > Do I understand the problem correctly that we need to make sure that we > don't have all the threads in the pool staying in the allocation stage > waiting for unlocking of potentially the same pages? I.e. we need at > least one thread for exec and later stages to ensure forward progress to > unlock pages for the next cmd? Forward progress is guaranteed even if all command threads try to grab page locks simultaneously since page locks are obtained in the order of increasing file offsets. However, if all command threads are busy processing zero-copy I/O and SCST receives a task management function that task management function will only be processed once a command thread becomes available again. If such delays are not acceptable then we should make sure that at least one command thread is not busy processing zero-copy I/O. The patch below realizes that. [PATCH] scst_vdisk, zc: Leave at least one thread unreserved Signed-off-by: Bart Van Assche <bva...@ac...> --- scst/include/scst.h | 2 +- scst/src/dev_handlers/scst_vdisk.c | 7 ++++++- scst/src/scst_main.c | 29 +++++++++++++++++++++-------- 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a/scst/include/scst.h b/scst/include/scst.h index a2d1270..e62192b 100644 --- a/scst/include/scst.h +++ b/scst/include/scst.h @@ -4412,7 +4412,7 @@ void scst_init_threads(struct scst_cmd_threads *cmd_threads); void scst_deinit_threads(struct scst_cmd_threads *cmd_threads); struct scst_cmd_thread_t *scst_lookup_cmd_thread(struct scst_cmd_threads *p, struct task_struct *thread); -void scst_reserve_thread(struct scst_cmd *cmd); +int scst_reserve_thread(struct scst_cmd *cmd); void scst_pass_through_cmd_done(void *data, char *sense, int result, int resid); #if (LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 30)) && defined(SCSI_EXEC_REQ_FIFO_DEFINED) diff --git a/scst/src/dev_handlers/scst_vdisk.c b/scst/src/dev_handlers/scst_vdisk.c index 0065b99..eb88f5c 100644 --- a/scst/src/dev_handlers/scst_vdisk.c +++ b/scst/src/dev_handlers/scst_vdisk.c @@ -1826,8 +1826,13 @@ static int fileio_alloc_data_buf(struct scst_cmd *cmd) cmd->tgt_i_data_buf_alloced) p->use_zero_copy = false; + if (p->use_zero_copy && scst_reserve_thread(cmd) < 0) { + TRACE_DBG("%s: Not using zero copy because too many threads are" + "already processing zero-copy I/O", virt_dev->name); + p->use_zero_copy = false; + } + if (p->use_zero_copy) { - scst_reserve_thread(cmd); cmd->sg = alloc_sg(cmd->bufflen, p->loff & ~PAGE_MASK, gfp_mask, p->small_sg, ARRAY_SIZE(p->small_sg), &cmd->sg_cnt); diff --git a/scst/src/scst_main.c b/scst/src/scst_main.c index 8429a24..cdceaab 100644 --- a/scst/src/scst_main.c +++ b/scst/src/scst_main.c @@ -2000,17 +2000,30 @@ EXPORT_SYMBOL(scst_lookup_cmd_thread); * Returns 0 upon success or -ESRCH if only one thread was remaining that was * not reserved for a specific command. */ -void scst_reserve_thread(struct scst_cmd *cmd) +int scst_reserve_thread(struct scst_cmd *cmd) { - struct scst_cmd_threads *p; - struct scst_cmd_thread_t *t; + struct scst_cmd_threads *p = cmd->cmd_threads; + struct scst_cmd_thread_t *t, *t2 = NULL; + int nonreserved = 0; + int res = -ESRCH; sBUG_ON(in_interrupt()); - t = scst_lookup_cmd_thread(cmd->cmd_threads, current); - sBUG_ON(!t); - p = t->cmd_threads; - cmd->on_thread = t; - t->single_thread_cmd = cmd; + + mutex_lock(&scst_mutex); + list_for_each_entry(t, &p->threads_list, thread_list_entry) { + if (t->cmd_thread == current) + t2 = t; + if (!t->single_thread_cmd) + nonreserved++; + } + if (nonreserved > 1) { + cmd->on_thread = t2; + t2->single_thread_cmd = cmd; + res = 0; + } + mutex_unlock(&scst_mutex); + + return res; } EXPORT_SYMBOL(scst_reserve_thread); -- 1.7.7 |