valgrind-developers — Discussion list for developers of valgrind

Re: [Valgrind-developers] [valgrind] Helgrind: fix unused result of write warnings

[Bart V. A. <bar...@gm...>]

On 11/21/24 11:01 PM, Paul Floyd via Valgrind-developers wrote:
> Anyone else have an opinion on this?

I prefer disabling the unused-result warning rather than adding #pragma
directives. #pragma directives clutter the code. There are other tools
that can catch unused result value issues, e.g. Coverity.

Thanks,

Bart.

Re: [Valgrind-developers] [valgrind] Helgrind: fix unused result of write warnings

[Paul F. <pj...@wa...>]

On 21-11-24 10:48, Florian Krohm wrote:
> On 21.11.24 08:34, Paul Floyd wrote:

> Yeah, this is annoying. That warning is brought to you by -Ox with x > 0
> In my experience (many years doing static analysis and looking at other 
> prople's code) I can say that this warning is a weak symptom for 
> detecting anything serious. How about disabling this globally?

Hi Florian

I think that the warning is useful but mainly for less experienced 
developers and to protect against silly mistakes.

Also we get relatively little benefit from this since we don't have that 
much C-with-libc code. If we really want the extra protection in the 
Valgrind libc replacement functions we should use annotation - either 
[[nodiscard]] (but that requires a very recent C compiler) or '__wur' 
and _FORTIFY_SOURCE like in glibc.

Anyone else have an opinion on this?

A+
Paul

Re: [Valgrind-developers] [valgrind] Helgrind: fix unused result of write warnings

[Florian K. <fl...@ei...>]

On 21.11.24 08:34, Paul Floyd wrote:
> https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=3a80792278be69b9270485b07a79a62a5978c4bf
> 
> commit 3a80792278be69b9270485b07a79a62a5978c4bf
> Author: Paul Floyd <pj...@wa...>
> Date:   Thu Nov 21 08:24:35 2024 +0100
> 
>      Helgrind: fix unused result of write warnings
>      
>      Thanks to GCC deciding that we can't ignore wur annotated
>      functions by casting to void we need to do otherwise. It was
>      either pragmas or adding an annotated unused local. Pragmas
>      seem to be the least awful.

Yeah, this is annoying. That warning is brought to you by -Ox with x > 0
In my experience (many years doing static analysis and looking at other prople's 
code) I can say that this warning is a weak symptom for detecting anything 
serious. How about disabling this globally?

   Florian

Re: [Valgrind-developers] [valgrind] Helgrind: fix unused result of write warnings

[Florian K. <fl...@ei...>]

On 21.11.24 08:34, Paul Floyd wrote:
> https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=3a80792278be69b9270485b07a79a62a5978c4bf
> 
> commit 3a80792278be69b9270485b07a79a62a5978c4bf
> Author: Paul Floyd <pj...@wa...>
> Date:   Thu Nov 21 08:24:35 2024 +0100
> 
>      Helgrind: fix unused result of write warnings
>      
>      Thanks to GCC deciding that we can't ignore wur annotated
>      functions by casting to void we need to do otherwise. It was
>      either pragmas or adding an annotated unused local. Pragmas
>      seem to be the least awful.

Yeah, this is annoying. That warning is brought to you by -Ox with x > 0
In my experience (many years doing static analysis and looking at other prople's 
code) I can say that this warning is a weak symptom for detecting anything 
serious. How about disabling this globally?

   Florian

[Valgrind-developers] [valgrind] regtest: add a fdleak filter for write on write on linux arm64

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=9150b3c7cfad2fdbeb7cf707175c359ee12d8f75

commit 9150b3c7cfad2fdbeb7cf707175c359ee12d8f75
Author: Paul Floyd <pj...@wa...>
Date:   Thu Nov 21 08:44:04 2024 +0100

    regtest: add a fdleak filter for write on write on linux arm64

Diff:
---
 none/tests/filter_fdleak | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/none/tests/filter_fdleak b/none/tests/filter_fdleak
index 7de2f6eb18..1f4a8c7bd6 100755
--- a/none/tests/filter_fdleak
+++ b/none/tests/filter_fdleak
@@ -19,6 +19,8 @@ perl -p -e 's/socket\.c:[1-9][0-9]*/in \/...libc.../' |
 # arm systems substitute open for creat
 perl -p -e 's/open \(open64\.c:[1-9][0-9]*\)/creat (in \/...libc...)/' |
 perl -p -e "s/: open \(/: creat (/" |
+# arm64 write resolved to file:line with debuginfo
+perl -p -e "s/write\.c:[1-9][0-9]*/in \/...libc.../" |
 
 # FreeBSD specific fdleak filters
 perl -p -e 's/ _close / close /;s/ _openat / creat /;s/ _write/ write/;s/internet/AF_INET socket 4: 127.0.0.1:... <-> 127.0.0.1:.../' |

[Valgrind-developers] [valgrind] Helgrind: fix unused result of write warnings

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=3a80792278be69b9270485b07a79a62a5978c4bf

commit 3a80792278be69b9270485b07a79a62a5978c4bf
Author: Paul Floyd <pj...@wa...>
Date:   Thu Nov 21 08:24:35 2024 +0100

    Helgrind: fix unused result of write warnings
    
    Thanks to GCC deciding that we can't ignore wur annotated
    functions by casting to void we need to do otherwise. It was
    either pragmas or adding an annotated unused local. Pragmas
    seem to be the least awful.

Diff:
---
 helgrind/hg_intercepts.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/helgrind/hg_intercepts.c b/helgrind/hg_intercepts.c
index 43afd89f47..ac2449e609 100644
--- a/helgrind/hg_intercepts.c
+++ b/helgrind/hg_intercepts.c
@@ -978,7 +978,10 @@ static int mutex_lock_WRK(pthread_mutex_t *mutex)
    if (TRACE_PTH_FNS) {
       char buf[30];
       snprintf(buf, 30, "<< pthread_mxlock %p", mutex);
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wunused-result"
       write(STDERR_FILENO, buf, strlen(buf));
+#pragma GCC diagnostic pop
       fsync(STDERR_FILENO);
    }
 
@@ -1014,7 +1017,10 @@ HG_MUTEX_LOCK_OUT:
    if (TRACE_PTH_FNS) {
       char buf[30];
       snprintf(buf, 30, " :: mxlock -> %d >>\n", ret);
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wunused-result"1251
       write(STDERR_FILENO, buf, strlen(buf));
+#pragma GCC diagnostic pop
    }
    return ret;
 }
@@ -1242,7 +1248,10 @@ static int mutex_unlock_WRK(pthread_mutex_t *mutex)
    if (TRACE_PTH_FNS) {
       char buf[30];
       snprintf(buf, 30, "<< pthread_mxunlk %p", mutex);
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wunused-result"
       write(STDERR_FILENO, buf, strlen(buf));
+#pragma GCC diagnostic pop
       fsync(STDERR_FILENO);
    }
 
@@ -1261,7 +1270,10 @@ static int mutex_unlock_WRK(pthread_mutex_t *mutex)
    if (TRACE_PTH_FNS) {
       char buf[30];
       snprintf(buf, 30, " :: mxunlk -> %d >>\n", ret);
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wunused-result"
       write(STDERR_FILENO, buf, strlen(buf));
+#pragma GCC diagnostic pop
    }
    return ret;
 }

[Valgrind-developers] [valgrind] Bug 469782 - Valgrind does not support zstd-compressed debug sections

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=309d96d3e4f17930da28d76f6163957849553140

commit 309d96d3e4f17930da28d76f6163957849553140
Author: Mikhail Gorodetsky <mih...@gm...>
Date:   Sun Nov 17 21:14:17 2024 +0100

    Bug 469782 - Valgrind does not support zstd-compressed debug sections

Diff:
---
 .gitignore                            |     1 +
 README_DEVELOPERS                     |    25 +
 configure.ac                          |    20 +
 coregrind/Makefile.am                 |     1 +
 coregrind/m_debuginfo/image.c         |    53 +-
 coregrind/m_debuginfo/priv_image.h    |     2 +-
 coregrind/m_debuginfo/readelf.c       |    12 +-
 coregrind/m_debuginfo/zstd.h          |  3089 +++++
 coregrind/m_debuginfo/zstddeclib.c    | 22132 ++++++++++++++++++++++++++++++++
 docs/internals/release-HOWTO.txt      |     3 +
 memcheck/tests/Makefile.am            |     7 +
 memcheck/tests/cdebug_zstd.stderr.exp |     3 +
 memcheck/tests/cdebug_zstd.vgtest     |     5 +
 13 files changed, 25334 insertions(+), 19 deletions(-)

diff --git a/.gitignore b/.gitignore
index cfd943bfc6..46c9126716 100644
--- a/.gitignore
+++ b/.gitignore
@@ -876,6 +876,7 @@
 /memcheck/tests/calloc-overflow
 /memcheck/tests/cdebug_zlib
 /memcheck/tests/cdebug_zlib_gnu
+/memcheck/tests/cdebug_zstd
 /memcheck/tests/client-msg
 /memcheck/tests/clientperm
 /memcheck/tests/clientstackperm
diff --git a/README_DEVELOPERS b/README_DEVELOPERS
index 37cffa2fb7..0be02b7dae 100644
--- a/README_DEVELOPERS
+++ b/README_DEVELOPERS
@@ -481,3 +481,28 @@ integration with a text editor, it is possible to reformat arbitrary blocks
 of code with a single keystroke.  Refer to the upstream documentation which
 describes integration with various editors and IDEs:
 https://clang.llvm.org/docs/ClangFormat.html.
+
+Updating zstd
+~~~~~~~~~~~~~
+Similar to libiberty, we have to import a copy of zstd rather than linking
+with a library. There isn't (yet) a script to automate this, so it has to be
+done manually.
+
+The version currently in use can be seen in coregrind/m_debuginfo/zstd.h.
+Look for ZSTD_VERSION_MAJOR ZSTD_VERSION_MINOR and ZSTD_VERSION_RELEASE.
+
+ - Get the source of zstd from
+
+   https://github.com/facebook/zstd
+
+ - Checkout the latest release tag (should be vMAJ.MIN.REL)
+
+ - Copy {zstd git repo}/lib/zstd.h to coregrind/m_debuginfo/zstd.h
+ 
+ -  cd to {zstd git repo}/build/single_file_libs and run ./create_single_file_decoder.sh
+ 
+ - You cannot simply copy and use the generated zstddeclib.c!
+   All calls to libc functions in this file need replacing with VG_ versions.
+   Merge the newly generated zstddeclib.c with coregrind/m_debuginfo/zstddeclib.c.
+   Make sure to keep the copy of the BSD license in the C file.
+   
diff --git a/configure.ac b/configure.ac
index ee18cc83a1..7c417eb9d7 100755
--- a/configure.ac
+++ b/configure.ac
@@ -2880,6 +2880,26 @@ AM_CONDITIONAL(GZ_ZLIB_GNU, test x$ac_have_gz_zlib_gnu = xyes)
 CFLAGS=$safe_CFLAGS
 
 
+# does this compiler support -g -gz=zstd ?
+
+AC_MSG_CHECKING([if gcc accepts -g -gz=zstd])
+
+safe_CFLAGS=$CFLAGS
+CFLAGS="-g -gz=zstd"
+
+AC_LINK_IFELSE([AC_LANG_PROGRAM([[ ]], [[
+  return 0;
+]])], [
+ac_have_gz_zstd=yes
+AC_MSG_RESULT([yes])
+], [
+ac_have_gz_zstd=no
+AC_MSG_RESULT([no])
+])
+AM_CONDITIONAL(GZ_ZSTD, test x$ac_have_gz_zstd = xyes)
+CFLAGS=$safe_CFLAGS
+
+
 # does this compiler support nested functions ?
 
 AC_MSG_CHECKING([if gcc accepts nested functions])
diff --git a/coregrind/Makefile.am b/coregrind/Makefile.am
index e3e31a73bd..4eeb6e3d29 100644
--- a/coregrind/Makefile.am
+++ b/coregrind/Makefile.am
@@ -372,6 +372,7 @@ COREGRIND_SOURCES_COMMON = \
 	m_debuginfo/storage.c \
 	m_debuginfo/tinfl.c \
 	m_debuginfo/tytypes.c \
+	m_debuginfo/zstddeclib.c \
 	m_demangle/cp-demangle.c \
 	m_demangle/cplus-dem.c \
 	m_demangle/demangle.c \
diff --git a/coregrind/m_debuginfo/image.c b/coregrind/m_debuginfo/image.c
index 4ef01c5251..d00535cda2 100644
--- a/coregrind/m_debuginfo/image.c
+++ b/coregrind/m_debuginfo/image.c
@@ -45,6 +45,7 @@
 #include "minilzo.h"
 #define TINFL_HEADER_FILE_ONLY
 #include "tinfl.c"
+#include "zstd.h"
 
 /* These values (1024 entries of 8192 bytes each) gives a cache
    size of 8MB. */
@@ -55,6 +56,9 @@
 
 #define COMPRESSED_SLICE_ARRAY_GROW_SIZE 64
 
+#define ELFCOMPRESS_ZLIB 1
+#define ELFCOMPRESS_ZSTD 2
+
 /* An entry in the cache. */
 typedef
    struct {
@@ -69,10 +73,11 @@ typedef
 /* Compressed slice */
 typedef
    struct {
-      DiOffT offD;  // offset of decompressed data
-      SizeT  szD;   // size of decompressed data
-      DiOffT offC;  // offset of compressed data
-      SizeT  szC;   // size of compressed data
+      DiOffT offD;   // offset of decompressed data
+      SizeT  szD;    // size of decompressed data
+      DiOffT offC;   // offset of compressed data
+      SizeT  szC;    // size of compressed data
+      UChar  typeC;  // type of compressed data
    }
    CSlc;
 
@@ -756,11 +761,20 @@ static UChar get_slowcase ( DiImage* img, DiOffT off )
          vg_assert(is_sane_CEnt("get_slowcase-case-1", img, i));
          vg_assert(img->ces_used == ces_used_at_entry + 1);
       } else {
-         SizeT len = tinfl_decompress_mem_to_mem(
-                        img->ces[i]->data, cslc->szD,
-                        cbuf, cslc->szC,
-                        TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF
-                        | TINFL_FLAG_PARSE_ZLIB_HEADER);
+         SizeT len;
+         if(cslc->typeC == ELFCOMPRESS_ZLIB) {
+            len = tinfl_decompress_mem_to_mem(
+               img->ces[i]->data, cslc->szD,
+               cbuf, cslc->szC,
+               TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF
+               | TINFL_FLAG_PARSE_ZLIB_HEADER);
+         }
+         else {
+            len = ZSTD_decompress(
+               img->ces[i]->data, cslc->szD,
+               cbuf, cslc->szC);
+         }
+
          vg_assert(len == cslc->szD); // sanity check on data, FIXME
          vg_assert(cslc->szD == size);
          img->ces[i]->used = cslc->szD;
@@ -806,11 +820,19 @@ static UChar get_slowcase ( DiImage* img, DiOffT off )
       img->ces[i]->fromC = False;
       vg_assert(is_sane_CEnt("get_slowcase-case-2", img, i));
    } else {
-      SizeT len = tinfl_decompress_mem_to_mem(
-                     img->ces[i]->data, cslc->szD,
-                     cbuf, cslc->szC,
-                     TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF
-                     | TINFL_FLAG_PARSE_ZLIB_HEADER);
+      SizeT len;
+      if(cslc->typeC == ELFCOMPRESS_ZLIB) {
+         len = tinfl_decompress_mem_to_mem(
+            img->ces[i]->data, cslc->szD,
+            cbuf, cslc->szC,
+            TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF
+            | TINFL_FLAG_PARSE_ZLIB_HEADER);
+      }
+      else {
+         len = ZSTD_decompress(
+            img->ces[i]->data, cslc->szD,
+            cbuf, cslc->szC);
+      }
       vg_assert(len == size);
       img->ces[i]->used = size;
       img->ces[i]->off = cslc->offD;
@@ -1048,7 +1070,7 @@ DiImage* ML_(img_from_di_server)(const HChar* filename,
 }
 
 DiOffT ML_(img_mark_compressed_part)(DiImage* img, DiOffT offset, SizeT szC,
-                                     SizeT szD)
+                                     SizeT szD, UChar typeC)
 {
    DiOffT ret;
    vg_assert(img != NULL);
@@ -1065,6 +1087,7 @@ DiOffT ML_(img_mark_compressed_part)(DiImage* img, DiOffT offset, SizeT szC,
    img->cslc[img->cslc_used].szC = szC;
    img->cslc[img->cslc_used].offD = img->size;
    img->cslc[img->cslc_used].szD = szD;
+   img->cslc[img->cslc_used].typeC = typeC;
    img->size += szD;
    img->cslc_used++;
    return ret;
diff --git a/coregrind/m_debuginfo/priv_image.h b/coregrind/m_debuginfo/priv_image.h
index c91e49f015..1632fa71f0 100644
--- a/coregrind/m_debuginfo/priv_image.h
+++ b/coregrind/m_debuginfo/priv_image.h
@@ -142,7 +142,7 @@ UInt ML_(img_calc_gnu_debuglink_crc32)(DiImage* img);
    Returns (virtual) position in image from which decompressed data can be
    read. */
 DiOffT ML_(img_mark_compressed_part)(DiImage* img, DiOffT offset, SizeT szC,
-                                     SizeT szD);
+                                     SizeT szD, UChar typeC);
 
 
 /*------------------------------------------------------------*/
diff --git a/coregrind/m_debuginfo/readelf.c b/coregrind/m_debuginfo/readelf.c
index da548048df..77ed4798b0 100644
--- a/coregrind/m_debuginfo/readelf.c
+++ b/coregrind/m_debuginfo/readelf.c
@@ -90,6 +90,10 @@
    #define ELFCOMPRESS_ZLIB 1
 #endif
 
+#if !defined(ELFCOMPRESS_ZSTD)
+   #define ELFCOMPRESS_ZSTD 2
+#endif
+
 #define SIZE_OF_ZLIB_HEADER 12
 
 /*------------------------------------------------------------*/
@@ -1730,12 +1734,13 @@ static Bool check_compression(ElfXX_Shdr* h, DiSlice* s) {
    if (h->sh_flags & SHF_COMPRESSED) {
       ElfXX_Chdr chdr;
       ML_(img_get)(&chdr, s->img, s->ioff, sizeof(ElfXX_Chdr));
-      if (chdr.ch_type != ELFCOMPRESS_ZLIB)
+      if (chdr.ch_type != ELFCOMPRESS_ZLIB && chdr.ch_type != ELFCOMPRESS_ZSTD )
          return False;
       s->ioff = ML_(img_mark_compressed_part)(s->img,
                                               s->ioff + sizeof(ElfXX_Chdr),
                                               s->szB - sizeof(ElfXX_Chdr),
-                                              (SizeT)chdr.ch_size);
+                                              (SizeT)chdr.ch_size,
+                                              (UChar)chdr.ch_type);
       s->szB = chdr.ch_size;
     } else if (h->sh_size > SIZE_OF_ZLIB_HEADER) {
        /* Read the zlib header.  In this case, it should be "ZLIB"
@@ -1761,7 +1766,8 @@ static Bool check_compression(ElfXX_Shdr* h, DiSlice* s) {
           s->ioff = ML_(img_mark_compressed_part)(s->img,
                                                   s->ioff + SIZE_OF_ZLIB_HEADER,
                                                   s->szB - SIZE_OF_ZLIB_HEADER,
-                                                  size);
+                                                  size,
+                                                  ELFCOMPRESS_ZLIB);
           s->szB = size;
        }
     }
diff --git a/coregrind/m_debuginfo/zstd.h b/coregrind/m_debuginfo/zstd.h
new file mode 100644
index 0000000000..5d1fef8a6b
--- /dev/null
+++ b/coregrind/m_debuginfo/zstd.h
@@ -0,0 +1,3089 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+#if defined (__cplusplus)
+extern "C" {
+#endif
+
+#ifndef ZSTD_H_235446
+#define ZSTD_H_235446
+
+/* ======   Dependencies   ======*/
+#include <limits.h>   /* INT_MAX */
+#include <stddef.h>   /* size_t */
+
+
+/* =====   ZSTDLIB_API : control library symbols visibility   ===== */
+#ifndef ZSTDLIB_VISIBLE
+   /* Backwards compatibility with old macro name */
+#  ifdef ZSTDLIB_VISIBILITY
+#    define ZSTDLIB_VISIBLE ZSTDLIB_VISIBILITY
+#  elif defined(__GNUC__) && (__GNUC__ >= 4) && !defined(__MINGW32__)
+#    define ZSTDLIB_VISIBLE __attribute__ ((visibility ("default")))
+#  else
+#    define ZSTDLIB_VISIBLE
+#  endif
+#endif
+
+#ifndef ZSTDLIB_HIDDEN
+#  if defined(__GNUC__) && (__GNUC__ >= 4) && !defined(__MINGW32__)
+#    define ZSTDLIB_HIDDEN __attribute__ ((visibility ("hidden")))
+#  else
+#    define ZSTDLIB_HIDDEN
+#  endif
+#endif
+
+#if defined(ZSTD_DLL_EXPORT) && (ZSTD_DLL_EXPORT==1)
+#  define ZSTDLIB_API __declspec(dllexport) ZSTDLIB_VISIBLE
+#elif defined(ZSTD_DLL_IMPORT) && (ZSTD_DLL_IMPORT==1)
+#  define ZSTDLIB_API __declspec(dllimport) ZSTDLIB_VISIBLE /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/
+#else
+#  define ZSTDLIB_API ZSTDLIB_VISIBLE
+#endif
+
+/* Deprecation warnings :
+ * Should these warnings be a problem, it is generally possible to disable them,
+ * typically with -Wno-deprecated-declarations for gcc or _CRT_SECURE_NO_WARNINGS in Visual.
+ * Otherwise, it's also possible to define ZSTD_DISABLE_DEPRECATE_WARNINGS.
+ */
+#ifdef ZSTD_DISABLE_DEPRECATE_WARNINGS
+#  define ZSTD_DEPRECATED(message) /* disable deprecation warnings */
+#else
+#  if defined (__cplusplus) && (__cplusplus >= 201402) /* C++14 or greater */
+#    define ZSTD_DEPRECATED(message) [[deprecated(message)]]
+#  elif (defined(GNUC) && (GNUC > 4 || (GNUC == 4 && GNUC_MINOR >= 5))) || defined(__clang__)
+#    define ZSTD_DEPRECATED(message) __attribute__((deprecated(message)))
+#  elif defined(__GNUC__) && (__GNUC__ >= 3)
+#    define ZSTD_DEPRECATED(message) __attribute__((deprecated))
+#  elif defined(_MSC_VER)
+#    define ZSTD_DEPRECATED(message) __declspec(deprecated(message))
+#  else
+#    pragma message("WARNING: You need to implement ZSTD_DEPRECATED for this compiler")
+#    define ZSTD_DEPRECATED(message)
+#  endif
+#endif /* ZSTD_DISABLE_DEPRECATE_WARNINGS */
+
+
+/*******************************************************************************
+  Introduction
+
+  zstd, short for Zstandard, is a fast lossless compression algorithm, targeting
+  real-time compression scenarios at zlib-level and better compression ratios.
+  The zstd compression library provides in-memory compression and decompression
+  functions.
+
+  The library supports regular compression levels from 1 up to ZSTD_maxCLevel(),
+  which is currently 22. Levels >= 20, labeled `--ultra`, should be used with
+  caution, as they require more memory. The library also offers negative
+  compression levels, which extend the range of speed vs. ratio preferences.
+  The lower the level, the faster the speed (at the cost of compression).
+
+  Compression can be done in:
+    - a single step (described as Simple API)
+    - a single step, reusing a context (described as Explicit context)
+    - unbounded multiple steps (described as Streaming compression)
+
+  The compression ratio achievable on small data can be highly improved using
+  a dictionary. Dictionary compression can be performed in:
+    - a single step (described as Simple dictionary API)
+    - a single step, reusing a dictionary (described as Bulk-processing
+      dictionary API)
+
+  Advanced experimental functions can be accessed using
+  `#define ZSTD_STATIC_LINKING_ONLY` before including zstd.h.
+
+  Advanced experimental APIs should never be used with a dynamically-linked
+  library. They are not "stable"; their definitions or signatures may change in
+  the future. Only static linking is allowed.
+*******************************************************************************/
+
+/*------   Version   ------*/
+#define ZSTD_VERSION_MAJOR    1
+#define ZSTD_VERSION_MINOR    5
+#define ZSTD_VERSION_RELEASE  6
+#define ZSTD_VERSION_NUMBER  (ZSTD_VERSION_MAJOR *100*100 + ZSTD_VERSION_MINOR *100 + ZSTD_VERSION_RELEASE)
+
+/*! ZSTD_versionNumber() :
+ *  Return runtime library version, the value is (MAJOR*100*100 + MINOR*100 + RELEASE). */
+ZSTDLIB_API unsigned ZSTD_versionNumber(void);
+
+#define ZSTD_LIB_VERSION ZSTD_VERSION_MAJOR.ZSTD_VERSION_MINOR.ZSTD_VERSION_RELEASE
+#define ZSTD_QUOTE(str) #str
+#define ZSTD_EXPAND_AND_QUOTE(str) ZSTD_QUOTE(str)
+#define ZSTD_VERSION_STRING ZSTD_EXPAND_AND_QUOTE(ZSTD_LIB_VERSION)
+
+/*! ZSTD_versionString() :
+ *  Return runtime library version, like "1.4.5". Requires v1.3.0+. */
+ZSTDLIB_API const char* ZSTD_versionString(void);
+
+/* *************************************
+ *  Default constant
+ ***************************************/
+#ifndef ZSTD_CLEVEL_DEFAULT
+#  define ZSTD_CLEVEL_DEFAULT 3
+#endif
+
+/* *************************************
+ *  Constants
+ ***************************************/
+
+/* All magic numbers are supposed read/written to/from files/memory using little-endian convention */
+#define ZSTD_MAGICNUMBER            0xFD2FB528    /* valid since v0.8.0 */
+#define ZSTD_MAGIC_DICTIONARY       0xEC30A437    /* valid since v0.7.0 */
+#define ZSTD_MAGIC_SKIPPABLE_START  0x184D2A50    /* all 16 values, from 0x184D2A50 to 0x184D2A5F, signal the beginning of a skippable frame */
+#define ZSTD_MAGIC_SKIPPABLE_MASK   0xFFFFFFF0
+
+#define ZSTD_BLOCKSIZELOG_MAX  17
+#define ZSTD_BLOCKSIZE_MAX     (1<<ZSTD_BLOCKSIZELOG_MAX)
+
+
+/***************************************
+*  Simple API
+***************************************/
+/*! ZSTD_compress() :
+ *  Compresses `src` content as a single zstd compressed frame into already allocated `dst`.
+ *  NOTE: Providing `dstCapacity >= ZSTD_compressBound(srcSize)` guarantees that zstd will have
+ *        enough space to successfully compress the data.
+ *  @return : compressed size written into `dst` (<= `dstCapacity),
+ *            or an error code if it fails (which can be tested using ZSTD_isError()). */
+ZSTDLIB_API size_t ZSTD_compress( void* dst, size_t dstCapacity,
+                            const void* src, size_t srcSize,
+                                  int compressionLevel);
+
+/*! ZSTD_decompress() :
+ *  `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames.
+ *  `dstCapacity` is an upper bound of originalSize to regenerate.
+ *  If user cannot imply a maximum upper bound, it's better to use streaming mode to decompress data.
+ *  @return : the number of bytes decompressed into `dst` (<= `dstCapacity`),
+ *            or an errorCode if it fails (which can be tested using ZSTD_isError()). */
+ZSTDLIB_API size_t ZSTD_decompress( void* dst, size_t dstCapacity,
+                              const void* src, size_t compressedSize);
+
+/*! ZSTD_getFrameContentSize() : requires v1.3.0+
+ *  `src` should point to the start of a ZSTD encoded frame.
+ *  `srcSize` must be at least as large as the frame header.
+ *            hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
+ *  @return : - decompressed size of `src` frame content, if known
+ *            - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
+ *            - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
+ *   note 1 : a 0 return value means the frame is valid but "empty".
+ *   note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode.
+ *            When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
+ *            In which case, it's necessary to use streaming mode to decompress data.
+ *            Optionally, application can rely on some implicit limit,
+ *            as ZSTD_decompress() only needs an upper bound of decompressed size.
+ *            (For example, data could be necessarily cut into blocks <= 16 KB).
+ *   note 3 : decompressed size is always present when compression is completed using single-pass functions,
+ *            such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict().
+ *   note 4 : decompressed size can be very large (64-bits value),
+ *            potentially larger than what local system can handle as a single memory segment.
+ *            In which case, it's necessary to use streaming mode to decompress data.
+ *   note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
+ *            Always ensure return value fits within application's authorized limits.
+ *            Each application can set its own limits.
+ *   note 6 : This function replaces ZSTD_getDecompressedSize() */
+#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
+#define ZSTD_CONTENTSIZE_ERROR   (0ULL - 2)
+ZSTDLIB_API unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
+
+/*! ZSTD_getDecompressedSize() :
+ *  NOTE: This function is now obsolete, in favor of ZSTD_getFrameContentSize().
+ *  Both functions work the same way, but ZSTD_getDecompressedSize() blends
+ *  "empty", "unknown" and "error" results to the same return value (0),
+ *  while ZSTD_getFrameContentSize() gives them separate return values.
+ * @return : decompressed size of `src` frame content _if known and not empty_, 0 otherwise. */
+ZSTD_DEPRECATED("Replaced by ZSTD_getFrameContentSize")
+ZSTDLIB_API
+unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);
+
+/*! ZSTD_findFrameCompressedSize() : Requires v1.4.0+
+ * `src` should point to the start of a ZSTD frame or skippable frame.
+ * `srcSize` must be >= first frame size
+ * @return : the compressed size of the first frame starting at `src`,
+ *           suitable to pass as `srcSize` to `ZSTD_decompress` or similar,
+ *        or an error code if input is invalid */
+ZSTDLIB_API size_t ZSTD_findFrameCompressedSize(const void* src, size_t srcSize);
+
+
+/*======  Helper functions  ======*/
+/* ZSTD_compressBound() :
+ * maximum compressed size in worst case single-pass scenario.
+ * When invoking `ZSTD_compress()` or any other one-pass compression function,
+ * it's recommended to provide @dstCapacity >= ZSTD_compressBound(srcSize)
+ * as it eliminates one potential failure scenario,
+ * aka not enough room in dst buffer to write the compressed frame.
+ * Note : ZSTD_compressBound() itself can fail, if @srcSize > ZSTD_MAX_INPUT_SIZE .
+ *        In which case, ZSTD_compressBound() will return an error code
+ *        which can be tested using ZSTD_isError().
+ *
+ * ZSTD_COMPRESSBOUND() :
+ * same as ZSTD_compressBound(), but as a macro.
+ * It can be used to produce constants, which can be useful for static allocation,
+ * for example to size a static array on stack.
+ * Will produce constant value 0 if srcSize too large.
+ */
+#define ZSTD_MAX_INPUT_SIZE ((sizeof(size_t)==8) ? 0xFF00FF00FF00FF00ULL : 0xFF00FF00U)
+#define ZSTD_COMPRESSBOUND(srcSize)   (((size_t)(srcSize) >= ZSTD_MAX_INPUT_SIZE) ? 0 : (srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) /* margin, from 64 to 0 */ : 0))  /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */
+ZSTDLIB_API size_t ZSTD_compressBound(size_t srcSize); /*!< maximum compressed size in worst case single-pass scenario */
+/* ZSTD_isError() :
+ * Most ZSTD_* functions returning a size_t value can be tested for error,
+ * using ZSTD_isError().
+ * @return 1 if error, 0 otherwise
+ */
+ZSTDLIB_API unsigned    ZSTD_isError(size_t code);          /*!< tells if a `size_t` function result is an error code */
+ZSTDLIB_API const char* ZSTD_getErrorName(size_t code);     /*!< provides readable string from an error code */
+ZSTDLIB_API int         ZSTD_minCLevel(void);               /*!< minimum negative compression level allowed, requires v1.4.0+ */
+ZSTDLIB_API int         ZSTD_maxCLevel(void);               /*!< maximum compression level available */
+ZSTDLIB_API int         ZSTD_defaultCLevel(void);           /*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */
+
+
+/***************************************
+*  Explicit context
+***************************************/
+/*= Compression context
+ *  When compressing many times,
+ *  it is recommended to allocate a context just once,
+ *  and reuse it for each successive compression operation.
+ *  This will make workload friendlier for system's memory.
+ *  Note : re-using context is just a speed / resource optimization.
+ *         It doesn't change the compression ratio, which remains identical.
+ *  Note 2 : In multi-threaded environments,
+ *         use one different context per thread for parallel execution.
+ */
+typedef struct ZSTD_CCtx_s ZSTD_CCtx;
+ZSTDLIB_API ZSTD_CCtx* ZSTD_createCCtx(void);
+ZSTDLIB_API size_t     ZSTD_freeCCtx(ZSTD_CCtx* cctx);  /* accept NULL pointer */
+
+/*! ZSTD_compressCCtx() :
+ *  Same as ZSTD_compress(), using an explicit ZSTD_CCtx.
+ *  Important : in order to mirror `ZSTD_compress()` behavior,
+ *  this function compresses at the requested compression level,
+ *  __ignoring any other advanced parameter__ .
+ *  If any advanced parameter was set using the advanced API,
+ *  they will all be reset. Only `compressionLevel` remains.
+ */
+ZSTDLIB_API size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx,
+                                     void* dst, size_t dstCapacity,
+                               const void* src, size_t srcSize,
+                                     int compressionLevel);
+
+/*= Decompression context
+ *  When decompressing many times,
+ *  it is recommended to allocate a context only once,
+ *  and reuse it for each successive compression operation.
+ *  This will make workload friendlier for system's memory.
+ *  Use one context per thread for parallel execution. */
+typedef struct ZSTD_DCtx_s ZSTD_DCtx;
+ZSTDLIB_API ZSTD_DCtx* ZSTD_createDCtx(void);
+ZSTDLIB_API size_t     ZSTD_freeDCtx(ZSTD_DCtx* dctx);  /* accept NULL pointer */
+
+/*! ZSTD_decompressDCtx() :
+ *  Same as ZSTD_decompress(),
+ *  requires an allocated ZSTD_DCtx.
+ *  Compatible with sticky parameters (see below).
+ */
+ZSTDLIB_API size_t ZSTD_decompressDCtx(ZSTD_DCtx* dctx,
+                                       void* dst, size_t dstCapacity,
+                                 const void* src, size_t srcSize);
+
+
+/*********************************************
+*  Advanced compression API (Requires v1.4.0+)
+**********************************************/
+
+/* API design :
+ *   Parameters are pushed one by one into an existing context,
+ *   using ZSTD_CCtx_set*() functions.
+ *   Pushed parameters are sticky : they are valid for next compressed frame, and any subsequent frame.
+ *   "sticky" parameters are applicable to `ZSTD_compress2()` and `ZSTD_compressStream*()` !
+ *   __They do not apply to one-shot variants such as ZSTD_compressCCtx()__ .
+ *
+ *   It's possible to reset all parameters to "default" using ZSTD_CCtx_reset().
+ *
+ *   This API supersedes all other "advanced" API entry points in the experimental section.
+ *   In the future, we expect to remove API entry points from experimental which are redundant with this API.
+ */
+
+
+/* Compression strategies, listed from fastest to strongest */
+typedef enum { ZSTD_fast=1,
+               ZSTD_dfast=2,
+               ZSTD_greedy=3,
+               ZSTD_lazy=4,
+               ZSTD_lazy2=5,
+               ZSTD_btlazy2=6,
+               ZSTD_btopt=7,
+               ZSTD_btultra=8,
+               ZSTD_btultra2=9
+               /* note : new strategies _might_ be added in the future.
+                         Only the order (from fast to strong) is guaranteed */
+} ZSTD_strategy;
+
+typedef enum {
+
+    /* compression parameters
+     * Note: When compressing with a ZSTD_CDict these parameters are superseded
+     * by the parameters used to construct the ZSTD_CDict.
+     * See ZSTD_CCtx_refCDict() for more info (superseded-by-cdict). */
+    ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table.
+                              * Note that exact compression parameters are dynamically determined,
+                              * depending on both compression level and srcSize (when known).
+                              * Default level is ZSTD_CLEVEL_DEFAULT==3.
+                              * Special: value 0 means default, which is controlled by ZSTD_CLEVEL_DEFAULT.
+                              * Note 1 : it's possible to pass a negative compression level.
+                              * Note 2 : setting a level does not automatically set all other compression parameters
+                              *   to default. Setting this will however eventually dynamically impact the compression
+                              *   parameters which have not been manually set. The manually set
+                              *   ones will 'stick'. */
+    /* Advanced compression parameters :
+     * It's possible to pin down compression parameters to some specific values.
+     * In which case, these values are no longer dynamically selected by the compressor */
+    ZSTD_c_windowLog=101,    /* Maximum allowed back-reference distance, expressed as power of 2.
+                              * This will set a memory budget for streaming decompression,
+                              * with larger values requiring more memory
+                              * and typically compressing more.
+                              * Must be clamped between ZSTD_WINDOWLOG_MIN and ZSTD_WINDOWLOG_MAX.
+                              * Special: value 0 means "use default windowLog".
+                              * Note: Using a windowLog greater than ZSTD_WINDOWLOG_LIMIT_DEFAULT
+                              *       requires explicitly allowing such size at streaming decompression stage. */
+    ZSTD_c_hashLog=102,      /* Size of the initial probe table, as a power of 2.
+                              * Resulting memory usage is (1 << (hashLog+2)).
+                              * Must be clamped between ZSTD_HASHLOG_MIN and ZSTD_HASHLOG_MAX.
+                              * Larger tables improve compression ratio of strategies <= dFast,
+                              * and improve speed of strategies > dFast.
+                              * Special: value 0 means "use default hashLog". */
+    ZSTD_c_chainLog=103,     /* Size of the multi-probe search table, as a power of 2.
+                              * Resulting memory usage is (1 << (chainLog+2)).
+                              * Must be clamped between ZSTD_CHAINLOG_MIN and ZSTD_CHAINLOG_MAX.
+                              * Larger tables result in better and slower compression.
+                              * This parameter is useless for "fast" strategy.
+                              * It's still useful when using "dfast" strategy,
+                              * in which case it defines a secondary probe table.
+                              * Special: value 0 means "use default chainLog". */
+    ZSTD_c_searchLog=104,    /* Number of search attempts, as a power of 2.
+                              * More attempts result in better and slower compression.
+                              * This parameter is useless for "fast" and "dFast" strategies.
+                              * Special: value 0 means "use default searchLog". */
+    ZSTD_c_minMatch=105,     /* Minimum size of searched matches.
+                              * Note that Zstandard can still find matches of smaller size,
+                              * it just tweaks its search algorithm to look for this size and larger.
+                              * Larger values increase compression and decompression speed, but decrease ratio.
+                              * Must be clamped between ZSTD_MINMATCH_MIN and ZSTD_MINMATCH_MAX.
+                              * Note that currently, for all strategies < btopt, effective minimum is 4.
+                              *                    , for all strategies > fast, effective maximum is 6.
+                              * Special: value 0 means "use default minMatchLength". */
+    ZSTD_c_targetLength=106, /* Impact of this field depends on strategy.
+                              * For strategies btopt, btultra & btultra2:
+                              *     Length of Match considered "good enough" to stop search.
+                              *     Larger values make compression stronger, and slower.
+                              * For strategy fast:
+                              *     Distance between match sampling.
+                              *     Larger values make compression faster, and weaker.
+                              * Special: value 0 means "use default targetLength". */
+    ZSTD_c_strategy=107,     /* See ZSTD_strategy enum definition.
+                              * The higher the value of selected strategy, the more complex it is,
+                              * resulting in stronger and slower compression.
+                              * Special: value 0 means "use default strategy". */
+
+    ZSTD_c_targetCBlockSize=130, /* v1.5.6+
+                                  * Attempts to fit compressed block size into approximatively targetCBlockSize.
+                                  * Bound by ZSTD_TARGETCBLOCKSIZE_MIN and ZSTD_TARGETCBLOCKSIZE_MAX.
+                                  * Note that it's not a guarantee, just a convergence target (default:0).
+                                  * No target when targetCBlockSize == 0.
+                                  * This is helpful in low bandwidth streaming environments to improve end-to-end latency,
+                                  * when a client can make use of partial documents (a prominent example being Chrome).
+                                  * Note: this parameter is stable since v1.5.6.
+                                  * It was present as an experimental parameter in earlier versions,
+                                  * but it's not recommended using it with earlier library versions
+                                  * due to massive performance regressions.
+                                  */
+    /* LDM mode parameters */
+    ZSTD_c_enableLongDistanceMatching=160, /* Enable long distance matching.
+                                     * This parameter is designed to improve compression ratio
+                                     * for large inputs, by finding large matches at long distance.
+                                     * It increases memory usage and window size.
+                                     * Note: enabling this parameter increases default ZSTD_c_windowLog to 128 MB
+                                     * except when expressly set to a different value.
+                                     * Note: will be enabled by default if ZSTD_c_windowLog >= 128 MB and
+                                     * compression strategy >= ZSTD_btopt (== compression level 16+) */
+    ZSTD_c_ldmHashLog=161,   /* Size of the table for long distance matching, as a power of 2.
+                              * Larger values increase memory usage and compression ratio,
+                              * but decrease compression speed.
+                              * Must be clamped between ZSTD_HASHLOG_MIN and ZSTD_HASHLOG_MAX
+                              * default: windowlog - 7.
+                              * Special: value 0 means "automatically determine hashlog". */
+    ZSTD_c_ldmMinMatch=162,  /* Minimum match size for long distance matcher.
+                              * Larger/too small values usually decrease compression ratio.
+                              * Must be clamped between ZSTD_LDM_MINMATCH_MIN and ZSTD_LDM_MINMATCH_MAX.
+                              * Special: value 0 means "use default value" (default: 64). */
+    ZSTD_c_ldmBucketSizeLog=163, /* Log size of each bucket in the LDM hash table for collision resolution.
+                              * Larger values improve collision resolution but decrease compression speed.
+                              * The maximum value is ZSTD_LDM_BUCKETSIZELOG_MAX.
+                              * Special: value 0 means "use default value" (default: 3). */
+    ZSTD_c_ldmHashRateLog=164, /* Frequency of inserting/looking up entries into the LDM hash table.
+                              * Must be clamped between 0 and (ZSTD_WINDOWLOG_MAX - ZSTD_HASHLOG_MIN).
+                              * Default is MAX(0, (windowLog - ldmHashLog)), optimizing hash table usage.
+                              * Larger values improve compression speed.
+                              * Deviating far from default value will likely result in a compression ratio decrease.
+                              * Special: value 0 means "automatically determine hashRateLog". */
+
+    /* frame parameters */
+    ZSTD_c_contentSizeFlag=200, /* Content size will be written into frame header _whenever known_ (default:1)
+                              * Content size must be known at the beginning of compression.
+                              * This is automatically the case when using ZSTD_compress2(),
+                              * For streaming scenarios, content size must be provided with ZSTD_CCtx_setPledgedSrcSize() */
+    ZSTD_c_checksumFlag=201, /* A 32-bits checksum of content is written at end of frame (default:0) */
+    ZSTD_c_dictIDFlag=202,   /* When applicable, dictionary's ID is written into frame header (default:1) */
+
+    /* multi-threading parameters */
+    /* These parameters are only active if multi-threading is enabled (compiled with build macro ZSTD_MULTITHREAD).
+     * Otherwise, trying to set any other value than default (0) will be a no-op and return an error.
+     * In a situation where it's unknown if the linked library supports multi-threading or not,
+     * setting ZSTD_c_nbWorkers to any value >= 1 and consulting the return value provides a quick way to check this property.
+     */
+    ZSTD_c_nbWorkers=400,    /* Select how many threads will be spawned to compress in parallel.
+                              * When nbWorkers >= 1, triggers asynchronous mode when invoking ZSTD_compressStream*() :
+                              * ZSTD_compressStream*() consumes input and flush output if possible, but immediately gives back control to caller,
+                              * while compression is performed in parallel, within worker thread(s).
+                              * (note : a strong exception to this rule is when first invocation of ZSTD_compressStream2() sets ZSTD_e_end :
+                              *  in which case, ZSTD_compressStream2() delegates to ZSTD_compress2(), which is always a blocking call).
+                              * More workers improve speed, but also increase memory usage.
+                              * Default value is `0`, aka "single-threaded mode" : no worker is spawned,
+                              * compression is performed inside Caller's thread, and all invocations are blocking */
+    ZSTD_c_jobSize=401,      /* Size of a compression job. This value is enforced only when nbWorkers >= 1.
+                              * Each compression job is completed in parallel, so this value can indirectly impact the nb of active threads.
+                              * 0 means default, which is dynamically determined based on compression parameters.
+                              * Job size must be a minimum of overlap size, or ZSTDMT_JOBSIZE_MIN (= 512 KB), whichever is largest.
+                              * The minimum size is automatically and transparently enforced. */
+    ZSTD_c_overlapLog=402,   /* Control the overlap size, as a fraction of window size.
+                              * The overlap size is an amount of data reloaded from previous job at the beginning of a new job.
+                              * It helps preserve compression ratio, while each job is compressed in parallel.
+                              * This value is enforced only when nbWorkers >= 1.
+                              * Larger values increase compression ratio, but decrease speed.
+                              * Possible values range from 0 to 9 :
+                              * - 0 means "default" : value will be determined by the library, depending on strategy
+                              * - 1 means "no overlap"
+                              * - 9 means "full overlap", using a full window size.
+                              * Each intermediate rank increases/decreases load size by a factor 2 :
+                              * 9: full window;  8: w/2;  7: w/4;  6: w/8;  5:w/16;  4: w/32;  3:w/64;  2:w/128;  1:no overlap;  0:default
+                              * default value varies between 6 and 9, depending on strategy */
+
+    /* note : additional experimental parameters are also available
+     * within the experimental section of the API.
+     * At the time of this writing, they include :
+     * ZSTD_c_rsyncable
+     * ZSTD_c_format
+     * ZSTD_c_forceMaxWindow
+     * ZSTD_c_forceAttachDict
+     * ZSTD_c_literalCompressionMode
+     * ZSTD_c_srcSizeHint
+     * ZSTD_c_enableDedicatedDictSearch
+     * ZSTD_c_stableInBuffer
+     * ZSTD_c_stableOutBuffer
+     * ZSTD_c_blockDelimiters
+     * ZSTD_c_validateSequences
+     * ZSTD_c_useBlockSplitter
+     * ZSTD_c_useRowMatchFinder
+     * ZSTD_c_prefetchCDictTables
+     * ZSTD_c_enableSeqProducerFallback
+     * ZSTD_c_maxBlockSize
+     * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
+     * note : never ever use experimentalParam? names directly;
+     *        also, the enums values themselves are unstable and can still change.
+     */
+     ZSTD_c_experimentalParam1=500,
+     ZSTD_c_experimentalParam2=10,
+     ZSTD_c_experimentalParam3=1000,
+     ZSTD_c_experimentalParam4=1001,
+     ZSTD_c_experimentalParam5=1002,
+     /* was ZSTD_c_experimentalParam6=1003; is now ZSTD_c_targetCBlockSize */
+     ZSTD_c_experimentalParam7=1004,
+     ZSTD_c_experimentalParam8=1005,
+     ZSTD_c_experimentalParam9=1006,
+     ZSTD_c_experimentalParam10=1007,
+     ZSTD_c_experimentalParam11=1008,
+     ZSTD_c_experimentalParam12=1009,
+     ZSTD_c_experimentalParam13=1010,
+     ZSTD_c_experimentalParam14=1011,
+     ZSTD_c_experimentalParam15=1012,
+     ZSTD_c_experimentalParam16=1013,
+     ZSTD_c_experimentalParam17=1014,
+     ZSTD_c_experimentalParam18=1015,
+     ZSTD_c_experimentalParam19=1016
+} ZSTD_cParameter;
+
+typedef struct {
+    size_t error;
+    int lowerBound;
+    int upperBound;
+} ZSTD_bounds;
+
+/*! ZSTD_cParam_getBounds() :
+ *  All parameters must belong to an interval with lower and upper bounds,
+ *  otherwise they will either trigger an error or be automatically clamped.
+ * @return : a structure, ZSTD_bounds, which contains
+ *         - an error status field, which must be tested using ZSTD_isError()
+ *         - lower and upper bounds, both inclusive
+ */
+ZSTDLIB_API ZSTD_bounds ZSTD_cParam_getBounds(ZSTD_cParameter cParam);
+
+/*! ZSTD_CCtx_setParameter() :
+ *  Set one compression parameter, selected by enum ZSTD_cParameter.
+ *  All parameters have valid bounds. Bounds can be queried using ZSTD_cParam_getBounds().
+ *  Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter).
+ *  Setting a parameter is generally only possible during frame initialization (before starting compression).
+ *  Exception : when using multi-threading mode (nbWorkers >= 1),
+ *              the following parameters can be updated _during_ compression (within same frame):
+ *              => compressionLevel, hashLog, chainLog, searchLog, minMatch, targetLength and strategy.
+ *              new parameters will be active for next job only (after a flush()).
+ * @return : an error code (which can be tested using ZSTD_isError()).
+ */
+ZSTDLIB_API size_t ZSTD_CCtx_setParameter(ZSTD_CCtx* cctx, ZSTD_cParameter param, int value);
+
+/*! ZSTD_CCtx_setPledgedSrcSize() :
+ *  Total input data size to be compressed as a single frame.
+ *  Value will be written in frame header, unless if explicitly forbidden using ZSTD_c_contentSizeFlag.
+ *  This value will also be controlled at end of frame, and trigger an error if not respected.
+ * @result : 0, or an error code (which can be tested with ZSTD_isError()).
+ *  Note 1 : pledgedSrcSize==0 actually means zero, aka an empty frame.
+ *           In order to mean "unknown content size", pass constant ZSTD_CONTENTSIZE_UNKNOWN.
+ *           ZSTD_CONTENTSIZE_UNKNOWN is default value for any new frame.
+ *  Note 2 : pledgedSrcSize is only valid once, for the next frame.
+ *           It's discarded at the end of the frame, and replaced by ZSTD_CONTENTSIZE_UNKNOWN.
+ *  Note 3 : Whenever all input data is provided and consumed in a single round,
+ *           for example with ZSTD_compress2(),
+ *           or invoking immediately ZSTD_compressStream2(,,,ZSTD_e_end),
+ *           this value is automatically overridden by srcSize instead.
+ */
+ZSTDLIB_API size_t ZSTD_CCtx_setPledgedSrcSize(ZSTD_CCtx* cctx, unsigned long long pledgedSrcSize);
+
+typedef enum {
+    ZSTD_reset_session_only = 1,
+    ZSTD_reset_parameters = 2,
+    ZSTD_reset_session_and_parameters = 3
+} ZSTD_ResetDirective;
+
+/*! ZSTD_CCtx_reset() :
+ *  There are 2 different things that can be reset, independently or jointly :
+ *  - The session : will stop compressing current frame, and make CCtx ready to start a new one.
+ *                  Useful after an error, or to interrupt any ongoing compression.
+ *                  Any internal data not yet flushed is cancelled.
+ *                  Compression parameters and dictionary remain unchanged.
+ *                  They will be used to compress next frame.
+ *                  Resetting session never fails.
+ *  - The parameters : changes all parameters back to "default".
+ *                  This also removes any reference to any dictionary or external sequence producer.
+ *                  Parameters can only be changed between 2 sessions (i.e. no compression is currently ongoing)
+ *                  otherwise the reset fails, and function returns an error value (which can be tested using ZSTD_isError())
+ *  - Both : similar to resetting the session, followed by resetting parameters.
+ */
+ZSTDLIB_API size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset);
+
+/*! ZSTD_compress2() :
+ *  Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced API.
+ *  (note that this entry point doesn't even expose a compression level parameter).
+ *  ZSTD_compress2() always starts a new frame.
+ *  Should cctx hold data from a previously unfinished frame, everything about it is forgotten.
+ *  - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
+ *  - The function is always blocking, returns when compression is completed.
+ *  NOTE: Providing `dstCapacity >= ZSTD_compressBound(srcSize)` guarantees that zstd will have
+ *        enough space to successfully compress the data, though it is possible it fails for other reasons.
+ * @return : compressed size written into `dst` (<= `dstCapacity),
+ *           or an error code if it fails (which can be tested using ZSTD_isError()).
+ */
+ZSTDLIB_API size_t ZSTD_compress2( ZSTD_CCtx* cctx,
+                                   void* dst, size_t dstCapacity,
+                             const void* src, size_t srcSize);
+
+
+/***********************************************
+*  Advanced decompression API (Requires v1.4.0+)
+************************************************/
+
+/* The advanced API pushes parameters one by one into an existing DCtx context.
+ * Parameters are sticky, and remain valid for all following frames
+ * using the same DCtx context.
+ * It's possible to reset parameters to default values using ZSTD_DCtx_reset().
+ * Note : This API is compatible with existing ZSTD_decompressDCtx() and ZSTD_decompressStream().
+ *        Therefore, no new decompression function is necessary.
+ */
+
+typedef enum {
+
+    ZSTD_d_windowLogMax=100, /* Select a size limit (in power of 2) beyond which
+                              * the streaming API will refuse to allocate memory buffer
+                              * in order to protect the host from unreasonable memory requirements.
+                              * This parameter is only useful in streaming mode, since no internal buffer is allocated in single-pass mode.
+                              * By default, a decompression context accepts window sizes <= (1 << ZSTD_WINDOWLOG_LIMIT_DEFAULT).
+                              * Special: value 0 means "use default maximum windowLog". */
+
+    /* note : additional experimental parameters are also available
+     * within the experimental section of the API.
+     * At the time of this writing, they include :
+     * ZSTD_d_format
+     * ZSTD_d_stableOutBuffer
+     * ZSTD_d_forceIgnoreChecksum
+     * ZSTD_d_refMultipleDDicts
+     * ZSTD_d_disableHuffmanAssembly
+     * ZSTD_d_maxBlockSize
+     * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
+     * note : never ever use experimentalParam? names directly
+     */
+     ZSTD_d_experimentalParam1=1000,
+     ZSTD_d_experimentalParam2=1001,
+     ZSTD_d_experimentalParam3=1002,
+     ZSTD_d_experimentalParam4=1003,
+     ZSTD_d_experimentalParam5=1004,
+     ZSTD_d_experimentalParam6=1005
+
+} ZSTD_dParameter;
+
+/*! ZSTD_dParam_getBounds() :
+ *  All parameters must belong to an interval with lower and upper bounds,
+ *  otherwise they will either trigger an error or be automatically clamped.
+ * @return : a structure, ZSTD_bounds, which contains
+ *         - an error status field, which must be tested using ZSTD_isError()
+ *         - both lower and upper bounds, inclusive
+ */
+ZSTDLIB_API ZSTD_bounds ZSTD_dParam_getBounds(ZSTD_dParameter dParam);
+
+/*! ZSTD_DCtx_setParameter() :
+ *  Set one compression parameter, selected by enum ZSTD_dParameter.
+ *  All parameters have valid bounds. Bounds can be queried using ZSTD_dParam_getBounds().
+ *  Providing a value beyond bound will either clamp it, or trigger an error (depending on parameter).
+ *  Setting a parameter is only possible during frame initialization (before starting decompression).
+ * @return : 0, or an error code (which can be tested using ZSTD_isError()).
+ */
+ZSTDLIB_API size_t ZSTD_DCtx_setParameter(ZSTD_DCtx* dctx, ZSTD_dParameter param, int value);
+
+/*! ZSTD_DCtx_reset() :
+ *  Return a DCtx to clean state.
+ *  Session and parameters can be reset jointly or separately.
+ *  Parameters can only be reset when no active frame is being decompressed.
+ * @return : 0, or an error code, which can be tested with ZSTD_isError()
+ */
+ZSTDLIB_API size_t ZSTD_DCtx_reset(ZSTD_DCtx* dctx, ZSTD_ResetDirective reset);
+
+
+/****************************
+*  Streaming
+****************************/
+
+typedef struct ZSTD_inBuffer_s {
+  const void* src;    /**< start of input buffer */
+  size_t size;        /**< size of input buffer */
+  size_t pos;         /**< position where reading stopped. Will be updated. Necessarily 0 <= pos <= size */
+} ZSTD_inBuffer;
+
+typedef struct ZSTD_outBuffer_s {
+  void*  dst;         /**< start of output buffer */
+  size_t size;        /**< size of output buffer */
+  size_t pos;         /**< position where writing stopped. Will be updated. Necessarily 0 <= pos <= size */
+} ZSTD_outBuffer;
+
+
+
+/*-***********************************************************************
+*  Streaming compression - HowTo
+*
+*  A ZSTD_CStream object is required to track streaming operation.
+*  Use ZSTD_createCStream() and ZSTD_freeCStream() to create/release resources.
+*  ZSTD_CStream objects can be reused multiple times on consecutive compression operations.
+*  It is recommended to reuse ZSTD_CStream since it will play nicer with system's memory, by re-using already allocated memory.
+*
+*  For parallel execution, use one separate ZSTD_CStream per thread.
+*
+*  note : since v1.3.0, ZSTD_CStream and ZSTD_CCtx are the same thing.
+*
+*  Parameters are sticky : when starting a new compression on the same context,
+*  it will reuse the same sticky parameters as previous compression session.
+*  When in doubt, it's recommended to fully initialize the context before usage.
+*  Use ZSTD_CCtx_reset() to reset the context and ZSTD_CCtx_setParameter(),
+*  ZSTD_CCtx_setPledgedSrcSize(), or ZSTD_CCtx_loadDictionary() and friends to
+*  set more specific parameters, the pledged source size, or load a dictionary.
+*
+*  Use ZSTD_compressStream2() with ZSTD_e_continue as many times as necessary to
+*  consume input stream. The function will automatically update both `pos`
+*  fields within `input` and `output`.
+*  Note that the function may not consume the entire input, for example, because
+*  the output buffer is already full, in which case `input.pos < input.size`.
+*  The caller must check if input has been entirely consumed.
+*  If not, the caller must make some room to receive more compressed data,
+*  and then present again remaining input data.
+*  note: ZSTD_e_continue is guaranteed to make some forward progress when called,
+*        but doesn't guarantee maximal forward progress. This is especially relevant
+*        when compressing with multiple threads. The call won't block if it can
+*        consume some input, but if it can't it will wait for some, but not all,
+*        output to be flushed.
+* @return : provides a minimum amount of data remaining to be flushed from internal buffers
+*           or an error code, which can be tested using ZSTD_isError().
+*
+*  At any moment, it's possible to flush whatever data might remain stuck within internal buffer,
+*  using ZSTD_compressStream2() with ZSTD_e_flush. `output->pos` will be updated.
+*  Note that, if `output->size` is too small, a single invocation with ZSTD_e_flush might not be enough (return code > 0).
+*  In which case, make some room to receive more compressed data, and call again ZSTD_compressStream2() with ZSTD_e_flush.
+*  You must continue calling ZSTD_compressStream2() with ZSTD_e_flush until it returns 0, at which point you can change the
+*  operation.
+*  note: ZSTD_e_flush will flush as much output as possible, meaning when compressing with multiple threads, it will
+*        block until the flush is complete or the output buffer is full.
+*  @return : 0 if internal buffers are entirely flushed,
+*            >0 if some data still present within internal buffer (the value is minimal estimation of remaining size),
+*            or an error code, which can be tested using ZSTD_isError().
+*
+*  Calling ZSTD_compressStream2() with ZSTD_e_end instructs to finish a frame.
+*  It will perform a flush and write frame epilogue.
+*  The epilogue is required for decoders to consider a frame completed.
+*  flush operation is the same, and follows same rules as calling ZSTD_compressStream2() with ZSTD_e_flush.
+*  You must continue calling ZSTD_compressStream2() with ZSTD_e_end until it returns 0, at which point you are free to
+*  start a new frame.
+*  note: ZSTD_e_end will flush as much output as possible, meaning when compressing with multiple threads, it will
+*        block until the flush is complete or the output buffer is full.
+*  @return : 0 if frame fully completed and fully flushed,
+*            >0 if some data still present within internal buffer (the value is minimal estimation of remaining size),
+*            or an error code, which can be tested using ZSTD_isError().
+*
+* *******************************************************************/
+
+typedef ZSTD_CCtx ZSTD_CStream;  /**< CCtx and CStream are now effectively same object (>= v1.3.0) */
+                                 /* Continue to distinguish them for compatibility with older versions <= v1.2.0 */
+/*===== ZSTD_CStream management functions =====*/
+ZSTDLIB_API ZSTD_CStream* ZSTD_createCStream(void);
+ZSTDLIB_API size_t ZSTD_freeCStream(ZSTD_CStream* zcs);  /* accept NULL pointer */
+
+/*===== Streaming compression functions =====*/
+typedef enum {
+    ZSTD_e_continue=0, /* collect more data, encoder decides when to output compressed result, for optimal compression ratio */
+    ZSTD_e_flush=1,    /* flush any data provided so far,
+                        * it creates (at least) one new block, that can be decoded immediately on reception;
+                        * frame will continue: any future data can still reference previously compressed data, improving compression.
+                        * note : multithreaded compression will block to flush as much output as possible. */
+    ZSTD_e_end=2       /* flush any remaining data _and_ close current frame.
+                        * note that frame is only closed after compressed data is fully flushed (return value == 0).
+                        * After that point, any additional data starts a new frame.
+                        * note : each frame is independent (does not reference any content from previous frame).
+                        : note : multithreaded compression will block to flush as much output as possible. */
+} ZSTD_EndDirective;
+
+/*! ZSTD_compressStream2() : Requires v1.4.0+
+ *  Behaves about the same as ZSTD_compressStream, with additional control on end directive.
+ *  - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*()
+ *  - Compression parameters cannot be changed once compression is started (save a list of exceptions in multi-threading mode)
+ *  - output->pos must be <= dstCapacity, input->pos must be <= srcSize
+ *  - output->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
+ *  - endOp must be a valid directive
+ *  - When nbWorkers==0 (default), function is blocking : it completes its job before returning to caller.
+ *  - When nbWorkers>=1, function is non-blocking : it copies a portion of input, distributes jobs to internal worker threads, flush to output whatever is available,
+ *                                                  and then immediately returns, just indicating that there is some data remaining to be flushed.
+ *                                                  The function nonetheless guarantees forward progress : it will return only after it reads or write at least 1+ byte.
+ *  - Exception : if the first call requests a ZSTD_e_end directive and provides enough dstCapacity, the function delegates to ZSTD_compress2() which is always blocking.
+ *  - @return provides a minimum amount of data remaining to be flushed from internal buffers
+ *            or an error code, which can be tested using ZSTD_isError().
+ *            if @return != 0, flush is not fully completed, there is still some data left within internal buffers.
+ *            This is useful for ZSTD_e_flush, since in this case more flushes are necessary to empty all buffers.
+ *            For ZSTD_e_end, @return == 0 when internal buffers are fully flushed and frame is completed.
+ *  - after a ZSTD_e_end directive, if internal buffer is not fully flushed (@return != 0),
+ *            only ZSTD_e_end or ZSTD_e_flush operations are allowed.
+ *            Before starting a new compression job, or changing compression parameters,
+ *            it is required to fully flush internal buffers.
+ *  - note: if an operation ends with an error, it may leave @cctx in an undefined state.
+ *          Therefore, it's UB to invoke ZSTD_compressStream2() of ZSTD_compressStream() on such a state.
+ *          In order to be re-employed after an error, a state must be reset,
+ *          which can be done explicitly (ZSTD_CCtx_reset()),
+ *          or is sometimes implied by methods starting a new compression job (ZSTD_initCStream(), ZSTD_compressCCtx())
+ */
+ZSTDLIB_API size_t ZSTD_compressStream2( ZSTD_CCtx* cctx,
+                                         ZSTD_outBuffer* output,
+                                         ZSTD_inBuffer* input,
+                                         ZSTD_EndDirective endOp);
+
+
+/* These buffer sizes are softly recommended.
+ * They are not required : ZSTD_compressStream*() happily accepts any buffer size, for both input and output.
+ * Respecting the recommended size just makes it a bit easier for ZSTD_compressStream*(),
+ * reducing the amount of memory shuffling and buffering, resulting in minor performance savings.
+ *
+ * However, note that these recommendations are from the perspective of a C caller program.
+ * If the streaming interface is invoked from some other language,
+ * especially managed ones such as Java or Go, through a foreign function interface such as jni or cgo,
+ * a major performance rule is to reduce crossing such interface to an absolute minimum.
+ * It's not rare that performance ends being spent more into the interface, rather than compression itself.
+ * In which cases, prefer using large buffers, as large as practical,
+ * for both input and output, to reduce the nb of roundtrips.
+ */
+ZSTDLIB_API size_t ZSTD_CStreamInSize(void);    /**< recommended size for input buffer */
+ZSTDLIB_API size_t ZSTD_CStreamOutSize(void);   /**< recommended size for output buffer. Guarantee to successfully flush at least one complete compressed block. */
+
+
+/* *****************************************************************************
+ * This following ...
 
[truncated message content]

[Valgrind-developers] [valgrind] drd: Split handle_client_request()

[Bart V. A. <bva...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=b6c90012c2ee11b138432914caefe51d1e2d8f2c

commit b6c90012c2ee11b138432914caefe51d1e2d8f2c
Author: Bart Van Assche <bva...@ac...>
Date:   Mon Nov 18 13:15:36 2024 -0800

    drd: Split handle_client_request()
    
    Make handle_client_request() easier to read by splitting it into two
    functions: one for Valgrind core client requests and one for thread-
    related client requests.

Diff:
---
 drd/drd_clientreq.c | 65 ++++++++++++++++++++++++++++++++---------------------
 1 file changed, 40 insertions(+), 25 deletions(-)

diff --git a/drd/drd_clientreq.c b/drd/drd_clientreq.c
index 2f30cb794d..914d9a3482 100644
--- a/drd/drd_clientreq.c
+++ b/drd/drd_clientreq.c
@@ -1,7 +1,7 @@
 /*
   This file is part of drd, a thread error detector.
 
-  Copyright (C) 2006-2020 Bart Van Assche <bva...@ac...>.
+  Copyright (C) 2006-2024 Bart Van Assche <bva...@ac...>.
 
   This program is free software; you can redistribute it and/or
   modify it under the terms of the GNU General Public License as
@@ -48,32 +48,9 @@ Bool DRD_(g_free_is_write);
 
 /* Function definitions. */
 
-/**
- * DRD's handler for Valgrind client requests. The code below handles both
- * DRD's public and tool-internal client requests.
- */
-#if defined(VGP_mips32_linux) || defined(VGP_mips64_linux)
- /* There is a cse related issue in gcc for MIPS. Optimization level
-    has to be lowered, so cse related optimizations are not
-    included. */
- __attribute__((optimize("O1")))
-#endif
-static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
+static Bool handle_core_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
 {
-   UWord result = 0;
-   const DrdThreadId drd_tid = DRD_(thread_get_running_tid)();
-
    tl_assert(vg_tid == VG_(get_running_tid)());
-   tl_assert(DRD_(VgThreadIdToDrdThreadId)(vg_tid) == drd_tid
-             || (VG_USERREQ__GDB_MONITOR_COMMAND == arg[0]
-                 && vg_tid == VG_INVALID_THREADID));
-   /* Check the consistency of vg_tid and drd_tid, unless
-      vgdb has forced the invocation of a gdb monitor cmd
-      when no threads was running (i.e. all threads blocked
-      in a syscall. In such a case, vg_tid is invalid,
-      its conversion to a drd thread id gives also an invalid
-      drd thread id, but drd_tid is not invalid (probably
-      equal to the last running drd thread. */
 
    switch (arg[0])
    {
@@ -124,7 +101,28 @@ static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
                                  &GEI);
       }
       break;
+   }
+
+   *ret = 0;
+   return True;
+}
+
+#if defined(VGP_mips32_linux) || defined(VGP_mips64_linux)
+ /* There is a cse related issue in gcc for MIPS. Optimization level
+    has to be lowered, so cse related optimizations are not
+    included. */
+ __attribute__((optimize("O1")))
+#endif
+static Bool handle_thr_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
+{
+   const DrdThreadId drd_tid = DRD_(thread_get_running_tid)();
+   UWord result = 0;
+
+   tl_assert(vg_tid == VG_(get_running_tid)());
+   tl_assert(DRD_(VgThreadIdToDrdThreadId)(vg_tid) == drd_tid);
 
+   switch (arg[0])
+   {
    case VG_USERREQ__DRD_GET_VALGRIND_THREAD_ID:
       result = vg_tid;
       break;
@@ -623,6 +621,23 @@ static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
    return True;
 }
 
+/**
+ * DRD's handler for Valgrind client requests. The code below handles both
+ * DRD's public and tool-internal client requests.
+ */
+static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
+{
+   if (VG_IS_TOOL_USERREQ(0, 0, arg[0]))
+      return handle_core_client_request(vg_tid, arg, ret);
+
+   if (VG_IS_TOOL_USERREQ('D', 'R', arg[0]) ||
+       VG_IS_TOOL_USERREQ('D', 'r', arg[0]) ||
+       VG_IS_TOOL_USERREQ('H', 'G', arg[0]))
+      return handle_thr_client_request(vg_tid, arg, ret);
+
+   return False;
+}
+
 /**
  * Tell the Valgrind core the address of the DRD function that processes
  * client requests. Must be called before any client code is run.

[Valgrind-developers] [valgrind] drd: Reorder functions

[Bart V. A. <bva...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=a00e024f62818abdedf17fdf87508b237798c2e3

commit a00e024f62818abdedf17fdf87508b237798c2e3
Author: Bart Van Assche <bva...@ac...>
Date:   Mon Nov 18 13:08:29 2024 -0800

    drd: Reorder functions
    
    Reorder two functions such that a forward declaration can be removed.

Diff:
---
 drd/drd_clientreq.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/drd/drd_clientreq.c b/drd/drd_clientreq.c
index 8a462253df..2f30cb794d 100644
--- a/drd/drd_clientreq.c
+++ b/drd/drd_clientreq.c
@@ -46,22 +46,8 @@
 Bool DRD_(g_free_is_write);
 
 
-/* Local function declarations. */
-
-static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret);
-
-
 /* Function definitions. */
 
-/**
- * Tell the Valgrind core the address of the DRD function that processes
- * client requests. Must be called before any client code is run.
- */
-void DRD_(clientreq_init)(void)
-{
-   VG_(needs_client_requests)(handle_client_request);
-}
-
 /**
  * DRD's handler for Valgrind client requests. The code below handles both
  * DRD's public and tool-internal client requests.
@@ -636,3 +622,12 @@ static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
    *ret = result;
    return True;
 }
+
+/**
+ * Tell the Valgrind core the address of the DRD function that processes
+ * client requests. Must be called before any client code is run.
+ */
+void DRD_(clientreq_init)(void)
+{
+   VG_(needs_client_requests)(handle_client_request);
+}

[Valgrind-developers] [valgrind] drd: Remove a superfluous if-statement

[Bart V. A. <bva...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=bc57ecb85b97347e31334769d615b4553d3463d4

commit bc57ecb85b97347e31334769d615b4553d3463d4
Author: Bart Van Assche <bva...@ac...>
Date:   Sun Nov 17 16:20:39 2024 -0800

    drd: Remove a superfluous if-statement
    
    Check the client request ID once instead of twice.

Diff:
---
 drd/drd_clientreq.c | 14 --------------
 1 file changed, 14 deletions(-)

diff --git a/drd/drd_clientreq.c b/drd/drd_clientreq.c
index 79f515dda2..8a462253df 100644
--- a/drd/drd_clientreq.c
+++ b/drd/drd_clientreq.c
@@ -77,20 +77,6 @@ static Bool handle_client_request(ThreadId vg_tid, UWord* arg, UWord* ret)
    UWord result = 0;
    const DrdThreadId drd_tid = DRD_(thread_get_running_tid)();
 
-   if (!VG_IS_TOOL_USERREQ('D','R',arg[0])
-       && !VG_IS_TOOL_USERREQ('D','r',arg[0])
-       && !VG_IS_TOOL_USERREQ('H','G',arg[0])
-       && VG_USERREQ__MALLOCLIKE_BLOCK != arg[0]
-       && VG_USERREQ__RESIZEINPLACE_BLOCK != arg[0]
-       && VG_USERREQ__FREELIKE_BLOCK != arg[0]
-#if defined(VGO_solaris)
-       && VG_USERREQ_DRD_RTLD_BIND_GUARD != arg[0]
-       && VG_USERREQ_DRD_RTLD_BIND_CLEAR != arg[0]
-#endif /* VGO_solaris */
-       && VG_USERREQ__GDB_MONITOR_COMMAND != arg[0]) {
-         return False;
-   }
-
    tl_assert(vg_tid == VG_(get_running_tid)());
    tl_assert(DRD_(VgThreadIdToDrdThreadId)(vg_tid) == drd_tid
              || (VG_USERREQ__GDB_MONITOR_COMMAND == arg[0]

[Valgrind-developers] [valgrind] Solaris - fix a couple of warnings for message formatting

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=e1d563f182644e58868b403a9f36ade78ed6e165

commit e1d563f182644e58868b403a9f36ade78ed6e165
Author: Paul Floyd <pj...@wa...>
Date:   Sun Nov 17 19:47:24 2024 +0100

    Solaris - fix a couple of warnings for message formatting

Diff:
---
 coregrind/m_syswrap/syswrap-solaris.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/coregrind/m_syswrap/syswrap-solaris.c b/coregrind/m_syswrap/syswrap-solaris.c
index 9d869d9b5f..104a809d3d 100644
--- a/coregrind/m_syswrap/syswrap-solaris.c
+++ b/coregrind/m_syswrap/syswrap-solaris.c
@@ -2739,7 +2739,7 @@ PRE(sys_shmsys)
 #if defined(SOLARIS_SHM_NEW)
    case VKI_SHMADV:
       /* Libc: int shmadv(int shmid, uint_t cmd, uint_t *advice); */
-      PRINT("sys_shmsys ( %ld, %ld, %lu, %ld )",
+      PRINT("sys_shmsys ( %ld, %ld, %lu, %lu )",
             SARG1, SARG2, ARG3, ARG4);
       PRE_REG_READ4(long, SC2("shmsys", "shmadv"), int, opcode,
                     int, shmid, vki_uint_t, cmd, vki_uint_t *, advice);
@@ -6895,7 +6895,7 @@ PRE(sys_modctl)
 
       default:
          VG_(unimplemented)("Syswrap of the modctl call with command "
-                            "MODNVL_DEVLINKSYNC and op %ld.", ARG2);
+                            "MODNVL_DEVLINKSYNC and op %lu.", ARG2);
          /*NOTREACHED*/
          break;
       }

[Valgrind-developers] [valgrind] Illumos regtest: turn off memcheck/tests/x86/bug487993

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=3af6f369e7cac5275c3e3bdb8470e084568f3443

commit 3af6f369e7cac5275c3e3bdb8470e084568f3443
Author: Paul Floyd <pj...@wa...>
Date:   Sun Nov 17 11:08:34 2024 +0100

    Illumos regtest: turn off memcheck/tests/x86/bug487993
    
    For some reason GCC 13.3 on Illumos does not use aligned operator
    new when compiling this testcase. So turn it off.

Diff:
---
 memcheck/tests/x86/bug487993.vgtest | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/memcheck/tests/x86/bug487993.vgtest b/memcheck/tests/x86/bug487993.vgtest
index ca5db1515d..a5f00a4821 100644
--- a/memcheck/tests/x86/bug487993.vgtest
+++ b/memcheck/tests/x86/bug487993.vgtest
@@ -1,3 +1,3 @@
-prereq: test -e bug487993
+prereq: test -e bug487993 && ! ../../../tests/os_test solaris
 prog: bug487993
 vgopts: -q

Re: [Valgrind-developers] [PATCH] libcfile: Resolve deleted fd to procfs path

[Dominik B. <fre...@gm...>]

Previous commit linked the wrong bug report. Sorry.

-- >8 --
Applies to Linux and Solaris.

If the file referenced by the file descriptor passed to execveat(2) has
been deleted prior to the syscall or is an anonymous file created by
memfd_create(2), i.e. if the file has no hard links (st_nlinks == 0),
the execveat wrapper would fail with errno set to ENOENT.

The execveat wrapper uses VG_(resolve_filename) to retrieve an absolute
path to the file or directory pointed to by the dirfd argument. This
function uses readlink(2) to read the appropriate link in the procfs
directory /proc/self/fd. In the case of files with no hard links,
the output of readlink contains the suffix ' (deleted)', which leads to
execveat passing the incorrect path on to execve. Links in procfs to
files created with memfd_create additionally point to a bogus path in
the format '/memfd:NAME', where 'NAME' is determined by an argument to
the function.

This commit changes VG_(resolve_filename) to execute a stat on the
procfs link and check if nlinks is equal to 0. If so, the procfs path is
returned and readlink is not called.

https://bugs.kde.org/show_bug.cgi?id=496353
---
 coregrind/m_libcfile.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/coregrind/m_libcfile.c b/coregrind/m_libcfile.c
index fb93b4867..2c72e3185 100644
--- a/coregrind/m_libcfile.c
+++ b/coregrind/m_libcfile.c
@@ -111,6 +111,14 @@ Bool VG_(resolve_filename) ( Int fd, const HChar** result )
 #     endif
    }

+   struct vg_stat statbuf;
+   if (VG_(fstat)(fd, &statbuf) == 0 && statbuf.nlink == 0) {
+       buf = VG_(malloc)("resolve_filename", sizeof(tmp));
+       VG_(strcpy)(buf, tmp);
+       *result = buf;
+       return True;
+   }
+
    while (42) {
       SSizeT res = VG_(readlink)(tmp, buf, bufsiz);
       if (res < 0) break;
-- 
2.47.0

On Sat, Nov 16, 2024 at 6:39 PM Dominik Banaszak <fre...@gm...> wrote:
>
> Applies to Linux and Solaris.
>
> If the file referenced by the file descriptor passed to execveat(2) has
> been deleted prior to the syscall or is an anonymous file created by
> memfd_create(2), i.e. if the file has no hard links (st_nlinks == 0),
> the execveat wrapper would fail with errno set to ENOENT.
>
> The execveat wrapper uses VG_(resolve_filename) to retrieve an absolute
> path to the file or directory pointed to by the dirfd argument. This
> function uses readlink(2) to read the appropriate link in the procfs
> directory /proc/self/fd. In the case of files with no hard links,
> the output of readlink contains the suffix ' (deleted)', which leads to
> execveat passing the incorrect path on to execve. Links in procfs to
> files created with memfd_create additionally point to a bogus path in
> the format '/memfd:NAME', where 'NAME' is determined by an argument to
> the function.
>
> This commit changes VG_(resolve_filename) to execute a stat on the
> procfs link and check if nlinks is equal to 0. If so, the procfs path is
> returned and readlink is not called.
>
> https://bugs.kde.org/show_bug.cgi?id=379094
> ---
>  coregrind/m_libcfile.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/coregrind/m_libcfile.c b/coregrind/m_libcfile.c
> index fb93b4867..2c72e3185 100644
> --- a/coregrind/m_libcfile.c
> +++ b/coregrind/m_libcfile.c
> @@ -111,6 +111,14 @@ Bool VG_(resolve_filename) ( Int fd, const HChar** result )
>  #     endif
>     }
>
> +   struct vg_stat statbuf;
> +   if (VG_(fstat)(fd, &statbuf) == 0 && statbuf.nlink == 0) {
> +       buf = VG_(malloc)("resolve_filename", sizeof(tmp));
> +       VG_(strcpy)(buf, tmp);
> +       *result = buf;
> +       return True;
> +   }
> +
>     while (42) {
>        SSizeT res = VG_(readlink)(tmp, buf, bufsiz);
>        if (res < 0) break;
> --
> 2.47.0

[Valgrind-developers] [PATCH] libcfile: Resolve deleted fd to procfs path

[Dominik B. <fre...@gm...>]

Applies to Linux and Solaris.

If the file referenced by the file descriptor passed to execveat(2) has
been deleted prior to the syscall or is an anonymous file created by
memfd_create(2), i.e. if the file has no hard links (st_nlinks == 0),
the execveat wrapper would fail with errno set to ENOENT.

The execveat wrapper uses VG_(resolve_filename) to retrieve an absolute
path to the file or directory pointed to by the dirfd argument. This
function uses readlink(2) to read the appropriate link in the procfs
directory /proc/self/fd. In the case of files with no hard links,
the output of readlink contains the suffix ' (deleted)', which leads to
execveat passing the incorrect path on to execve. Links in procfs to
files created with memfd_create additionally point to a bogus path in
the format '/memfd:NAME', where 'NAME' is determined by an argument to
the function.

This commit changes VG_(resolve_filename) to execute a stat on the
procfs link and check if nlinks is equal to 0. If so, the procfs path is
returned and readlink is not called.

https://bugs.kde.org/show_bug.cgi?id=379094
---
 coregrind/m_libcfile.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/coregrind/m_libcfile.c b/coregrind/m_libcfile.c
index fb93b4867..2c72e3185 100644
--- a/coregrind/m_libcfile.c
+++ b/coregrind/m_libcfile.c
@@ -111,6 +111,14 @@ Bool VG_(resolve_filename) ( Int fd, const HChar** result )
 #     endif
    }

+   struct vg_stat statbuf;
+   if (VG_(fstat)(fd, &statbuf) == 0 && statbuf.nlink == 0) {
+       buf = VG_(malloc)("resolve_filename", sizeof(tmp));
+       VG_(strcpy)(buf, tmp);
+       *result = buf;
+       return True;
+   }
+
    while (42) {
       SSizeT res = VG_(readlink)(tmp, buf, bufsiz);
       if (res < 0) break;
-- 
2.47.0

[Valgrind-developers] [valgrind] arm: hook kcmp syscall

[Mark W. <ma...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=88b42358eb10783896b327faccae3d99e1853062

commit 88b42358eb10783896b327faccae3d99e1853062
Author: Lucas Stach <l....@pe...>
Date:   Thu Nov 14 16:58:23 2024 +0100

    arm: hook kcmp syscall

Diff:
---
 coregrind/m_syswrap/syswrap-arm-linux.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/coregrind/m_syswrap/syswrap-arm-linux.c b/coregrind/m_syswrap/syswrap-arm-linux.c
index d326fdb9ee..c8c23a9282 100644
--- a/coregrind/m_syswrap/syswrap-arm-linux.c
+++ b/coregrind/m_syswrap/syswrap-arm-linux.c
@@ -1008,6 +1008,7 @@ static SyscallTableEntry syscall_main_table[] = {
    LINX_(__NR_setns,             sys_setns),            // 375
    LINXY(__NR_process_vm_readv,  sys_process_vm_readv), // 376
    LINX_(__NR_process_vm_writev, sys_process_vm_writev),// 377
+   LINX_(__NR_kcmp,              sys_kcmp),             // 378
 
    LINX_(__NR_sched_setattr,     sys_sched_setattr),    // 380
    LINXY(__NR_sched_getattr,     sys_sched_getattr),    // 381

Re: [Valgrind-developers] [PATCH] arm: hook kcmp syscall

[Mark W. <ma...@kl...>]

This makes sense. It is how other linux arches hook kcmp. Thanks.

Pushed,

Mark

On Thu, Nov 14, 2024 at 04:58:23PM +0100, Lucas Stach wrote:
> ---
>  coregrind/m_syswrap/syswrap-arm-linux.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/coregrind/m_syswrap/syswrap-arm-linux.c b/coregrind/m_syswrap/syswrap-arm-linux.c
> index d326fdb9eeda..c8c23a928209 100644
> --- a/coregrind/m_syswrap/syswrap-arm-linux.c
> +++ b/coregrind/m_syswrap/syswrap-arm-linux.c
> @@ -1008,6 +1008,7 @@ static SyscallTableEntry syscall_main_table[] = {
>     LINX_(__NR_setns,             sys_setns),            // 375
>     LINXY(__NR_process_vm_readv,  sys_process_vm_readv), // 376
>     LINX_(__NR_process_vm_writev, sys_process_vm_writev),// 377
> +   LINX_(__NR_kcmp,              sys_kcmp),             // 378
>  
>     LINX_(__NR_sched_setattr,     sys_sched_setattr),    // 380
>     LINXY(__NR_sched_getattr,     sys_sched_getattr),    // 381
> -- 
> 2.47.0
> 
> 
> 
> _______________________________________________
> Valgrind-developers mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-developers

[Valgrind-developers] [PATCH] arm: hook kcmp syscall

[Lucas S. <l....@pe...>]

---
 coregrind/m_syswrap/syswrap-arm-linux.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/coregrind/m_syswrap/syswrap-arm-linux.c b/coregrind/m_syswrap/syswrap-arm-linux.c
index d326fdb9eeda..c8c23a928209 100644
--- a/coregrind/m_syswrap/syswrap-arm-linux.c
+++ b/coregrind/m_syswrap/syswrap-arm-linux.c
@@ -1008,6 +1008,7 @@ static SyscallTableEntry syscall_main_table[] = {
    LINX_(__NR_setns,             sys_setns),            // 375
    LINXY(__NR_process_vm_readv,  sys_process_vm_readv), // 376
    LINX_(__NR_process_vm_writev, sys_process_vm_writev),// 377
+   LINX_(__NR_kcmp,              sys_kcmp),             // 378
 
    LINX_(__NR_sched_setattr,     sys_sched_setattr),    // 380
    LINXY(__NR_sched_getattr,     sys_sched_getattr),    // 381
-- 
2.47.0

[Valgrind-developers] [valgrind] coregrind/m_gdbserver/remote-utils.c (prepare_resume_reply): Use memcpy

[Mark W. <ma...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=d7db69fbf482149e18ed1977df41fe320f5cbef2

commit d7db69fbf482149e18ed1977df41fe320f5cbef2
Author: Mark Wielaard <ma...@kl...>
Date:   Thu Nov 14 12:25:27 2024 +0100

    coregrind/m_gdbserver/remote-utils.c (prepare_resume_reply): Use memcpy
    
    GCC8 (but apparently not later versions) complain about the use of
    strncpy when not actually copying a string:
    
    remote-utils.c:1140:14: warning: 'char* strncpy(char*, const char*, size_t)' output truncated before terminating nul copying 6 bytes from a string of the same length [-Wstringop-truncation]
              strncpy (buf, "watch:", 6);
              ~~~~~~~~^~~~~~~~~~~~~~~~~~
    
    This is "harmless" because buf is large enough and we will add more
    chars (including a zero terminator) later. But using strncpy here is a
    bit odd because we don't really want to copy a string, but an array of
    6 chars. So use memcpy here to do so, simplyfing the code.

Diff:
---
 coregrind/m_gdbserver/remote-utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/coregrind/m_gdbserver/remote-utils.c b/coregrind/m_gdbserver/remote-utils.c
index b22dc4482b..8dfc768d49 100644
--- a/coregrind/m_gdbserver/remote-utils.c
+++ b/coregrind/m_gdbserver/remote-utils.c
@@ -1137,7 +1137,7 @@ void prepare_resume_reply (char *buf, char status, unsigned char sig)
          CORE_ADDR addr;
          int i;
 
-         strncpy (buf, "watch:", 6);
+         VG_(memcpy) (buf, "watch:", 6);
          buf += 6;
 
          addr = valgrind_stopped_data_address ();

[Valgrind-developers] [valgrind] Illumos regtest: x86 mdb diffs wrt Solaris

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=1698bd8d62fa6d3c9cf4628e289bb7aafb858136

commit 1698bd8d62fa6d3c9cf4628e289bb7aafb858136
Author: Paul Floyd <pj...@wa...>
Date:   Thu Nov 14 08:57:37 2024 +0100

    Illumos regtest: x86 mdb diffs wrt Solaris

Diff:
---
 none/tests/x86-solaris/Makefile.am                         |  1 +
 none/tests/x86-solaris/coredump_single_thread_mdb          |  3 ++-
 .../coredump_single_thread_sse.post.exp-illumos            | 14 ++++++++++++++
 3 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/none/tests/x86-solaris/Makefile.am b/none/tests/x86-solaris/Makefile.am
index aecc7b7d36..6744edfb0f 100644
--- a/none/tests/x86-solaris/Makefile.am
+++ b/none/tests/x86-solaris/Makefile.am
@@ -13,6 +13,7 @@ EXTRA_DIST = \
 	coredump_single_thread.stdout.exp coredump_single_thread.vgtest \
 	coredump_single_thread_sse.post.exp coredump_single_thread_sse.stderr.exp \
 	coredump_single_thread_sse.stdout.exp coredump_single_thread_sse.vgtest \
+	coredump_single_thread_sse.post.exp-illumos \
 	syscalls.stderr.exp syscalls.stdout.exp syscalls.vgtest
 
 check_PROGRAMS = \
diff --git a/none/tests/x86-solaris/coredump_single_thread_mdb b/none/tests/x86-solaris/coredump_single_thread_mdb
index b0053c60e6..90a02e55b6 100755
--- a/none/tests/x86-solaris/coredump_single_thread_mdb
+++ b/none/tests/x86-solaris/coredump_single_thread_mdb
@@ -20,4 +20,5 @@ sed '/id=/ d' | \
 sed '/status=/ d'
 
 echo "\n---Stack trace---"
-echo "::stack ! perl -p -e 's/^(\S+)\+.*/\$1/g'" | /usr/bin/mdb vgcore.*
+echo "::stack ! perl -p -e 's/^(\S+)\+.*/\$1/g'" | /usr/bin/mdb vgcore.* | \
+sed '/_start_crt/d'
diff --git a/none/tests/x86-solaris/coredump_single_thread_sse.post.exp-illumos b/none/tests/x86-solaris/coredump_single_thread_sse.post.exp-illumos
new file mode 100644
index 0000000000..18b7a470b5
--- /dev/null
+++ b/none/tests/x86-solaris/coredump_single_thread_sse.post.exp-illumos
@@ -0,0 +1,14 @@
+---Status---
+debugging core file of coredump_single (32-bit) from ...
+initial argv: ./coredump_single_thread_sse
+status: process terminated by SIGSEGV (Segmentation Fault), addr=........
+
+---SSE Registers---
+%xmm0  [127:0]   0x12345678 9abcdef0 fedbca98 76543210
+%xmm1  [127:0]   0x23456789 09876543 21fedcba 9467feca
+%xmm2  [127:0]   0xabcdabcd cedecede fabafaba 50656754
+%xmm3  [127:0]   0x03050608 1d1b4b15 25272120 373a3d35
+%xmm4  [127:0]   0x9abcdef0 76543210 12345678 fedbca98
+%xmm5  [127:0]   0x9467feca 23456789 21fedcba 09876543
+%xmm6  [127:0]   0x50656754 cedecede abcdabcd fabafaba
+%xmm7  [127:0]   0x373a3d35 1d1b4b15 03050608 25272120

Re: [Valgrind-developers] Critical bug report ignored and closed

[Paul F. <pj...@wa...>]

On 12-11-24 20:05, Rm Beer wrote:
> Hello, I come here to complain about the negligence committed by the
> developers in the 'Bug List' of the valgrind site:

Getting shirty will get you nowhere.

Your report should have been closed as a duplicate of 
https://bugs.kde.org/show_bug.cgi?id=286864

You should read the above item

I'll try to summarize the problem.

After Valgrind has bootstrapped itself it will load and execute the 
guest exe. In practice that means also loading and executing the 
interpreter, ld.so. ld.so uses some C functions, but since it runs 
before libc has been loaded it has to provide its own versions (when 
libc loads it will replace the ones in ld.so). Valgrind may need to 
redirect some of those functions. The case in point is strlen. When 
strlen implementations are optimized to use SIMD to count multiple 
characters at a time in parallel it may deliberately read past the end 
of the string in the knowledge that it is safe because memory is 
allocated in multiples of 8 or 16 bytes. If Valgrind doesn't replace 
strlen then it may result in large numbers of invalid read errors. We 
have no way of knowing whether the strlen in ld.so uses SIMD.

So we have a choice.

1. We don't ensure that ld.so strlen is redirected, which will make some 
systems unusable.
2. We do ensure that strlen is redirected. That works as long as ld.so 
is not stripped.

If you are really not happy that we have chosen the first one then you 
can build your own Valgrind from source using the patch in the Bugzilla 
item above.

A+
Paul

Re: [Valgrind-developers] Critical bug report ignored and closed

[Mark W. <ma...@kl...>]

Hi Rm,

On Tue, Nov 12, 2024 at 04:05:41PM -0300, Rm Beer wrote:
> Hello, I come here to complain about the negligence committed by the
> developers in the 'Bug List' of the valgrind site:
> 
> https://bugs.kde.org/show_bug.cgi?id=495761
> 
> Nobody is able to read what I write, I repeatedly repeat that the
> Operating System or the distro has nothing to do with it, that the
> error comes from the valgrind source code.

We do read what you write and we care about making sure things work
for you. But it isn't clear you reading our answers. Your distro needs
to provide the symbols/debuginfo for ld.so otherwise valgrind cannot
work.

In the bug you say you are using ArchLinux. A quick search seems to
indicate the other people using ArchLinux that encounter the issue you
are seeing do have a solution.

export DEBUGINFOD_URLS="https://debuginfod.archlinux.org"

See here:
https://bbs.archlinux.org/viewtopic.php?pid=2036171#p2036171

More background info on your distribution and debuginfo can be found
here:

https://wiki.archlinux.org/title/Debuginfod
https://wiki.archlinux.org/title/Debugging

Hope that helps.

Cheers,

Mark

[Valgrind-developers] Critical bug report ignored and closed

[Rm B. <rm...@gm...>]

Hello, I come here to complain about the negligence committed by the
developers in the 'Bug List' of the valgrind site:

https://bugs.kde.org/show_bug.cgi?id=495761

Nobody is able to read what I write, I repeatedly repeat that the
Operating System or the distro has nothing to do with it, that the
error comes from the valgrind source code.

The evidence is already in the posts of the thread, this lack of
responsibility or act of total negligence deserves attention. No
wonder the valgrind bug persists for years.

The thread needs to be reopened and examined.

Thanks.

[Valgrind-developers] [valgrind] vgdb.c (fork_and_exec_valgrind): Fix another off-by-one error write

[Mark W. <ma...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=747ca4eb5fed5dd58a14391a997bb9e658e3b1c8

commit 747ca4eb5fed5dd58a14391a997bb9e658e3b1c8
Author: Mark Wielaard <ma...@kl...>
Date:   Tue Nov 12 13:34:09 2024 +0100

    vgdb.c (fork_and_exec_valgrind): Fix another off-by-one error write
    
    commit 646978d9adc5 ("vgdb: Handle EINTR and EAGAIN more
    consistently") introduced another off-by-one issue trying to write
    back the error from child to parent.
    
    Instead of +1 it should have been +written (which initially is zero).
    
    This is when the child needs to do a chdir and that chdir fails. If
    that happens the parent would have gotten the wrong error code.

Diff:
---
 coregrind/vgdb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/coregrind/vgdb.c b/coregrind/vgdb.c
index 90e3f15020..409ceac5d6 100644
--- a/coregrind/vgdb.c
+++ b/coregrind/vgdb.c
@@ -1289,7 +1289,7 @@ int fork_and_exec_valgrind (int argc, char **argv, const char *working_dir,
             // We try to write the result to the parent, but always exit.
             size_t written = 0;
             while (written < sizeof (int)) {
-               int nrw = write (pipefd[1], ((char *)&err) + 1,
+               int nrw = write (pipefd[1], ((char *)&err) + written,
                                 sizeof (int) - written);
                if (nrw == -1) {
                   if (errno == EINTR || errno == EAGAIN)

[Valgrind-developers] [valgrind] vgdb.c (fork_and_exec_valgrind): Fix off-by-one error write

[Mark W. <ma...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=f4fe5faf3d0f45b3824bbb9070232682df52a582

commit f4fe5faf3d0f45b3824bbb9070232682df52a582
Author: Mark Wielaard <ma...@kl...>
Date:   Tue Nov 12 13:23:03 2024 +0100

    vgdb.c (fork_and_exec_valgrind): Fix off-by-one error write
    
    commit 646978d9adc5 ("vgdb: Handle EINTR and EAGAIN more
    consistently") introduced an off-by-one issue trying to write back the
    error from child to parent.
    
    Instead of +1 it should have been +written (which initially is zero).
    
    This is in an "should never happen" path, so hopefully didn't really
    cause issues. But if it did happen the parent would have gotten the
    wrong error code.

Diff:
---
 coregrind/vgdb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/coregrind/vgdb.c b/coregrind/vgdb.c
index d6740aecd4..90e3f15020 100644
--- a/coregrind/vgdb.c
+++ b/coregrind/vgdb.c
@@ -1368,7 +1368,7 @@ int fork_and_exec_valgrind (int argc, char **argv, const char *working_dir,
       // We try to write the result to the parent, but always exit.
       size_t written = 0;
       while (written < sizeof (int)) {
-         ssize_t nrw = write (pipefd[1], ((char *) &err) + 1,
+         ssize_t nrw = write (pipefd[1], ((char *) &err) + written,
                               sizeof (int) - written);
          if (nrw == -1) {
             if (errno == EINTR || errno == EAGAIN)

[Valgrind-developers] [valgrind] Illumos regtest: updates in none/tests/amd64-solaris

[Paul F. <pa...@so...>]

https://sourceware.org/git/gitweb.cgi?p=valgrind.git;h=bc9daa9cd15aeee69a1180268e1a132d5574d8cd

commit bc9daa9cd15aeee69a1180268e1a132d5574d8cd
Author: Paul Floyd <pj...@wa...>
Date:   Mon Nov 11 09:23:34 2024 +0100

    Illumos regtest: updates in none/tests/amd64-solaris
    
    Add a filter for an extra level in a callstack.
    Add an expected for changed mdb output.

Diff:
---
 none/tests/amd64-solaris/Makefile.am               |  1 +
 .../tests/amd64-solaris/coredump_single_thread_mdb |  3 ++-
 .../coredump_single_thread_sse.post.exp-illumos    | 22 ++++++++++++++++++++++
 3 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/none/tests/amd64-solaris/Makefile.am b/none/tests/amd64-solaris/Makefile.am
index 5eb6905f58..f34d23f703 100644
--- a/none/tests/amd64-solaris/Makefile.am
+++ b/none/tests/amd64-solaris/Makefile.am
@@ -11,6 +11,7 @@ EXTRA_DIST = \
 	coredump_single_thread.post.exp coredump_single_thread.stderr.exp \
 	coredump_single_thread.stdout.exp coredump_single_thread.vgtest \
 	coredump_single_thread_sse.post.exp coredump_single_thread_sse.stderr.exp \
+	coredump_single_thread_sse.post.exp-illumos \
 	coredump_single_thread_sse.stdout.exp coredump_single_thread_sse.vgtest \
 	syscall_return_args.stderr.exp syscall_return_args.vgtest
 
diff --git a/none/tests/amd64-solaris/coredump_single_thread_mdb b/none/tests/amd64-solaris/coredump_single_thread_mdb
index 082d78a665..8d0779addc 100755
--- a/none/tests/amd64-solaris/coredump_single_thread_mdb
+++ b/none/tests/amd64-solaris/coredump_single_thread_mdb
@@ -23,4 +23,5 @@ sed '/^%gsbase = / d' | \
 sed '/^%fsbase = / d' \
 
 echo "\n---Stack trace---"
-echo "::stack ! perl -p -e 's/^(\S+)\+.*/\$1/g'" | /usr/bin/mdb vgcore.*
+echo "::stack ! perl -p -e 's/^(\S+)\+.*/\$1/g'" | /usr/bin/mdb vgcore.* | \
+sed '/_start_crt/d'
diff --git a/none/tests/amd64-solaris/coredump_single_thread_sse.post.exp-illumos b/none/tests/amd64-solaris/coredump_single_thread_sse.post.exp-illumos
new file mode 100644
index 0000000000..d8eeef4726
--- /dev/null
+++ b/none/tests/amd64-solaris/coredump_single_thread_sse.post.exp-illumos
@@ -0,0 +1,22 @@
+---Status---
+debugging core file of coredump_single (64-bit) from ...
+initial argv: ./coredump_single_thread_sse
+status: process terminated by SIGSEGV (Segmentation Fault), addr=........
+
+---SSE Registers---
+%xmm0  [127:0]   0x10325476 98cadbfe f0debc9a 78563412
+%xmm1  [127:0]   0xcafe6794 badcfe21 43658709 89674523
+%xmm2  [127:0]   0x54676550 bafabafa decedece cdabcdab
+%xmm3  [127:0]   0x353d3a37 20212725 154b1b1d 08060503
+%xmm4  [127:0]   0x98cadbfe 78563412 10325476 f0debc9a
+%xmm5  [127:0]   0x43658709 badcfe21 89674523 cafe6794
+%xmm6  [127:0]   0xbafabafa cdabcdab decedece 54676550
+%xmm7  [127:0]   0x20212725 08060503 154b1b1d 353d3a37
+%xmm8  [127:0]   0x8ab135bc 296692dc dd839600 5eb11524
+%xmm9  [127:0]   0x226834d5 30d3f7c6 959a8ebd 4b5072c6
+%xmm10 [127:0]   0xbdf31a8a b5c8db03 c7d15cf4 b6cad1c8
+%xmm11 [127:0]   0x9d5c4ba3 e5fa0835 2175d7a2 526d6010
+%xmm12 [127:0]   0x6e8063f5 6f00222c 7232bcb0 be2187ab
+%xmm13 [127:0]   0x0a0755e9 a620c325 f03b9fef ab4a333d
+%xmm14 [127:0]   0x04069c6a ba536a25 dfd2fda5 b8a12978
+%xmm15 [127:0]   0x2a67ef93 9534a7e1 242ddd04 9b9e3936