#815 new plotting style 'marks', 'linesmarks' and new object 'mark'

[Hiroki Motoyoshi]

This patch introduces new 2D plotting styles 'marks', 'linesmarks' and 'mark' object.

For more information, see the attached documentation about the patch.
The sample programs for the figures in the documentaion are also attached as the tar+gzipped archive file.

[Ethan Merritt]

I had assumed that only the final code would be committed to the main branch once the discussion and specification work here had settled down.

That was what I planned, but I became concerned that my revisions ( 548685a5e and 059c46d5d) touched so many lines of code that any further patches from your side would not apply cleanly. Maybe I should have created a new branch on SourceForge instead? But I have not yet figured out how to share access to those or to cherry-pick from them. I am still a newcomer to the world of git and have not yet learned how best to work in parallel with someone else.

one of my motivations for creating the original patch because the gnuplot's polygon routine can not draw transparent holes

Could you give me an example of what you mean by a "transparent hole" in a set of polygons? I don't understand how that relates to colors. How is it different from an empty (no fill) facet in a larger surface?

One issue I came across was whether a "marktype 0" should be accepted. I can see that it makes the nice windbarbs example simpler. But all the other gnuplot entities created by set <something> <tag> ... require that tag > 0. (linetypes, objects, labels, pixmaps, arrow,...) and treat tag=0 as an error.

[Hiroki Motoyoshi]

One issue I came across was whether a "marktype 0" should be accepted. I can see that it makes the nice windbarbs example simpler. But all the other gnuplot entities created by set <something> <tag> ... require that tag > 0. (linetypes, objects, labels, pixmaps, arrow,...) and treat tag=0 as an error.</tag></something>

My intention is that the marktypes that the user can set with set mark are 0 and positive number. I want to reserve the negative number marktype for internal use, built-in marks, etc.

Could you give me an example of what you mean by a "transparent hole" in a set of polygons? I don't understand how that relates to colors. How is it different from an empty (no fill) facet in a larger surface?

For example, it doesn't seem possible to drill a hole in a polygon with another polygon in a simple way like fill-rule in SVG.
https://www.w3.org/TR/SVG/painting.html#FillRuleProperty

If this is not possible, my intention is that if we fill the polygon with a polygon with a white (or background color) fillcolor attribute as the polygon representing the inner hole, we can obtain a shape similar to that. If I want to represent it as a single mark, I need to assign a color to each of the polygons that make up the mark.

[Hiroki Motoyoshi]

Since there is no 4th vertex property, you can't switch colors in the middle of a mark definition. On the other hand, the fill and stroke colors can be different from the start. It would not be that much work to add back a vertex color if needed.

In my opinion, it is more valuable to be able to switch colors in the middle of a mark definition. By doing so, fill and stroke can be superimposed as separate definitions, and as a result, the border color can be specified separately. This is the case for fillstyle as well as color.

You may sigh again, I have created a patch file to again consider adding color and fillstyle columns to the set mark command input, after applying 0004_EAM to the main branch of SourceForge. This modification is for the proof of concept. If possible, I would appreciate it if you could try this modified version yourself to see what kind of drawing is possible. I think it would be better to incorporate the marks style into the release version after careful consideration.

Memory footprint

What I tried to keep in mind in this implementation was not to increase the footprint of the memory occupied by mark_data. For this purpose, the mode, color, and fillstyle data are packed and stored in the member z of the 't_position' struct of 'polygon.vertex'. For details, see 'markstyle_pack()' function in 'gadget.c'.

Uniform or Mixed

A mark may consist of multiple elements. In this implementation, the case where all elements have the same 'mode', 'color', and 'fillstyle' is defined as uniform, and the case where they contain different ones is defined as mixed. This is distinguished by the 'uniform' member of 'mark_data' struct. In the case of uniform, 'mark->mark_fillstyle' and 'mark->mark_fillcolor' are used as the drawing attributes. On the other hand, in the case of mixed, 'mark->polygon.vertex[N].z' data is used as the drawing attributes. The disadvantage of this distinction is that it complicates the logic of 'do_mark()'. On the other hand, the advantage is that the output of 'save mark' command in the uniform case can be simplified to a 2-column output as in the current implementation. In the mixed case, it is a 5-column format because each element may have different 'mode', 'color', and 'fillstyle'.

Handling of border

The value of mode has the highest priority for whether or not to draw the border.
Borders are always drawn when mode=1,3, and borders are not drawn when mode=2. If mode=0, no border is drawn if 'noborder' is given to fillstyle in the plot command.

set mark <DATA> using 1:2:(0)   ### mode 0
set mark <DATA> using 1:2:(1)   ### mode 1
set mark <DATA> using 1:2:(2)   ### mode 2
set mark <DATA> using 1:2:(3)   ### mode 3

Basically, the mode is not changed by the 'fillstyle' option of the 'set mark' command. But only 'noborder' option in 'set mark ... fillstyle' makes mode value 2 to match user's intuition.

set mark <DATA> fs empty border ### mode 0 but fill is empty
set mark <DATA> fs solid border ### mode 0 but fill is solid
set mark <DATA> fs noborder     ### mode 2

Sample scripts for trying mixed mark concept are attached.

[Ethan Merritt]

Status of "marks" code in 6.1
You wrote "I think it would be better to incorporate the marks style into the release version after careful consideration". Everything in 6.1 can be considered as experimental, that is the point of having a development version that is separate from the stable version. The experimental features may change during development before being included in a release version, or reverted if they don't work out. I suppose in this era of git repositories it would be possible to split 6.1 further into branches with work-in-progress on separate projects continuing in parallel branches. That's what I do here, but I have never pushed any of my in-progress branches back to SourceForge. Would you prefer that I create a new "marks" branch and roll back the master branch of 6.1 on SourceForge to before any of the marks patches were added?

Continuing work from my side
In trying your new demos I discovered several bugs in the code I had already added. These bugs are fixed in four commits that I will push later this evening. The fifth commit below adds a separate mode value for background fill. See comments below.

00d4071d5 marks - add a fourth mode MARKS_FILL_BACKGROUND
37e73d65b marks - mark border only gets varcolor if fillstyle is FS_EMPTY
96e8b96e4 do_mark() must track fillcolor as a full colorspec rather than rgb only
dc8eb53c8 marks - add a "title" attribute
c1f15ec48 marks - fix show/save of mark fillcolor

Comments on your patch 101 and the various demos
The memory footprint of mark data is not a major issue. I am more concerned about ease-of-use and any tradeoffs between code complexity and benefits gained. From that perspective it is extremely useful to present demos that show what the new code offers that was difficult or impossible to achieve before. The windbarbs demo is a great example, and for me it is the strongest selling point so far. The Fig3+Fig4 demos are also very good because they show that I can revert the chunks of terminal-specific code I added to implement a fixed set of special point types.

• arrow.gp is cute. With the MARKS_FILL_BACKGROUND commit in place, I can duplicate this with no per-vertex fill or color

• bars.gp is also nice. This also does not require any per-vertex or color. I think you could already generate exactly these same plots without using marks at all by using multiplot with a separate inset plot for each bar chart. Nevertheless it shows a clever use of marks as an alternative to multiplot.

• ball.gp. I do not like this one. To me this seems like a mis-use of marks to try to turn gnuplot into a primitive sketching program or drawing tool. If there is a need to place objects that are more than simple shapes or symbols, it would be better to import them from a more general tool. I attach a demo that uses pixmaps to place ray-traced balls onto the sin waves rather than marks that approximate them. There is not currently a with pixmaps plot style, but it would be trivial to add one if there is demand for it.

• donut.gp With the MARKS_FILL_BACKGROUND mode in place, all of these donut shapes are easy to generate. Admittedly there is a difference from your 2nd style example as it stands because the mark is defined to have a border around the donut hole and I can't think of a way to turn that off to generate a borderless donut. But it is easy to define a borderless donut if needed; it's just that you would need a separate mark type.

So I guess I am still wanting to see examples of plots that could be constructed in gnuplot using marks, but only if the marks support additional per-node properties. Can you link to any such plots on the web or point to publications that use them?

[Hiroki Motoyoshi]

After much consideration, I have given up on introducing color and fillstyle columns, as I do not have enough material to convince you at this time. Even without introducing color and fillstyle columns, the 'marks' style would be a new feature that expands gnuplot's expression. So, please proceed with the current direction of the main branch of SourceForge. 

I have the following concerns about the current implementation of the main branch on SourceForge, and I hope you will consider them.

'fillcolor' and 'fillstyle' options in 'set mark N append' command

In the 'set mark <n> append' command, fillcolor and fillstyle cannot be changed, so these options should not be accepted (at least the warning should be shown). In the current version, these options are silently ignored.</n>

$data<<EOD
0 0
0 1
1 1
1 0
0 0
EOD

set mark 1 $data fc "red" fs solid 0.5
set mark 1 append $data using (0.5*$1):(0.5*$2) fc "blue" fs solid 0.2 # different fc and fs specified!

'title', 'fillcolor', 'fillstyle' option in 'set mark N empty'

Since 'set mark N empty' indicates an "empty mark", the 'title', 'fillcolor', 'fillstyle' options should also be accepted for this command.

set mark 1 empty title "blank"
                              ^
         unexpected or unrecognized token: title

Stroke color (mode=1) when 'noborder' is specified in both set mark and plot command

In this case, the stroke color should be determined from 'parent_lp_properties'. I've attached a patch and a test script. In the current implementation, this test script shows the 'black' borders, which should be drawn in the color implicitly specified by 'lc 1'.

set angle degrees
set mark 10 [0:360:10] "+" using (0.5*sin($1)):(0.5*cos($1)) fs solid 0.5 noborder
set mark 10 append "-" using 1:2:(1)
0 -1
0 1

-1 0
1 0
e
unset angle

plot x with marks mt 10 pn 10 ps 3  fs noborder

About background color setting for MARKS_FILL_BACKGROUND

In the current implementation, the background color for MARKS_FILL_BACKGROUND is chosen as the background color specified in the terminal setting. This is fine as the default, but is it possible to change the background color only for drawing marks by adding the option 'background' in the 'plot with marks'? I think it is more common to use 'set obj rect' to fill only the graph area, as in Figs. 1 and 2, rather than to fill the background color for all the canvas.

Utility command like "show mark N $datablock"

The "save marks" command outputs all mark data to be loaded again. However, there are cases where you may want to reuse the individual mark data. Is there an alternative syntax to the previously discussed "show mark N $datablock"?

Array of complex numbers for Input to 'set mark'

In a previous patch, I suggested the following to handle the case where only an Array is given for set mark, is this the correct approach?

ierr = df_open(name_str, 4, NULL);
if (ierr < 0)
        int_error(NO_CARET, "could not open %s", name_str);
if (ierr == 1)
        int_error(saved_token, "Need more than 1 input data column");

/* Special treatment of Array input without any data modifications */
if (df_array && df_no_use_specs == 0) {
    df_close();
    return read_mark_data_from_array(df_array);
}

[Hiroki Motoyoshi]

Another concerns about the current implementation of the main branch on SourceForge.

** border color selection for 'fc variable' **

If the 'fc variable' is specified and the border color is not, the border line is drawn as a black line. It should be matched with the variable fillcolor.

The patch is attached, and the test script is here.

$box << EOD
0 0
0 1
1 1
1 0
0 0
EOD

set mark 1 $box 

$data << EOD
0 0 
0 1 
1 0 
1 1 
EOD

set xrange [-1:3]
set yrange [-1:3]

plot $data using 1:2:($0+1) with marks mt 1 units xy fs solid 0.5 border fc variable

[Hiroki Motoyoshi]

Patch for fixes to 'set_mark()'

This patch contains the following Fixes and Modifications

1. [Fix] Correct rejection of pseudo file "++"
2. [Modification] 'set mark N empty' should accept 'title', 'fc', 'fs' options
3. [Modification] 'set mark N append' should not accept 'title', 'fc', 'fs' options (raises error)
4. [Modification] 'append' for undefined mark tag should be an error to avoid confusing
5. [Modification] macro variable MARK_ACTION_UNDEFINED is removed by refactoring

With this patch, the syntax of the "set mark" command is summarized as follows,

    set mark {<data-spec>|empty} title <string> fc <color-spec> fs <fillstyle-spec>
    set mark append <data-spec>

Thus, with 'append', only data (including 'mode' column) can be added.

The test script 'test_set_mark.gp' is also attached.
Please invoke it as follows,

% gnuplot < test_set_mark.gp

[Hiroki Motoyoshi]

Thank you for refactoring set_mark().

Here are two small patches, after your commit.

---

204_disable_append_in_first_set_mark.patch

More fixes to set_mark. Includes the following.

• [Fix] set mark 1 append $data when none of the marks are defined is not acceptable
• [Refactoring] Eliminate MARK_ACTION_CREATE and MARK_ACTION_APPEND macro variables

---

205_modify_default_fillstyle_of_object_mark.patch

The default fillstyle of "mark" object was "fs solid 1", but it should be changed to "fs empty" as well as "plot with marks".

[Ethan Merritt]

Re: 205_modify_default_fillstyle_of_object_mark.patch

Are you sure? This change would mean that you cannot modify the fillstyle of an object that contains a mark. I think the current initialization to FS_DEFAULT is correct. That tells it to inherit the fillstyle of the object that contains it.

[Hiroki Motoyoshi]

Are you sure? This change would mean that you cannot modify the fillstyle of an object that contains a mark. I think the current initialization to FS_DEFAULT is correct. That tells it to inherit the fillstyle of the object that contains it.

Yes, I'm sure. The 205 patch simply modifies the object mark's default fillstyle. My test code is like this,

$box <<EOD
0 0
0 1
1 1
1 0
0 0 
EOD

set mark 1 $box using ($1-0.5):($2-0.5)
set mark 1 append $box

set xrange [-1:2]
set yrange [-1:2]

set label at 0,1.2 "obj mark" center
set label at 1,1.2 "obj mark fs transparent solid 0.5" center

set label at 0,0.2 "plot with marks" center
set label at 1,0.2 "plot with marks fs transparent solid 0.5" center

set obj 1 mark mt 1 at 0,1 scale 2
set obj 2 mark mt 1 at 1,1 fs transparent solid 0.5 scale 2

array dummy[1]
plot dummy using (0):(0) with marks mt 1 ps 2 , \
     dummy using (1):(0) with marks mt 1 fs transparent solid ps 2, \

Before 205 patch, the upper left mark appears to be a superposition of opaque squares. After 205 patch, the upper left and lower left marks will be the same shape and will be overlapping transparent squares. This is due to the fact that the default value of fillstyle for object mark has been changed to 'fs empty border'.

The output of your test code appeared to be the same before and after the 205 patch.

The fillstyle setting in 'set object mark' is the parent's fillstyle in 'do_mark()'. The role of the parent's fillstyle in 'do_mark()' is the fallback for the mark data setting. The fallback should be fixed. The fillstyle's fallback value is 'fillstyle empty border', isn't it? Other object types (circles, polygons ...) also have the default fillstyle of 'fillstyle empty border'

The attached PDF file is a summary of my rules for filling polygon and drawing border. If these rules are not what you have in mind, we may need to change the current implimentations.

[Ethan Merritt]

You are right. I see it now. Thank you for pointing out my mistake. I mis-interpreted the patch as changing the default for the mark itself, not for the object.

[Hiroki Motoyoshi]

Thank you for confirming about the 205 patch.

mark's border drawing rule

Deciding whether or not to draw a border is a bit more complicated, but it is possible to establish a consistent rule that 'mode', 'mark border style', and 'parent border style' have priority in that order, as shown in 'DrawModeRules.pdf'.

Your commit “5af9157bcc6c85a91564464b08e090f86888c280” gives top priority to the 'noborder' directive in 'parent border style', which conflicts with the rule proposed here. If you agree with the rules I proposed here, please consider the attached patch "206_border_style_drawing_rule_consistensy.patch". The test script is also attached.

I think, if one wants to use marks that obeys the border style from 'plot' or 'set object', one should just define them with mode=0.

EDIT

In "206_border_style_drawing_rule_consistensy.patch", I assumes the border colors are determined by rules in the attached file 'ColoringRules.pdf'.

[Ethan Merritt]

Here is a proof-of-principle patchset that adds a "title" attribute to the "set mark" command. The program doesn't do anything with the title other than keep it attached to the mark and report it as part of a "show" or "save" command.
Example:

    set angles degree
    set mark 8 [0:360:10] "+" using (cos($1)):(0.8*(sqrt(abs(cos($1)))+sin($1))):(FILL) \
           fc "dark-red" title "red heart"
    unset angle
    show mark 8
       marktype 8  title "red heart" polygon vertices 37 fillcolor  rgb "red" fillstyle default

[Hiroki Motoyoshi]

Thank you for the patch.

This feature gives us a hint as to the shape, which we can't know from the number alone.

[Ethan Merritt]

Could you give me an example of what you mean by a "transparent hole" in a set of polygons? I don't understand how that relates to colors. How is it different from an empty (no fill) facet in a larger surface?

For example, it doesn't seem possible to drill a hole in a polygon with another polygon in a simple way like fill-rule in SVG.

Ah. I understand. You want control of the "winding rule" property. That is a good idea! It needs to be implemented at the level of the terminal settings, after which you would get this "for free" in the polygon and marks code. So far as I know, up until now no one has tried to make all the output terminals consistent. The default settings in the various support libraries are not the same.

• cairo
  cairo_set_fill_rule (*cr, CAIRO_FILL_RULE_WINDING | CAIRO_FILL_RULE_EVEN_ODD);
  CAIRO_FILL_RULE_WINDING is the default
• svg
  fill-rule=nonzero (default) or fill-rule=evenodd
• Qt
  drawPolygon(points, pointcount, Qt::FillRule fillRule = Qt::OddEvenFill) (default)
  drawPolygon(points, pointcount, Qt::FillRule fillRule = Qt::WindingFill)
• PostScript
  default is "nonzero"; "evenodd" can be achieved by using operators
  eoclip and eofill rather than clip and fill
• x11
  The Xlib manual claims the default is EvenOddRule in XGCValues.
  However the observed behavior is equivalent to "nonzero".

The attached script should show which rule is in effect; you can easily see the difference depending on what the terminal is chosen.

Modifying the terminal drivers to select "even/odd" if possible would give you the hole-drilling behavior you want. That will change the behavior of many terminals, which is unfortunate. I wish this had come up a year ago, since it is exactly the sort of change that would have been appropriate for version 6.

Meanwhile, the Qt and gdlib terminals already default to an "even/odd" winding rule. You can use those to explore hole-containing marks.

[Hiroki Motoyoshi]

Thanks for the information on the Fill-Rule for each terminal. It is very helpful.

I understand that it is possible to paint with a single stroke polygon, like the star shape in your example.

Apparently, the Fill-Rule setting is not the only essential thing for drawing holes. For example, whether the Fill-Rule is nonzero or even-odd, we should be able to represent a hole by drawing the outer polygon clockwise and the inner polygon counter clockwise. If we call the inner polygon a subpath of the outer polygon, gnuplot does not have the ability to handle subpaths. Or it may be said that there is no directive in the data format for 'polygons' style to indicate whether subpath or separate path the polygon to be drawn is.

As shown below, simply giving two polygons in succession will only render each as a separate polygon, not a hole. However, if all the second and subsequent polygons are recognized as subpaths, it is not possible to give each polygon a variable color (a dilemma?).

# outer rectangle (clock wise)
$rect_cw <<EOD
-1 1
1 1
1 -1
-1 -1
-1 1
EOD

# inner diamond (counter-clock wise)
$diamond <<EOD
0 0.5
-0.5 0
0 -0.5
0.5 0
0 0.5
EOD

set print $polygon_with_hole
print $rect
print $diamond
unset print

print $polygon_with_hole

set xrange [-2:2]
set yrange [-2:2]
set isotropic

plot $polygon_with_hole with polygons fillstyle solid

If you add a line that bridges the outer and inner polygons, you can simulate a hole by winding rule.

$polygon_with_hole <<EOD
-1 1
1 1
1 -1
-1 -1
-1 1
0 0.5
-0.5 0
0 -0.5
0.5 0
0 0.5
-1 1
EOD

set xrange [-2:2]
set yrange [-2:2]
set isotropic
unset key

plot $polygon_with_hole with polygons fillstyle solid 0.5, \
     xo=yo=NaN $polygon_with_hole using (xo):(yo):(dx=$1-xo,xo=$1,dx):(dy=$2-yo,yo=$2,dy) with vectors lw 3 lc "black", \

[Ethan Merritt]

I don't know if there is a consistent concept of "subpath" shared by the various support libraries. I am pretty sure there is no such thing in PostScript. gdlib doesn't really have paths at all, at least with regard to filled area. I get no hits on "subpath" from searching the cairographics documetation (which doesn't mean there is no such thing, but I didn't find it).

I tried to follow the definitions in the SVG documentation you linked to, and ended up with the following hand-modified svg file that consists of the "star.dem" output with an added rectangle inside the star. This is accomplished using the SVG command path d=<data> rather than what gnuplot uses now: polygon points=<data>. That works to generate a hole, but I don't see any way to change color since the only allowable content in <data> is move and line commands. So I'm not sure it gets you what you want. So far as I understand the documetation, "subpaths" are not possible using the polygon shape definition.

I think Qt works approximately the same was as the SVG path + <data> variant I showed above. You can string lists of points together into a single "path", and if there are any intervening "move" operations then the path is implicitly split into subpaths. The Qt output stream is obviously not hand-editable, and I didn't try to modify the program to test this. It's much easier to experiment with SVG.

This may be wandering off topic. I do think it is a good idea to make all gnuplot terminals consistent with regard to the winding attribute, but that may not actually achieve what you want. Do you have a real-world example of a polygon shape with holes in it that you want to draw? Is the trick of connecting interior "hole" shapes to the overall perimeter via a zero-width line (as in your example) sufficient?

[Hiroki Motoyoshi]

This may be wandering off topic. I do think it is a good idea to make all gnuplot terminals consistent with regard to the winding attribute, but that may not actually achieve what you want. Do you have a real-world example of a polygon shape with holes in it that you want to draw? Is the trick of connecting interior "hole" shapes to the overall perimeter via a zero-width line (as in your example) sufficient?

Marks does not yet have an actual example, but a 'with polygons' style example would be the filling of a map such as a world map that includes lakes and enclaves. Recently, many GIS polygon data can be available with the format GeoJSON, ESRI Shapefile which have multipolygon type consist of an outer polygon and some inner polygons.

If you look at fonts as polygons, they still have many holes (This is just an idea, but if the Latex formula could be represented by a path, it might be possible to load it into 'set mark' and place it wherever you want). In such cases, it is difficult to manually insert a zero width line with numerous holes or with complex polygon data.

Marks assumes something simpler, so the case like 10th in Fig. 11 is probably more usual. Such a simplest case I showed, the trick is sufficient.

[Hiroki Motoyoshi]

I would like to thank you for reorganizing the source code related to marks in the 'commit '6f281ed23ccbde5300aa7b598215f53a929e1a7a' and for including the manual and demo in subsequent commits.

I was worried that the marks-related functions were scattered across several source files, so this is a great help.

[Ethan Merritt]

Sorry for the long delay. I was focused on revisions and bug fixes for version 6.

Question: Each source file has a copyright notice and license info at the top. I put only the gnuplot license there since you had not indicated otherwise. Would you like me to add also the extra text to allow use under the BSD or GPL licenses?

[Hiroki Motoyoshi]

Thank you for your comment.

This is a file that brings together functions that were originally scattered in various places, such as graphics.c, so I think it is reasonable that it has the same license as those files.

Furthermore, the functions in marks.c rely heavily on gnuplot routines, so it is not particularly useful to use them in programs other than gnuplot, so there is no positive reason to add BSD or GPL licenses.