ZedGraph / Discussion / Help: Calculating graph pixel position

The control shows all the points of the graph even if some points fall on the same pixel on the screen. This make graphs with large amount of points (~100k) to be repainted painfully slow.
It would be great for the next version to discard these points (is it possible).
Just taking every Nth point of the graph cannot be done since the curve should show all the anomalities. This is the reason I cannot use average or other function do decrease number of points.
I am trying to dilute the graph by manually removing all the points that are duplicated on the screen from list of points:

private IPointList DiVect(IPointList plist, GraphPane pane, ZedGraphControl zgc)
{
byte[,] bVals = null;

/* get the axis rectangle */
RectangleF rectf = pane.CalcChartRect(zgc.CreateGraphics());

/* returned point pair list */
PointPairList ppl = new PointPairList();

bVals = new byte[(int)rectf.Width + 1, (int)rectf.Height + 1];
for (int i = 0; i &lt; plist.Count; i++)
{
    PointF pf = pane.GeneralTransform(
        new PointF((float)plist[i].X, (float)plist[i].Y),
        CoordType.AxisXYScale);

    if (pf.X &gt;= 0 &amp;&amp; pf.Y &gt;= 0 &amp;&amp;
        ((int)pf.X) &lt; bVals.GetLength(0) &amp;&amp;
        ((int)pf.Y) &lt; bVals.GetLength(1) &amp;&amp;
        bVals[(int)pf.X, (int)pf.Y] == 0)
    {
        ppl.Add(new PointPair(plist[i]));
        bVals[(int)pf.X, (int)pf.Y] = 1;
    }
}

return ppl;

}

This simple function makes second buffer with size equal to the size of drawn pane area and goes over the list of points of the curve. Each point coordinates are transformed into pixel coordinates and if this point is already on the screen, it is thrown from the list.
The only problem, GeneralTransform always returns me (0, 0).
What am I doing wrong with transformation so it does not work?
I am interested in hearing other suggestions for the solution (maybe, seeing dilute feature in v5 or ZedGraph, ah, dreams!).

John Champion - 2006-05-18

Hi,
I got to thinking about this...a number of people have been looking at filtering methods to reduce the point count for very large datasets. Your method of looking at the actual pixel location for overlaps seems like a good approach. So, to this end I went ahead and setup an IPointList based on "nearest pixel" logic. You can filter points that overlap exactly, or points that lie within a specified number of pixels of an existing point.

This is called NoDupePointList. To use it, you would do something like this:

NoDupePointList list = new NoDupePointList();
Loop to add 100000 points
{
list.Add( x, y );
}

LineItem curve = myPane.AddCurve( "title", list, ...);
curve.Line.IsVisible = false;

myPane.AxisChange();

// this is the number of pixels that would be
// considered close enough to overlap.
// 0 means they must overlap exactly
// 1 means within 1 pixel, etc.
list.FilterMode = 0;
// This does the actual filtering. You should
// call this method anytime your data change,
// the graph is resized, etc. in order to
// redo the filtering based on the current
// chart configuration.
list.FilterData( myPane, myPane.YAxis );

You can grab the NoDupePointList.cs file from the cvs under the V5_BRANCH.
John

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Jonathan Mist - 2006-06-22
  
  Hi,
  
  I am trying to use this (from v4.9.8) in a master pane layout with little success :-(
  
  I can add the first curve and filter it, but when I try to add more curves to the same pane, the pixel scaling does not work. The y value is always larger than the height so all points are filtered out.
  
  Before I call FilterData() I call AxisChange() to auto-scale the Y axis based on the full data set. Do I need to call anything else to make sure the scaling will work?
  
  Also, I added a check for points that contain NaN as I need to make sure these are not filtered out (so that discontinuous lines can be plotted).
  
  FWIW I also tried the FilteredPointList but I cannot get that to work - no points are shown at all unless I disable filtering with -1 in SetBounds. I've seen someone else with this problem in the forum.
  
  Cheers,
  
  Jonathan
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
John Champion - 2006-05-17

Hi,
You probably just need to call AxisChange() first, so that the scale ranges will be setup.

Another way to set this up would be to create an IPointList type that handles it automatically. In this manner, you could have the IPointList adjust itself if the graph is resized. (see the FilteredPointList and BasicArrayPointList for examples).

John

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous - 2006-05-22

The only thing I can say:
WOW! Thanks!
Going to test it now!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous - 2006-05-23

Is it possible to put V5_BRANCH somewhere for download via HTTP standard ports? I have no access to cvs ports of SourceForge.
Tonight I ended up with 740MB downloaded from CVS's web interface using "wget -r" and nothing that is usable... :(

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- John Champion - 2006-05-23
  
  Hi,
  You can download version 4.9.7 from the developer releases on sourceforge. I think the CVS browser is actually whacked, since it has no recent files.
  John
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

4.9.7 from devloper releases has no NoDupePointList class. :(
Or am I looking at wrong position?
http://sourceforge.net/project/showfiles.php?group_id=114675&package_id=162846&release_id=417180

Hi,
Sorry about that...I've just included the file here:
John

//============================================================================
//ZedGraph Class Library - A Flexible Line Graph/Bar Graph Library in C#
//Copyright © 2006 John Champion
//
//This library is free software; you can redistribute it and/or
//modify it under the terms of the GNU Lesser General Public
//License as published by the Free Software Foundation; either
//version 2.1 of the License, or (at your option) any later version.
//
//This library is distributed in the hope that it will be useful,
//but WITHOUT ANY WARRANTY; without even the implied warranty of
//MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
//Lesser General Public License for more details.
//
//You should have received a copy of the GNU Lesser General Public
//License along with this library; if not, write to the Free Software
//Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
//=============================================================================

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;

namespace ZedGraph
{
/// <summary>
/// A simple storage struct to maintain an individual sampling of data. This only
/// contains two data values in order to reduce to memory load for large datasets.
/// (e.g., no Tag or Z property)
/// </summary>
public struct DataPoint
{
public double X;
public double Y;
}

/// <summary>
/// A collection class to maintain a set of samples.
/// </summary>
/// <remarks>This type, intended for very
/// large datasets, will reduce the number of points displayed by eliminating
/// individual points that overlay (at the same pixel location) on the graph.
/// Note that this type probably does not make sense for line plots, but is intended
/// primarily for scatter plots.
/// </remarks>
///
/// <author> John Champion </author>
/// <version> $Revision: 1.1.2.1 $ $Date: 2006/05/19 05:11:32 $ </version>
[Serializable]
public class NoDupePointList : List<DataPoint>, IPointList, IPointListEdit
{
protected bool _isFiltered;
protected int _filteredCount;
protected int[] _visibleIndicies;

  protected int _filterMode;


  /// &lt;summary&gt;
  /// Gets or sets a value that determines how close a point must be to a prior
  /// neighbor in order to be filtered out.
  /// &lt;/summary&gt;
  /// &lt;remarks&gt;
  /// A value of 0 indicates that subsequent
  /// points must coincide exactly at the same pixel location.  A value of 1 or more
  /// indicates that number of pixels distance from a prior point that will cause
  /// a new point to be filtered out.  For example, a value of 2 means that, once
  /// a particular pixel location is taken, any subsequent point that lies within 2
  /// pixels of that location will be filtered out.
  /// &lt;/remarks&gt;
  public int FilterMode
  {
     get { return _filterMode; }
     set { _filterMode = value; }
  }

  /// &lt;summary&gt;
  /// Gets a value indicated whether or not the data have been filtered.  If the data
  /// have not been filtered, then &lt;see cref=&quot;Count&quot; /&gt; will be equal to
  /// &lt;see cref=&quot;TotalCount&quot; /&gt;.
  /// &lt;/summary&gt;
  public bool IsFiltered
  {
     get { return _isFiltered; }
  }

  /// &lt;summary&gt;
  /// Indexer: get the DataPoint instance at the specified ordinal position in the list
  /// &lt;/summary&gt;
  /// &lt;remarks&gt;
  /// This method will throw an exception if the index is out of range.  This can happen
  /// if the index is less than the number of filtered values, or if data points are
  /// removed from a filtered dataset with updating the filter (by calling
  /// &lt;see cref=&quot;FilterData&quot; /&gt;).
  /// &lt;/remarks&gt;
  /// &lt;param name=&quot;index&quot;&gt;The ordinal position in the list of points&lt;/param&gt;
  /// &lt;returns&gt;Returns a &lt;see cref=&quot;PointPair&quot; /&gt; instance.  The &lt;see cref=&quot;PointPair.Z&quot; /&gt;
  /// and &lt;see cref=&quot;PointPair.Tag&quot; /&gt; properties will be defaulted to
  /// &lt;see cref=&quot;PointPair.Missing&quot; /&gt; and null, respectively.
  /// &lt;/returns&gt;
  public new PointPair this[int index]
  {
     get
     {
        int j = index;
        if ( _isFiltered )
           j = _visibleIndicies[index];

        DataPoint dp = base[j];
        PointPair pt = new PointPair( dp.X, dp.Y );
        return pt;
     }
     set
     {
        int j = index;
        if ( _isFiltered )
           j = _visibleIndicies[index];

        DataPoint dp;
        dp.X = value.X;
        dp.Y = value.Y;
        base[j] = dp;
     }
  }

  /// &lt;summary&gt;
  /// Gets the number of active samples in the collection.  This is the number of
  /// samples that are non-duplicates.  See the &lt;see cref=&quot;TotalCount&quot; /&gt; property
  /// to get the total number of samples in the list.
  /// &lt;/summary&gt;
  public new int Count
  {
     get
     {
        if ( !_isFiltered )
           return base.Count;
        else
           return _filteredCount;
     }
  }

  /// &lt;summary&gt;
  /// Gets the total number of samples in the collection.  See the &lt;see cref=&quot;Count&quot; /&gt;
  /// property to get the number of active (non-duplicate) samples in the list.
  /// &lt;/summary&gt;
  public int TotalCount
  {
     get { return base.Count; }
  }

  /// &lt;summary&gt;
  /// Append a data point to the collection
  /// &lt;/summary&gt;
  /// &lt;param name=&quot;sample&quot;&gt;The DataPoint to append&lt;/param&gt;
  public void Add( PointPair pt )
  {
     DataPoint dp = new DataPoint();
     dp.X = pt.X;
     dp.Y = pt.Y;
     Add( dp );
  }


  /// &lt;summary&gt;
  /// Append a point to the collection
  /// &lt;/summary&gt;
  /// &lt;param name=&quot;x&quot;&gt;The x value of the point to append&lt;/param&gt;
  /// &lt;param name=&quot;y&quot;&gt;The y value of the point to append&lt;/param&gt;
  public void Add( double x, double y )
  {
     DataPoint dp = new DataPoint();
     dp.X = x;
     dp.Y = y;
     Add( dp );
  }


  // generic Clone: just call the typesafe version
  object ICloneable.Clone()
  {
     return this.Clone();
  }

  /// &lt;summary&gt;
  /// typesafe clone method
  /// &lt;/summary&gt;
  /// &lt;returns&gt;A new cloned NoDupePointList.  This returns a copy of the structure,
  /// but it does not duplicate the data (it just keeps a reference to the original)
  /// &lt;/returns&gt;
  public NoDupePointList Clone()
  {
     return new NoDupePointList( this );
  }

  /// &lt;summary&gt;
  /// default constructor
  /// &lt;/summary&gt;
  public NoDupePointList()
  {
     _isFiltered = false;
     _filteredCount = 0;
     _visibleIndicies = null;
     _filterMode = 0;
  }

  /// &lt;summary&gt;
  /// copy constructor -- this returns a copy of the structure,
  /// but it does not duplicate the data (it just keeps a reference to the original)
  /// &lt;/summary&gt;
  /// &lt;param name=&quot;rhs&quot;&gt;The NoDupePointList to be copied&lt;/param&gt;
  public NoDupePointList( NoDupePointList rhs )
  {
     int count = rhs.TotalCount;
     for ( int i = 0; i &lt; count; i++ )
        Add( rhs.GetDataPointAt( i ) );

     _filteredCount = rhs._filteredCount;
     _isFiltered = rhs._isFiltered;
     _filterMode = rhs._filterMode;

     if ( rhs._visibleIndicies != null )
        _visibleIndicies = (int[]) rhs._visibleIndicies.Clone();
     else
        _visibleIndicies = null;
  }

  /// &lt;summary&gt;
  /// Protected method to access the internal DataPoint collection, without any
  /// translation to a PointPair.
  /// &lt;/summary&gt;
  /// &lt;param name=&quot;index&quot;&gt;The ordinal position of the DataPoint of interest&lt;/param&gt;
  protected DataPoint GetDataPointAt( int index )
  {
     return base[index];
  }

  /// &lt;summary&gt;
  /// Clears any filtering previously done by a call to &lt;see cref=&quot;FilterData&quot; /&gt;.
  /// After calling this method, all data points will be visible, and
  /// &lt;see cref=&quot;Count&quot; /&gt; will be equal to &lt;see cref=&quot;TotalCount&quot; /&gt;.
  /// &lt;/summary&gt;
  public void ClearFilter()
  {
     _isFiltered = false;
     _filteredCount = 0;
  }

  /// &lt;summary&gt;
  /// Go through the collection, and hide (filter out) any points that fall on the
  /// same pixel location as a previously included point.
  /// &lt;/summary&gt;
  /// &lt;remarks&gt;
  /// This method does not delete any points, it just temporarily hides them until
  /// the next call to &lt;see cref=&quot;FilterData&quot; /&gt; or &lt;see cref=&quot;ClearFilter&quot; /&gt;.
  /// You should call &lt;see cref=&quot;FilterData&quot; /&gt; once your collection of points has
  /// been constructed.  You may need to call &lt;see cref=&quot;FilterData&quot; /&gt; again if
  /// you add points, or if the chart rect changes size (by resizing, printing,
  /// image save, etc.), or if the scale range changes.
  /// You must call &lt;see cref=&quot;AxisChange&quot; /&gt; before calling
  /// this method so that the &lt;see cref=&quot;Chart.Rect&quot;&gt;GraphPane.Chart.Rect&lt;/see&gt;
  /// and the scale ranges are valid.  This method is not valid for
  /// ordinal axes (but ordinal axes don't make sense for very large datasets
  /// anyway).
  /// &lt;/remarks&gt;
  /// &lt;param name=&quot;pane&quot;&gt;The &lt;see cref=&quot;GraphPane&quot; /&gt; into which the data
  /// will be plotted. &lt;/param&gt;
  /// &lt;param name=&quot;yAxis&quot;&gt;The &lt;see cref=&quot;Axis&quot; /&gt; class to be used in the Y direction
  /// for plotting these data.  This can be a &lt;see cref=&quot;YAxis&quot; /&gt; or a 
  /// &lt;see cref=&quot;Y2Axis&quot; /&gt;, and can be a primary or secondary axis (if multiple Y or Y2
  /// axes are being used).
  /// &lt;/param&gt;
  public void FilterData( GraphPane pane, Axis yAxis )
  {
     if ( _visibleIndicies == null || _visibleIndicies.Length &lt; base.Count )
        _visibleIndicies = new int[base.Count];

     _filteredCount = 0;
     _isFiltered = true;

     int width = (int)pane.Chart.Rect.Width;
     int height = (int)pane.Chart.Rect.Height;
     if ( width &lt;= 0 || height &lt;= 0 )
        throw new IndexOutOfRangeException( &quot;Error in FilterData: Chart rect not valid&quot; );

     bool[,] usedArray = new bool[width, height];
     for ( int i = 0; i &lt; width; i++ )
        for ( int j = 0; j &lt; height; j++ )
           usedArray[i, j] = false;

     Axis xAxis = pane.XAxis;
     xAxis.Scale.SetupScaleData( pane, pane.XAxis );
     yAxis.Scale.SetupScaleData( pane, yAxis );

     int n = _filterMode &lt; 0 ? 0 : _filterMode;

     for ( int i=0; i&lt;base.Count; i++ )
     {
        DataPoint dp = base[i];
        int x = (int)( xAxis.Scale.Transform( dp.X ) + 0.5 );
        int y = (int)( yAxis.Scale.Transform( dp.Y ) + 0.5 );

        if ( x &gt;= 0 &amp;&amp; x &lt; width &amp;&amp; y &gt;= 0 &amp;&amp; y &lt; height )
        {
           bool used = false;
           if ( n &lt;= 0 )
              used = usedArray[x, y];
           else
           {
              for ( int ix = x - n; ix &lt;= x + n; ix++ )
                 for ( int iy = y - n; iy &lt;= y + n; iy++ )
                    used |= ( ix &gt;= 0 &amp;&amp; ix &lt; width &amp;&amp; iy &gt;= 0 &amp;&amp; iy &lt; height &amp;&amp; usedArray[ix, iy] );
           }

           if ( !used )
           {
              usedArray[x, y] = true;
              _visibleIndicies[_filteredCount] = i;
              _filteredCount++;
           }
        }
     }
  }

}
}

Jonathon,
Are you sure that there is an independent instance of a NoDupePointList for each curve? That is,

NoDupePointList ndp1 = new NoDupePointList();
NoDupePointList ndp2 = new NoDupePointList();
NoDupePointList ndp3 = new NoDupePointList();

// Fill in ndp lists here

LineItem curve1 = myPane.AddCurve( "a", ndp1, ... );
LineItem curve2 = myPane.AddCurve( "b", ndp2, ... );
LineItem curve3 = myPane.AddCurve( "c", ndp3, ... );

AxisChange();
ndp1.FilterData(...);
ndp2.FilterData(...);
ndp3.FilterData(...);

John

Hi,

Yes they are all independant lists. The problem is the scaling or transforming.

pane.Chart.Rect.Height returns e.g. 89

then

yAxis.Scale.Transform always returns y > height.

I am assuming there is an error in SetupScaleData() or Transform() or that I need to call something other than just AxisChange() to get the axes set correctly. I haven't been able to make sense of the transform code yet to see if I can work out what is wrong.

My sequence is actually different to yours because I don't know when each curve will be added...

AddCurve()
{
NoDupePointList ndp1 = new NoDupePointList();

// Fill in ndp lists here

LineItem curve1 = myPane.AddCurve( "a", ndp1, ... );

//I've tried calling ClearFilter() here

AxisChange();

//I've tried calling control.Refresh() here

for(i = 0; i < myPane.CurveList.Count; i++)
{
myPane.CurveList[i].FilterData(...);
}

control.Refresh();
}

Regards,

Jonathan

I created a test file to demonstrate the problem. It seems to be related to the use of MasterPane, so I guess the pixel position code in NoDupePointList does not take this into account correctly.

NB I add the ZedGraph control as the only control on the form and set to DockStyle.Fill.

Jonathan

using System;
using System.Drawing;
using System.Windows.Forms;
using ZedGraph;

namespace NoDupePointListTest
{
public partial class Form1 : Form
{
const int NUMPOINTS = 10000;
public Form1()
{
InitializeComponent();

        zedGraphControl1.GraphPane.CurveList.Clear();
        zedGraphControl1.MasterPane.SetLayout(PaneLayout.SingleColumn);

        //These curves will be missing the last few points
        AddCurve(0, 0);
        AddCurve(0, 1);

        //These curves will be missing all points
        AddCurve(1, 0);
        AddCurve(1, 1);

        //If you comment out these two lines, the curves on pane 0 will look ok
        UpdateAxes();
        FilterAllCurves();

        zedGraphControl1.Refresh();
    }

    private void AddCurve(int pane, int curve)
    {
        if (pane &gt; zedGraphControl1.MasterPane.PaneList.Count)
        {
            throw (new System.Exception(&quot;You can only add a pane to the end of the list&quot;));
        }
        int i;
        if (pane &gt;= zedGraphControl1.MasterPane.PaneList.Count)
        {
            GraphPane pane1 = new GraphPane();
            zedGraphControl1.MasterPane.PaneList.Add(pane1);

            //Is this correct for the new layout?
            Graphics g = zedGraphControl1.CreateGraphics();
            zedGraphControl1.MasterPane.DoLayout(g);
            g.Dispose();

            //New pane layout to refresh all the axes and filtered data
            UpdateAxes();
            FilterAllCurves();
        }

        //Create the data
        NoDupePointList ndp1 = new NoDupePointList();
        for (i = 0; i &lt; NUMPOINTS; i++)
        {
            if ((curve &amp; 1) == 0)
                ndp1.Add(i, -(NUMPOINTS / 2) + i);
            else
                ndp1.Add(i, (NUMPOINTS / 2) - i);
        }

        //Create the curve
        zedGraphControl1.MasterPane.PaneList[pane].AddCurve(&quot;ndp&quot;+curve.ToString(), ndp1, Color.Red);

        //New curve so update the axes and filtering for this pane
        UpdateAxes();
        FilterCurves(pane);
    }

    private void FilterCurves(int pane)
    {
        int i;
        for (i = 0; i &lt; zedGraphControl1.MasterPane.PaneList[pane].CurveList.Count; i++)
        {
            NoDupePointList ndp = zedGraphControl1.MasterPane.PaneList[pane].CurveList[i].Points as NoDupePointList;
            if (ndp != null)
            {
                //If you comment this out you will see what it is supposed to look like
                ndp.FilterData(zedGraphControl1.MasterPane.PaneList[pane], zedGraphControl1.MasterPane.PaneList[pane].YAxis);
            }
        }
    }

    private void FilterAllCurves()
    {
        for (int pane = 0; pane &lt; zedGraphControl1.MasterPane.PaneList.Count; pane++)
        {
            FilterCurves(pane);
        }
    }

    private void UpdateAxes()
    {
        //Clear the filtering so that AxisChange has all the data to work with
        int curve, pane;
        for (pane = 0; pane &lt; zedGraphControl1.MasterPane.PaneList.Count; pane++)
        {
            for (curve = 0; curve &lt; zedGraphControl1.MasterPane.PaneList[pane].CurveList.Count; curve++)
            {
                NoDupePointList ndp = zedGraphControl1.MasterPane.PaneList[pane].CurveList[curve].Points as NoDupePointList;
                if (ndp != null)
                {
                    ndp.ClearFilter();
                }
            }
        }
        //Calculate new axes
        zedGraphControl1.AxisChange();
    }

    private void Form1_ResizeEnd(object sender, EventArgs e)
    {
        UpdateAxes();
        FilterAllCurves();
        zedGraphControl1.Refresh();
    }
}

}

John Champion - 2006-06-23

Jonathon,
I think I found it. It's a bust in the logic for the NoDupePointList. The Filter() method is expecting pixel values that go from 0 to width and 0 to height, but the actual pixel values have a non-zero origin. The code below is a quick fix. Can you give this a try? It's just a matter of adding the left & top values to the two Transform() calculations.
John

int left = (int)pane.Chart.Rect.Left;
int top = (int)pane.Chart.Rect.Top;

for ( int i=0; i<base.Count; i++ )
{
DataPoint dp = base[i];
int x = (int)( xAxis.Scale.Transform( dp.X ) + 0.5 ) - left;
int y = (int)( yAxis.Scale.Transform( dp.Y ) + 0.5 ) - top;

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Jonathan Mist - 2006-06-26
  
  Hi,
  
  Thanks, I tried it in v5.0.0 and it now works.
  
  :-)
  
  Jonathan
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Calculating graph pixel position

Forums

Help

Calculating graph pixel position document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

}

Calculating graph pixel position